9ff6a0ecc0
Despite the name, the crawler gets info for more services than just DNS:
- DNS:
- all A/AAAA records (for the 2nd level domain and www.subdomain), annotated
with GeoIP
- TXT records (with SPF and DMARC parsed for easier filtering)
- TLSA (for the 2nd level domain and www.subdomain)
- MX
- DNSSEC validation
- nameservers:
- each server IP annotated with GeoIP
- HOSTNAME.BIND, VERSION.BIND, AUTHORS.BIND and fortune (also for all IPs)
- users can add custom additional RRs in the config file
- E-mail (for every server from MX):
- SMTP server banners (optional, ports are configurable)
- TLSA records
- Web:
- HTTP status & headers (inc. parsed cookies) for ports 80 & 443 on each IP
from A/AAAA records
- certificate info for HTTPS (optionally with an entire cert chain)
- webpage content (optional)
- everything of the above is saved for each step in the redirect history --
the crawler follows redirects until it gets a non-redirecting status or hits
a configurable limit
WWW: https://gitlab.nic.cz/adam/dns-crawler
39 lines
1.8 KiB
Makefile
39 lines
1.8 KiB
Makefile
# Created by: Po-Chuan Hsieh <sunpoet@FreeBSD.org>
|
|
|
|
PORTNAME= dns-crawler
|
|
PORTVERSION= 1.4.9
|
|
CATEGORIES= dns python
|
|
MASTER_SITES= CHEESESHOP
|
|
PKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX}
|
|
|
|
MAINTAINER= sunpoet@FreeBSD.org
|
|
COMMENT= Crawler for getting info about DNS domains and services
|
|
|
|
LICENSE= GPLv3
|
|
LICENSE_FILE= ${WRKSRC}/LICENSE
|
|
|
|
BUILD_DEPENDS= ${PYTHON_PKGNAMEPREFIX}setuptools_scm>=0:devel/py-setuptools_scm@${PY_FLAVOR}
|
|
RUN_DEPENDS= ${PYTHON_PKGNAMEPREFIX}asn1crypto>=1.3.0:devel/py-asn1crypto@${PY_FLAVOR} \
|
|
${PYTHON_PKGNAMEPREFIX}cert-human>=1.0.7:security/py-cert-human@${PY_FLAVOR} \
|
|
${PYTHON_PKGNAMEPREFIX}cryptography>=2.8:security/py-cryptography@${PY_FLAVOR} \
|
|
${PYTHON_PKGNAMEPREFIX}dnspython>=0:dns/py-dnspython@${PY_FLAVOR} \
|
|
${PYTHON_PKGNAMEPREFIX}ecdsa>=0.15:security/py-ecdsa@${PY_FLAVOR} \
|
|
${PYTHON_PKGNAMEPREFIX}forcediphttpsadapter>=1.0.1:www/py-forcediphttpsadapter@${PY_FLAVOR} \
|
|
${PYTHON_PKGNAMEPREFIX}GeoIP2>=3.0.0:net/py-GeoIP2@${PY_FLAVOR} \
|
|
${PYTHON_PKGNAMEPREFIX}hstspreload>=0:www/py-hstspreload@${PY_FLAVOR} \
|
|
${PYTHON_PKGNAMEPREFIX}idna>=2.9:dns/py-idna@${PY_FLAVOR} \
|
|
${PYTHON_PKGNAMEPREFIX}pyaml>=19.12.0:textproc/py-pyaml@${PY_FLAVOR} \
|
|
${PYTHON_PKGNAMEPREFIX}pycryptodome>=3.9.7:security/py-pycryptodome@${PY_FLAVOR} \
|
|
${PYTHON_PKGNAMEPREFIX}pyicu>=2.4.3:devel/py-pyicu@${PY_FLAVOR} \
|
|
${PYTHON_PKGNAMEPREFIX}openssl>=19.1.0:security/py-openssl@${PY_FLAVOR} \
|
|
${PYTHON_PKGNAMEPREFIX}redis>=3.4.1:databases/py-redis@${PY_FLAVOR} \
|
|
${PYTHON_PKGNAMEPREFIX}requests>=2.23.0:www/py-requests@${PY_FLAVOR} \
|
|
${PYTHON_PKGNAMEPREFIX}requests-toolbelt>=0.9.1:www/py-requests-toolbelt@${PY_FLAVOR} \
|
|
${PYTHON_PKGNAMEPREFIX}rq>=1.2.2:devel/py-rq@${PY_FLAVOR}
|
|
|
|
USES= python:3.6+
|
|
USE_PYTHON= autoplist concurrent distutils
|
|
|
|
NO_ARCH= yes
|
|
|
|
.include <bsd.port.mk>
|