9381cc3bb0
robot_detection is a python module to detect if a given HTTP User Agent is a web crawler. It uses the list of registered robots from http://www.robotstxt.org: (Robots Database) |
||
---|---|---|
.. | ||
DESCR | ||
distinfo | ||
Makefile | ||
PLIST |