Martin Wilke 201a939790 WWW::RobotRules::Parser allows you to simply parse robots.txt files ... as described in http://www.robotstxt.org/wc/norobots.html. WWW: http://http://search.cpan.org/dist/WWW-RobotRules-Parser/ PR: ports/120160 Submitted by: Masahiro Teramoto <markun at onohara.to>		2008-02-16 13:52:51 +00:00
..
distinfo
Makefile
pkg-descr
pkg-plist