freebsd-ports/www/p5-WWW-RobotRules/distinfo
Frederic Culot 3395f95680 WWW::RobotRules parses /robots.txt files which are used to forbid conforming
robots from accessing parts of a web site. The parsed files are kept in a
WWW::RobotRules object, and this object provides methods to check if access
to a given URL is prohibited.

WWW: http://search.cpan.org/dist/WWW-RobotRules/

This new port is needed to update www/p5-libwww.
2011-04-14 13:18:39 +00:00

2 lines
144 B
Text

SHA256 (WWW-RobotRules-6.01.tar.gz) = f817e3e982c9d869c7796bcb5737c3422c2272355424acd162d0f3b132bec9d3
SIZE (WWW-RobotRules-6.01.tar.gz) = 9047