Package : perl-WWW-RobotRules > RPM : perl-WWW-RobotRules-6.20.0-3.mga4.src.rpm
Basic items
| Name | perl-WWW-RobotRules |
| Version | 6.20.0 |
| Release | 3.mga4 |
| URL | http://search.cpan.org/dist/WWW-RobotRules |
| Group | Development/Perl |
| Summary | Parse /robots.txt file |
| Size | 11KB |
| Arch | noarch |
| License | GPL+ or Artistic |
Description
This module parses _/robots.txt_ files as specified in "A Standard for
Robot Exclusion", at <http://www.robotstxt.org/wc/norobots.html> Webmasters
can use the _/robots.txt_ file to forbid conforming robots from accessing
parts of their web site.
The parsed files are kept in a WWW::RobotRules object, and this object
provides methods to check if access to a given URL is prohibited. The same
WWW::RobotRules object can be used for one or more parsed _/robots.txt_
files on any number of hosts.
The following methods are provided:
Robot Exclusion", at <http://www.robotstxt.org/wc/norobots.html> Webmasters
can use the _/robots.txt_ file to forbid conforming robots from accessing
parts of their web site.
The parsed files are kept in a WWW::RobotRules object, and this object
provides methods to check if access to a given URL is prohibited. The same
WWW::RobotRules object can be used for one or more parsed _/robots.txt_
files on any number of hosts.
The following methods are provided:
Media information
| Distribution release | Mageia 4 |
| Media name | core-release |
| Media arch | i586 |
Advanced items
| Source RPM | NOT IN DATABASE ?! |
| Build time | 2013-10-18 15:58:14 |
| Changelog | View in Sophie |
| Files | View in Sophie |
| Dependencies | View in Sophie |