Package : perl-WWW-RobotRules > RPM : perl-WWW-RobotRules-6.20.0-11.mga9.src.rpm
Basic items
| Name | perl-WWW-RobotRules |
| Version | 6.20.0 |
| Release | 11.mga9 |
| URL | https://metacpan.org/release/WWW-RobotRules |
| Group | Development/Perl |
| Summary | Parse /robots.txt file |
| Size | 12KB |
| Arch | noarch |
| License | GPL+ or Artistic |
Description
This module parses _/robots.txt_ files as specified in "A Standard for
Robot Exclusion", at <http://www.robotstxt.org/wc/norobots.html> Webmasters
can use the _/robots.txt_ file to forbid conforming robots from accessing
parts of their web site.
The parsed files are kept in a WWW::RobotRules object, and this object
provides methods to check if access to a given URL is prohibited. The same
WWW::RobotRules object can be used for one or more parsed _/robots.txt_
files on any number of hosts.
The following methods are provided:
Robot Exclusion", at <http://www.robotstxt.org/wc/norobots.html> Webmasters
can use the _/robots.txt_ file to forbid conforming robots from accessing
parts of their web site.
The parsed files are kept in a WWW::RobotRules object, and this object
provides methods to check if access to a given URL is prohibited. The same
WWW::RobotRules object can be used for one or more parsed _/robots.txt_
files on any number of hosts.
The following methods are provided:
Media information
| Distribution release | Mageia cauldron |
| Media name | core-release |
| Media arch | i586 |
Advanced items
| Source RPM | NOT IN DATABASE ?! |
| Build time | 2022-03-28 18:34:47 |
| Changelog | View in Sophie |
| Files | View in Sophie |
| Dependencies | View in Sophie |