mirror of
https://git.FreeBSD.org/ports.git
synced 2024-11-19 00:13:33 +00:00
be1be4cd15
Protego is a pure-Python robots.txt parser with support for modern conventions. It is mainly used by Scrapy application when crawling websites. WWW: https://pypi.org/project/Protego/ |
||
---|---|---|
.. | ||
distinfo | ||
Makefile | ||
pkg-descr |