Guix will hold its first online conference November 22nd. Propose a talk before November 6th. Learn more!

python-robot-detection 0.4 Detect web crawlers

robot_detection is a python module to detect if a given HTTP User Agent is a web crawler. It uses the list of registered robots from http://www.robotstxt.org.