Pure-Python robots.txt parser with support for modern conventions
Pure-Python robots.txt parser with support for modern conventions.