How to use this tool
- Paste input in the format shown in the example.
- Set the options and run the analysis.
- Review the output and any limitations before using the results.
Try this example
User-agent: * Disallow: /private/ Allow: /private/public/
Accuracy and scope
Supports user-agent groups, allow/disallow, * and terminal $. Does not model percent-encoding equivalence or every crawler behavior. Crawling permission is not indexing permission.
Is my data uploaded?
This tool runs locally in your browser. Input and output are not sent to a processing server. Copy or download your result before leaving the page.