Robots.txt Analyzer
Check which crawlers are allowed or blocked. Googlebot, GPTBot, ClaudeBot + 9 more.
Enter a domain above to fetch and analyze its robots.txt
Check which crawlers are allowed or blocked. Googlebot, GPTBot, ClaudeBot + 9 more.
Enter a domain above to fetch and analyze its robots.txt