Know when AI crawlers get blocked
Paste a domain. See the robots.txt matrix and live fetches for GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, Google-Extended and more. Watch alerts only on regressions (was OK → now blocked). Optional checkbox: also watch /llms.txt.
robots.txt matrix
Allow/deny for the common AI bots that matter for GEO and training.
Live UA fetches
We request your homepage with each bot's User-Agent to catch CDN/WAF 403s robots.txt cannot show.
Regression alerts
Email only when something that was OK becomes blocked — not on deliberate steady-state blocks you mark intentional.
What we don't claim
CrawlDiff does not guarantee ChatGPT citations, rankings, or that any model will read your site. It reports crawl access and file health.