Can AI find your site?
We check robots.txt rules against the 11 named AI crawlers — GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, Google-Extended, and the rest. We fetch your sitemap, look for an llms.txt, and — the part most audits miss — fingerprint whether your CDN is quietly turning AI bots away while robots.txt waves them in.
The CDN-vs-robots gap is the most damaging issue we look for. We fetch the homepage with each AI bot user-agent, compare against a baseline Chrome request, and flag mismatches. We won't pretend a 503 means "all bots allowed."
- robots.txt per bot
- Sitemap discovery
- llms.txt presence
- CDN baseline fetch
- Bot-vs-Chrome delta
- CDN vendor fingerprint
- Indexability (noindex)
- hreflang alternates