AI Crawl Audit
Check whether GPTBot, ClaudeBot, PerplexityBot and other AI crawlers can access your website.
Blocked bots = that AI platform cannot read your site = lower AI visibility.
We fetch your robots.txt — no data is stored.
Which AI crawlers are checked?
GPTBot
· ChatGPT (OpenAI)
Blocks ChatGPT from reading your site
ClaudeBot
· Claude (Anthropic)
Blocks Claude from reading your site
PerplexityBot
· Perplexity AI
Blocks Perplexity from reading your site
Google-Extended
· Google Gemini / AI Overview
Blocks Gemini training & AI Overview
Bingbot
· Microsoft Copilot
Blocks Copilot (runs on Bing)
Amazonbot
· Amazon Alexa / AI
Blocks Amazon AI products
cohere-ai
· Cohere AI
Blocks Cohere AI crawlers
CCBot
· Common Crawl (LLM training)
Many LLMs trained on Common Crawl data
Bytespider
· ByteDance / TikTok AI
Blocks TikTok AI products