We distinguish three main categories of AI bots in order to adapt our processing rules:
ai-assistant
Conversational assistants that interact directly with users.
`perplexity`, `duckassistbot`, `claude-web`, `anthropic`, `assistant`
ai-crawler
Crawlers used to collect content on a large scale.
`gptbot`, `chatgpt`, `openai`, `bard`, `google-extended`, `anthropic-ai`, `claude-bot`, `meta-externalagent`, `facebookbot`, `ccbot`, `common crawl`, `ai2bot`, `diffbot`, `omgili`, `omgilibot`, `bytespider`, `bytedance`.
ai-search
AI search engines that combine crawling and answer generation.
`oai-searchbot`, `searchgpt`, `bing.*gpt`, `you-bot`, `youbot`.
When you activate the SEO html to markdown recipe from the SEO Recommendation application, we automatically deliver the Markdown version of your pages to the bots listed above.
This ensures that assistants, crawlers and AI engines access lean, optimized content.
You remain in control: your human users continue to see the full HTML version, while the AI bots receive a format designed for their use.