GPTBot is OpenAI's official web crawler, identified by the User-Agent string 'GPTBot'. Active since August 2023, it crawls publicly accessible web pages to update OpenAI's index for ChatGPT Search, ChatGPT browsing, and (separately, with opt-in) model training corpora.
Allowing GPTBot in robots.txt is the foundational AIO move for any site that wants to be cited in ChatGPT answers. Many sites still block GPTBot by default; that's the 2026 equivalent of blocking Googlebot in 2005.
OpenAI publishes the IP ranges and crawler behavior at platform.openai.com/docs/bots. The crawler respects robots.txt directives. To verify crawler access, check your server logs for the 'GPTBot' user agent or use a crawl-simulation tool.
Read next
Frequently asked
How do I allow GPTBot in robots.txt?
Add this block to your robots.txt: User-agent: GPTBot followed by Allow: /. Specify disallow paths only for sign-in and admin routes that shouldn't be indexed.
Does GPTBot affect SEO ranking on Google?
No. GPTBot only affects OpenAI's index. Google has its own AI crawler (Google-Extended) and its main Googlebot crawler. Each is independent.
Should I block GPTBot if I don't want my content used for AI training?
Possibly. OpenAI offers separate opt-out for training data versus search indexing. Read OpenAI's bot policy and decide per your content licensing posture. For most marketing content, allowing GPTBot is the right default.
More from AIO and AEO
All termsAI Visibility
How often and how accurately a brand appears inside AI engine answers (ChatGPT, Claude, Perplexity).
ReadClaudeBot
Anthropic's web crawler that indexes content for Claude's search and citation features.
ReadPerplexityBot
Perplexity AI's web crawler that powers citation in Perplexity's search and answer engine.
Read