GPTBot: OpenAI's training crawler
GPTBot is OpenAI's crawler for content that may be used to train its generative AI models.
Picked is an AI visibility tool for agencies and B2B SaaS teams. It asks ChatGPT, Claude, Gemini, Perplexity and Google's AI answers the prompts your buyers ask, records which brands and pages they name, and writes the article built to become the source they quote.
At a glance
GPTBot in one table
- Company
- OpenAI
- What it's for
- Training: collected to train future models.
- robots.txt token
GPTBot- Full user agent
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot- Follows robots.txt
- Follows it. Blocking it tells OpenAI not to use the site's content for training. It doesn't affect ChatGPT search, which reads pages as OAI-SearchBot.
- How to verify
- Check the visitor's IP against the list OpenAI publishes. A request outside it isn't GPTBot, whatever its user agent says. The IP list
- Source
- OpenAI's documentation, read on October 12, 2026.
What to know
GPTBot, in practice
GPTBot is the bot people mean when they say "block ChatGPT", and blocking it doesn't do that. ChatGPT's answers read pages through two other agents: OAI-SearchBot builds the index ChatGPT search draws from, and ChatGPT-User opens a page when a person asks. GPTBot only collects what may train future models.
OpenAI's current string is GPTBot/1.4. When it fetches robots.txt, OpenAI says it may add a robots.txt marker to the user agent, so the line for /robots.txt in your logs can differ from the one for a page.
For a brand, letting it in is how a future model learns you exist before it searches anything. If your pages are the product, a publisher's archive or a paid course, the opt-out costs you nothing in today's answers.
Real data
What GPTBot read on a real site
LaunchOn.it, our own launch platform, tracked in Picked through Cloudflare from September 12 to October 11, 2026. One site, so read it as an example, not an average.
965
Requests in 30 days
465
Different pages read
30 of 30
Days it came
99%
Verified by Cloudflare as the real bot
89% of its requests got a page back; 3% hit an address that doesn't exist. The rest were redirects or refusals.
- /_next/image14%
- /compare/13%
- /tech/12%
- /_next/12%
- /products/12%
By the first folder of the address. /_next/ holds the site's scripts and images.
robots.txt
Allow GPTBot
Allowing it lets your future pages into what the model learns, which is how it knows your brand before it searches. Opting out is a choice you can make if your content is your product: search and live-answer bots run under other names, so it doesn't take you out of today's answers.
User-agent: GPTBotAllow: /Needed only if a wider rule blocks it: robots.txt lets every bot in by default. A bot named in its own group ignores the User-agent: * group, so repeat your private paths there as Disallow lines. A block in your CDN or firewall has to be lifted there.
User-agent: GPTBotDisallow: /A rule only changes what the bot reads next. To see whether GPTBot reads your pages now, and which ones, track AI crawlers on your server or see the full AI crawler list.
Questions
Frequently asked questions
From OpenAI's documentation, read on October 12, 2026.
GPTBot is OpenAI's crawler for content that may be used to train its generative AI models.
Blocking it tells OpenAI not to use the site's content for training. It doesn't affect ChatGPT search, which reads pages as OAI-SearchBot.
Check the visitor's IP against the list OpenAI publishes. A request outside it isn't GPTBot, whatever its user agent says.
Allowing it lets your future pages into what the model learns, which is how it knows your brand before it searches. Opting out is a choice you can make if your content is your product: search and live-answer bots run under other names, so it doesn't take you out of today's answers.
Find out what ChatGPT and Google AI say about you
Add your domain and the prompts your buyers ask. Unlimited seats for your whole team, and no promises we can't keep.