OpenAI uses three separate bots, and each has a different job:
- GPTBot collects pages that may be used to train OpenAI’s models.
- OAI-SearchBot indexes pages so they can appear and be cited in ChatGPT search.
- ChatGPT-User opens a page when a person asks ChatGPT to look at it, live, in a conversation.
You can allow or block each one on its own. Most importantly, blocking GPTBot (training) does not remove you from ChatGPT search, which depends on OAI-SearchBot.
The three bots side by side
| Bot | What it does | How it visits | Official IP list |
|---|---|---|---|
| GPTBot | Collects content for training models | Crawls many pages, on its own schedule | openai.com/gptbot.json |
| OAI-SearchBot | Indexes pages for ChatGPT search | Crawls, to keep the search index fresh | openai.com/searchbot.json |
| ChatGPT-User | Fetches a page for a person, right now | One or a few pages, when someone asks | openai.com/chatgpt-user.json |
Each bot has its own list. A request that says “ChatGPT-User” should come from the ChatGPT-User list; if it comes from somewhere else, treat it as fake. See how to verify any AI crawler.
GPTBot: training
GPTBot is the crawler most people mean when they talk about “blocking ChatGPT”. OpenAI describes it as collecting public web content that may be used to improve and train its generative AI models. If you disallow GPTBot in robots.txt, you are telling OpenAI not to use your site’s content for training.
Its user-agent contains GPTBot, for example:
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)In your logs, GPTBot usually looks like a classic crawler: many pages, spread over hours or days, often including old and deep pages.
OAI-SearchBot: ChatGPT search
OAI-SearchBot builds the index that ChatGPT search uses to find and link to websites. If you want your pages to show up as sources in ChatGPT’s answers, allow OAI-SearchBot. If you block it, OpenAI says your pages won’t be shown in ChatGPT search answers, although they may still appear as plain navigational links.
Blocking OAI-SearchBot and blocking GPTBot are independent choices. Many sites allow OAI-SearchBot for the visibility and the visitors it sends, and block GPTBot because they don’t want their content used for training.
ChatGPT-User: a person asked
ChatGPT-User isn’t a crawler. It fetches a page because a person asked ChatGPT something that needed it: “summarise this article”, “what does this pricing page say”, or a link a custom GPT opens. It reads one or a few pages and stops.
Two things to know:
- These visits mean someone is using ChatGPT to read your page, right now. They are a strong signal of interest, and worth counting.
- Because a person starts them, OpenAI says robots.txt rules may not apply to these fetches. Check OpenAI’s current bot documentation if this matters for you.
What about ChatGPT agent?
OpenAI’s agent features browse the web with a real browser, so their requests can carry an ordinary Chrome-like user-agent instead of a bot name. OpenAI signs these requests (a Signature-Agent header naming chatgpt.com), and they come from OpenAI’s addresses. A user-agent check alone will miss them; an address or signature check won’t. OZMAICO checks both.
Recommended robots.txt setups
Appear in ChatGPT search, but no training:
User-agent: OAI-SearchBot
Allow: /
User-agent: GPTBot
Disallow: /Keep OpenAI out completely:
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /Allow everything (the default when you don’t mention them): you don’t need to add anything.
Changes can take a while to show up. OpenAI says it can take about a day for its systems to notice a robots.txt change for search. For the full syntax, see how to block or allow AI bots in robots.txt.
How to tell which one visited you
- Look at the user-agent token:
GPTBot,OAI-SearchBotorChatGPT-User. - Check the address against that bot’s list. The GPTBot, OAI-SearchBot and ChatGPT-User pages show every current range.
- Or paste the visit into the bot checker.
Fakes using the GPTBot name are common. If one fails the check, don’t let it change your view of OpenAI’s real crawling; see fake AI bots and how to block them.
See all three on your own site
Create a free account to see how often each OpenAI bot visits, which pages each one reads, and how many people then click through to your site from ChatGPT.
Check your own website
See which AI bots really visit your site — real or fake, page by page — and how many people AI assistants send you. Free.