What is GPTBot?

GPTBot is OpenAI's web crawler for training: OpenAI says it is 'used to make our generative AI foundation models more useful and safe', and disallowing it tells OpenAI not to use your content for training. It is separate from OAI-SearchBot, which surfaces sites in ChatGPT search, so you can block one and allow the other.

By the Brisano team Updated November 18, 2026 4 min read

GPTBot is the web crawler OpenAI uses for its foundation models. OpenAI's own wording: GPTBot "is used to make our generative AI foundation models more useful and safe", and "disallowing GPTBot indicates a site's content should not be used in training generative AI foundation models" (OpenAI). It is one of four OpenAI user agents, and each one needs its own decision. Check whether GPTBot can crawl your site before you change any rule.

What GPTBot does, and what it doesn't

OpenAI's GPTBot reads public web pages so OpenAI can use them in training. It does not decide what ChatGPT shows in its search answers, and it does not fetch a page because a user asked for it. Those jobs belong to other crawlers.

That split matters because the three jobs lead to different results:

  • Training shapes what the model knows without searching, and changes only when a new model ships.
  • Search decides which pages ChatGPT can read and cite at answer time.
  • User actions fetch a page a person pointed ChatGPT to.

Why the difference matters for your site

Most site owners care about two outcomes at once: keeping their writing out of model training, and staying visible when buyers ask ChatGPT for a recommendation. With OpenAI the two don't collide: the training decision sits on GPTBot and the search decision sits on OAI-SearchBot, and you can answer each one differently. The mistake to avoid is treating "AI bots" as one thing, in robots.txt or in a CDN setting, and blocking search while you meant to block training.

OpenAI's crawlers: GPTBot, OAI-SearchBot, ChatGPT-User and OAI-AdsBot

User agent What OpenAI uses it for robots.txt IP ranges
GPTBot Training: "to make our generative AI foundation models more useful and safe" Honoured gptbot.json
OAI-SearchBot Surfaces websites in ChatGPT's search features Honoured searchbot.json
ChatGPT-User Actions a user starts in ChatGPT; "not used for crawling the web in an automatic fashion" "robots.txt rules may not apply" chatgpt-user.json
OAI-AdsBot Checks the safety of web pages submitted as ads on ChatGPT; not used for training Not stated adsbot.json

Source: OpenAI, Overview of OpenAI crawlers, checked on November 18, 2026. OpenAI says "each setting is independent of the others". For every AI bot, not only OpenAI's, see the AI crawlers list.

Check your robots.txt for AI crawlers Enter a domain: we show which AI bots may crawl it.

Should you block GPTBot?

Decide what you want from OpenAI, then pick the row:

You want GPTBot OAI-SearchBot Result
No training, but appear in ChatGPT search Disallow Allow Your pages can still be read and cited in ChatGPT search
Everything allowed Allow Allow Your content may be used in training and in search
Nothing from OpenAI Disallow Disallow Your pages are kept out of ChatGPT search answers too

To block training and allow search, the first row:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

Rules apply per user agent, so a Disallow for GPTBot doesn't affect OAI-SearchBot. For the other rules (one folder, one file type, the full string) see GPTBot user agent and robots.txt rules. Remember that a CDN or firewall "block AI bots" switch can override what your robots.txt says.

Does blocking GPTBot affect your visibility in ChatGPT?

Not in ChatGPT search. Visibility in ChatGPT's search answers depends on OAI-SearchBot, which OpenAI describes as the agent "used to surface websites in search results in ChatGPT's search features" (OpenAI). Answers that come from the model alone reflect what it learned in training, which is where GPTBot sits, and they change only when OpenAI ships a new model.

OpenAI says it can take about 24 hours from a robots.txt update for its systems to adjust. If AI visibility matters to you, track your brand in ChatGPT on a fixed set of prompts before and after the change, so you see what moved instead of guessing. The guide on how to rank in ChatGPT covers the rest of the checklist.

How to spot GPTBot in your logs

To monitor AI crawler activity, filter your access logs by user agent first, then confirm each hit by IP. A GPTBot request carries GPTBot in its user agent string; the complete string is on the GPTBot page of our AI crawlers list. Count requests per day and per status code: a sudden drop to zero after a deploy usually means a new rule is blocking it.

  • Match the IP, not only the name. A user agent string is easy to fake. OpenAI publishes the IP ranges of each crawler (linked in the table above); check a request's IP against the right list before you trust it.
  • Look for 403 and 429 responses. If your firewall or CDN answers OpenAI's crawlers with errors, your robots.txt isn't the rule that counts.
  • Check again after every change. After you edit robots.txt or a bot rule, review your logs over the next days and confirm the crawlers you allowed still get a 200.

Sources

  1. Overview of OpenAI crawlers · OpenAI, checked 2026-11-18
  2. GPTBot IP ranges · OpenAI
  3. OAI-SearchBot IP ranges · OpenAI
  4. ChatGPT-User IP ranges · OpenAI
  5. OAI-AdsBot IP ranges · OpenAI

Frequently asked questions

What is GPTBot used for?

OpenAI says GPTBot is used to make its generative AI foundation models more useful and safe. Disallowing it in robots.txt indicates that your content should not be used to train those models.

Does blocking GPTBot remove my site from ChatGPT search?

No. ChatGPT search uses a different crawler, OAI-SearchBot, and OpenAI says each setting is independent of the others. You can disallow GPTBot and still allow OAI-SearchBot.

How long does a robots.txt change take to apply?

OpenAI says it can take about 24 hours from a robots.txt update for its systems to adjust.

Does ChatGPT-User follow robots.txt?

ChatGPT-User visits pages when a user asks ChatGPT to, not as automatic crawling. Because those visits are started by users, OpenAI says robots.txt rules may not apply.

What is OAI-AdsBot?

OpenAI's crawler that checks the safety of web pages submitted as ads on ChatGPT. OpenAI says the data it collects is not used to train its foundation models.

How can I verify a request really comes from GPTBot?

OpenAI publishes the IP ranges of each crawler as JSON files. Compare the request's IP with the published list instead of trusting the user agent alone.

Should I block GPTBot?

It's a choice about training, not about search visibility. You can block training and still allow search crawlers. Decide per crawler, and check that your CDN rules match your robots.txt.

See where you stand in AI answers.

Choose a plan and see your first answers in minutes.