What is GPTBot? User agent and robots.txt rules

GPTBot is OpenAI’s crawler for content that may be used to train its generative AI foundation models. Disallowing it tells OpenAI not to use your site for training.

AI training Follows robots.txt OpenAI

GPTBot user agent

As documented by OpenAI. Version numbers change; match on the token.

User agent
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot

robots.txt rules for GPTBot

Paste one block into the robots.txt at the root of your domain. Changes can take up to a day to be read. Not sure what your file does today? Run the AI crawler checker.

Block GPTBot

Disallows every path for GPTBot. That keeps your future pages out of what OpenAI may use for training. It does not change search results.

robots.txt
User-agent: GPTBot
Disallow: /

Allow GPTBot

Lets GPTBot fetch every path, whatever “User-agent: *” says.

robots.txt
User-agent: GPTBot
Allow: /

Good to know

GPTBot is not the crawler behind ChatGPT search: that is OAI-SearchBot. You can block one and allow the other.

More in the guide: What is GPTBot?

Operator OpenAI
Used forAI training
robots.txtFollows robots.txt
robots.txt tokenGPTBot
Can GPTBot reach your site? Enter your domain: we read your robots.txt and show GPTBot’s status.

What GPTBot means for ChatGPT

Blocking GPTBot changes what OpenAI may use from your pages for training. It doesn’t tell you whether ChatGPT names your brand today: only the answers show that. Brisano asks ChatGPT your buyers’ questions as often as you choose and shows whether you are mentioned, in what position and which sources are cited.

GPTBot FAQ

What is GPTBot?

GPTBot is OpenAI’s crawler for content that may be used to train its generative AI foundation models. Disallowing it tells OpenAI not to use your site for training.

How do I block GPTBot?

Add “User-agent: GPTBot” and “Disallow: /” to your robots.txt. That keeps your future pages out of what OpenAI may use for training. It does not change search results.

Does GPTBot respect robots.txt?

Yes. OpenAI says GPTBot follows robots.txt rules written for it.

What is the GPTBot user agent?

OpenAI documents it as “Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot”. Version numbers change, so match on “GPTBot”.

How can I verify GPTBot requests?

Match the request IP against OpenAI’s published list (https://openai.com/gptbot.json). The user agent alone can be faked.

Is GPTBot the crawler behind ChatGPT search?

No. ChatGPT search uses OAI-SearchBot. GPTBot collects content that may train OpenAI’s models, so you can block GPTBot and still allow OAI-SearchBot.

Source: developers.openai.com, checked Nov 17, 2026. AI crawlers list and directory

Files are the start. Answers are the result.

Your buyers’ questions on up to eight AI engines, run as often as you choose, with full history. See whether ChatGPT mentions your brand.