What is GPTBot? User agent and robots.txt rules
GPTBot is OpenAI’s crawler for content that may be used to train its generative AI foundation models. Disallowing it tells OpenAI not to use your site for training.
GPTBot user agent
As documented by OpenAI. Version numbers change; match on the token.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot
robots.txt rules for GPTBot
Paste one block into the robots.txt at the root of your domain. Changes can take up to a day to be read. Not sure what your file does today? Run the AI crawler checker.
Block GPTBot
Disallows every path for GPTBot. That keeps your future pages out of what OpenAI may use for training. It does not change search results.
User-agent: GPTBot Disallow: /
Allow GPTBot
Lets GPTBot fetch every path, whatever “User-agent: *” says.
User-agent: GPTBot Allow: /
Good to know
GPTBot is not the crawler behind ChatGPT search: that is OAI-SearchBot. You can block one and allow the other.
More in the guide: What is GPTBot?
GPTBotWhat GPTBot means for ChatGPT
Blocking GPTBot changes what OpenAI may use from your pages for training. It doesn’t tell you whether ChatGPT names your brand today: only the answers show that. Brisano asks ChatGPT your buyers’ questions as often as you choose and shows whether you are mentioned, in what position and which sources are cited.
Other OpenAI crawlers
OAI-SearchBot
OAI-SearchBot is OpenAI’s crawler for surfacing websites in ChatGPT’s search features. Sites…
User-triggeredChatGPT-User
ChatGPT-User fetches pages for actions people start in ChatGPT and custom GPTs, such as…
OtherOAI-AdsBot
OAI-AdsBot checks the landing pages of ads submitted to ChatGPT, for safety and to decide when…
GPTBot FAQ
What is GPTBot?
GPTBot is OpenAI’s crawler for content that may be used to train its generative AI foundation models. Disallowing it tells OpenAI not to use your site for training.
How do I block GPTBot?
Add “User-agent: GPTBot” and “Disallow: /” to your robots.txt. That keeps your future pages out of what OpenAI may use for training. It does not change search results.
Does GPTBot respect robots.txt?
Yes. OpenAI says GPTBot follows robots.txt rules written for it.
What is the GPTBot user agent?
OpenAI documents it as “Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot”. Version numbers change, so match on “GPTBot”.
How can I verify GPTBot requests?
Match the request IP against OpenAI’s published list (https://openai.com/gptbot.json). The user agent alone can be faked.
Is GPTBot the crawler behind ChatGPT search?
No. ChatGPT search uses OAI-SearchBot. GPTBot collects content that may train OpenAI’s models, so you can block GPTBot and still allow OAI-SearchBot.
Source: developers.openai.com, checked Nov 17, 2026. AI crawlers list and directory
Files are the start. Answers are the result.
Your buyers’ questions on up to eight AI engines, run as often as you choose, with full history. See whether ChatGPT mentions your brand.