What is Google-Agent? User agent and robots.txt rules
Google-Agent is used by agents on Google infrastructure that browse the web and act on a user’s request.
Google-Agent user agent
Google documents the name only: requests contain “Google-Agent”.
robots.txt rules for Google-Agent
Paste one block into the robots.txt at the root of your domain. Changes can take up to a day to be read. Not sure what your file does today? Run the AI crawler checker.
Block Google-Agent
Disallows every path for Google-Agent. That asks Google not to fetch your pages when its users ask, though Google says user-triggered fetches may not always follow robots.txt.
User-agent: Google-Agent Disallow: /
Allow Google-Agent
Lets Google-Agent fetch every path, whatever “User-agent: *” says.
User-agent: Google-Agent Allow: /
Good to know
Google says user-triggered fetchers generally ignore robots.txt, because a person asked for the page.
Google-AgentOther Google crawlers
Googlebot
Googlebot is Google Search’s crawler. Google’s AI Overviews and AI Mode are Search features,…
AI trainingGoogle-Extended
Google-Extended is a robots.txt token, not a crawler. It controls whether pages Google already…
OtherGoogle-CloudVertexBot
Google-CloudVertexBot runs crawls that site owners request to build Vertex AI Agents. Google…
User-triggeredGoogle-GeminiNotebook
Google-GeminiNotebook fetches the URLs that people add as sources to Gemini Notebook projects.…
Google-Agent FAQ
What is Google-Agent?
Google-Agent is used by agents on Google infrastructure that browse the web and act on a user’s request.
How do I block Google-Agent?
Add “User-agent: Google-Agent” and “Disallow: /” to your robots.txt. That asks Google not to fetch your pages when its users ask, though Google says user-triggered fetches may not always follow robots.txt.
Does Google-Agent respect robots.txt?
Google says that because a person starts these requests, robots.txt rules may not apply to Google-Agent.
What is the Google-Agent user agent?
Google documents the name “Google-Agent”, which appears in its requests.
How can I verify Google-Agent requests?
Match the request IP against Google’s published list (https://developers.google.com/static/search/apis/ipranges/user-triggered-fetchers-google.json). The user agent alone can be faked.
Source: developers.google.com, checked Nov 17, 2026. AI crawlers list and directory
Files are the start. Answers are the result.
Your buyers’ questions on up to eight AI engines, run as often as you choose, with full history. See whether ChatGPT mentions your brand.