What is Google-Agent? User agent and robots.txt rules

Google-Agent is used by agents on Google infrastructure that browse the web and act on a user’s request.

User-triggered May skip robots.txt Google

Google-Agent user agent

Google documents the name only: requests contain “Google-Agent”.

robots.txt rules for Google-Agent

Paste one block into the robots.txt at the root of your domain. Changes can take up to a day to be read. Not sure what your file does today? Run the AI crawler checker.

Block Google-Agent

Disallows every path for Google-Agent. That asks Google not to fetch your pages when its users ask, though Google says user-triggered fetches may not always follow robots.txt.

robots.txt
User-agent: Google-Agent
Disallow: /

Allow Google-Agent

Lets Google-Agent fetch every path, whatever “User-agent: *” says.

robots.txt
User-agent: Google-Agent
Allow: /

Good to know

Google says user-triggered fetchers generally ignore robots.txt, because a person asked for the page.

Operator Google
Used forUser-triggered
robots.txtMay skip robots.txt
robots.txt tokenGoogle-Agent
Can Google-Agent reach your site? Enter your domain: we read your robots.txt and show Google-Agent’s status.

Google-Agent FAQ

What is Google-Agent?

Google-Agent is used by agents on Google infrastructure that browse the web and act on a user’s request.

How do I block Google-Agent?

Add “User-agent: Google-Agent” and “Disallow: /” to your robots.txt. That asks Google not to fetch your pages when its users ask, though Google says user-triggered fetches may not always follow robots.txt.

Does Google-Agent respect robots.txt?

Google says that because a person starts these requests, robots.txt rules may not apply to Google-Agent.

What is the Google-Agent user agent?

Google documents the name “Google-Agent”, which appears in its requests.

How can I verify Google-Agent requests?

Match the request IP against Google’s published list (https://developers.google.com/static/search/apis/ipranges/user-triggered-fetchers-google.json). The user agent alone can be faked.

Source: developers.google.com, checked Nov 17, 2026. AI crawlers list and directory

Files are the start. Answers are the result.

Your buyers’ questions on up to eight AI engines, run as often as you choose, with full history. See whether ChatGPT mentions your brand.