What is Claude-User? User agent and robots.txt rules

Claude-User fetches a page when a Claude user asks a question that needs it. Blocking it stops Claude from retrieving your content for those questions.

User-triggered Follows robots.txt Anthropic

Claude-User user agent

Anthropic documents the name only: requests contain “Claude-User”.

robots.txt rules for Claude-User

Paste one block into the robots.txt at the root of your domain. Changes can take up to a day to be read. Not sure what your file does today? Run the AI crawler checker.

Block Claude-User

Disallows every path for Claude-User. That asks Anthropic not to fetch your pages when its users ask, though Anthropic says user-triggered fetches may not always follow robots.txt.

robots.txt
User-agent: Claude-User
Disallow: /

Allow Claude-User

Lets Claude-User fetch every path, whatever “User-agent: *” says.

robots.txt
User-agent: Claude-User
Allow: /

Good to know

Anthropic says its bots honour robots.txt, including this one.

Operator Anthropic
Used forUser-triggered
robots.txtFollows robots.txt
robots.txt tokenClaude-User
Documentationsupport.claude.com
Can Claude-User reach your site? Enter your domain: we read your robots.txt and show Claude-User’s status.

What Claude-User means for Claude

When someone asks Claude about a page, Claude-User fetches it. That doesn’t decide whether your brand is named in the answer. Brisano measures that: it asks Claude your buyers’ questions as often as you choose and shows whether you are mentioned, in what position and which sources are cited.

Claude-User FAQ

What is Claude-User?

Claude-User fetches a page when a Claude user asks a question that needs it. Blocking it stops Claude from retrieving your content for those questions.

How do I block Claude-User?

Add “User-agent: Claude-User” and “Disallow: /” to your robots.txt. That asks Anthropic not to fetch your pages when its users ask, though Anthropic says user-triggered fetches may not always follow robots.txt.

Does Claude-User respect robots.txt?

Yes. Anthropic says Claude-User follows robots.txt rules written for it.

What is the Claude-User user agent?

Anthropic documents the name “Claude-User”, which appears in its requests.

How can I verify Claude-User requests?

Match the request IP against Anthropic’s published list (https://claude.com/crawling/bots.json). The user agent alone can be faked.

Source: support.claude.com, checked Nov 17, 2026. AI crawlers list and directory

Files are the start. Answers are the result.

Your buyers’ questions on up to eight AI engines, run as often as you choose, with full history. See whether ChatGPT mentions your brand.