What is Claude-User? User agent and robots.txt rules
Claude-User fetches a page when a Claude user asks a question that needs it. Blocking it stops Claude from retrieving your content for those questions.
Claude-User user agent
Anthropic documents the name only: requests contain “Claude-User”.
robots.txt rules for Claude-User
Paste one block into the robots.txt at the root of your domain. Changes can take up to a day to be read. Not sure what your file does today? Run the AI crawler checker.
Block Claude-User
Disallows every path for Claude-User. That asks Anthropic not to fetch your pages when its users ask, though Anthropic says user-triggered fetches may not always follow robots.txt.
User-agent: Claude-User Disallow: /
Allow Claude-User
Lets Claude-User fetch every path, whatever “User-agent: *” says.
User-agent: Claude-User Allow: /
Good to know
Anthropic says its bots honour robots.txt, including this one.
Claude-UserWhat Claude-User means for Claude
When someone asks Claude about a page, Claude-User fetches it. That doesn’t decide whether your brand is named in the answer. Brisano measures that: it asks Claude your buyers’ questions as often as you choose and shows whether you are mentioned, in what position and which sources are cited.
Claude-User FAQ
What is Claude-User?
Claude-User fetches a page when a Claude user asks a question that needs it. Blocking it stops Claude from retrieving your content for those questions.
How do I block Claude-User?
Add “User-agent: Claude-User” and “Disallow: /” to your robots.txt. That asks Anthropic not to fetch your pages when its users ask, though Anthropic says user-triggered fetches may not always follow robots.txt.
Does Claude-User respect robots.txt?
Yes. Anthropic says Claude-User follows robots.txt rules written for it.
What is the Claude-User user agent?
Anthropic documents the name “Claude-User”, which appears in its requests.
How can I verify Claude-User requests?
Match the request IP against Anthropic’s published list (https://claude.com/crawling/bots.json). The user agent alone can be faked.
Source: support.claude.com, checked Nov 17, 2026. AI crawlers list and directory
Files are the start. Answers are the result.
Your buyers’ questions on up to eight AI engines, run as often as you choose, with full history. See whether ChatGPT mentions your brand.