How to block Bedrockbot
Bedrockbot is the crawler associated with Amazon Bedrock for fetching web content. This page shows how to disallow Bedrockbot in robots.txt, how it differs from Amazonbot, and how to confirm the rule is honoured.
robots.txt rule
To disallow Bedrockbot site-wide, target its token in its own group:
User-agent: Bedrockbot Disallow: /
Confirm the exact token against your access logs and Amazon's published guidance before relying on it. Match the documented token rather than a full version string, and remember robots.txt is honoured by compliant crawlers, not enforced.
- robots.txt token: Bedrockbot
- Confirm the token from logs and Amazon docs
- Distinct from the Amazonbot token
Bedrockbot vs Amazonbot
Amazon operates more than one crawler. Bedrockbot relates to Amazon Bedrock, while Amazonbot is Amazon's general web crawler — they are separate tokens with separate purposes. Blocking one does not block the other.
If your intent is to decline Amazon's AI crawling broadly, target each relevant token explicitly in robots.txt rather than assuming a single rule covers all of Amazon's bots.
How it appears in analytics and logs
Bedrockbot hits indicate Amazon Bedrock-related crawling reached a URL. It is a bot event, not a human visit, and should be counted as AI-crawler activity.
Diagnostic use case
Opt your site out of Bedrockbot's crawling while keeping or separately deciding policy for Amazonbot and other AI crawlers.
What WebmasterID can help detect
WebmasterID classifies Bedrockbot as an AI crawler distinct from Amazonbot, so you can confirm a block lands and see Bedrock crawl coverage separately.
Common mistakes
- Blocking Amazonbot and assuming it also stops Bedrockbot.
- Using a stale token so the Disallow never matches.
- Counting Bedrockbot crawl hits as human traffic.
Privacy and accuracy notes
The rule matches the Bedrockbot token only. No visitor identity is involved; robots.txt is a request to compliant crawlers rather than enforcement.
Related pages
- Bedrockbot — Amazon Bedrock crawler
Bedrockbot is a crawler Amazon documents in association with Amazon Bedrock, used to retrieve web content for Bedrock features. Amazon lists it among its crawlers with a robots.txt token. It is distinct from Amazonbot; identify each by its own token and set policy separately.
- How to block Amazonbot in robots.txt
Amazonbot is Amazon's web crawler. This page gives the robots.txt rule to disallow it, notes that Amazon documents Amazonbot's robots.txt compliance and a way to verify its requests, and keeps the usual caveat that robots.txt is a request to compliant crawlers, not enforcement.
- Writing an AI crawler policy for robots.txt
An AI crawler policy is a deliberate decision about which AI-related tokens you allow and which you disallow in robots.txt. This page offers a structured way to make and document those choices, while staying realistic: robots.txt is a request to compliant crawlers, not a legal or technical guarantee.
- AI visibility analytics
See Bedrockbot crawl coverage separate from human visits.
Sources and verification notes
- Amazon — Bedrock crawler / bot documentationConfirm the Bedrockbot token from Amazon documentation and your logs.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.