How to block Brandwatch
Brandwatch operates crawlers that collect public web and social content for its consumer-intelligence and social-listening products. It is a declared crawler with a documented robots.txt token. Operators who do not want their pages collected into brand-monitoring datasets can disallow it; this page shows the token and rule.
What this means
Brandwatch is a consumer-intelligence and social-listening platform that collects public web content so brands can monitor mentions, sentiment and trends. Part of that collection uses a crawler that fetches public pages. Blocking it asks Brandwatch to stop fetching your pages directly.
A block does not remove your brand mentions that Brandwatch gathers from social platforms and other feeds; it only addresses direct crawling of your own site. robots.txt is honoured by compliant crawlers and is not an access-control mechanism.
How to block it
Target the Brandwatch crawler token in its own user-agent group, matching on the stable token rather than a version string.
User-agent: Brandwatch Disallow: /
Because robots.txt is advisory, confirm in your logs that token-carrying requests stop. If they continue, the source may be impersonating the token, which a firewall rule would address. Brandwatch and adjacent social-listening tools such as Meltwater and Talkwalker each use their own token, so block them independently.
- robots.txt token to target: Brandwatch
- Sits alongside other social-listening crawlers (Meltwater, Talkwalker)
- A block stops direct crawling, not third-party social feeds
How it appears in analytics and logs
A request carrying the Brandwatch crawler token is a fetch for consumer-intelligence datasets, not a human visit. It is bot traffic. The user agent is a claim, so treat sustained activity as crawl coverage rather than audience.
Diagnostic use case
Stop Brandwatch from crawling your site for social-listening datasets, and confirm in your logs that the crawler honours the rule.
What WebmasterID can help detect
WebmasterID classifies social-listening crawlers like Brandwatch server-side and surfaces them on the bot-intelligence surface, so you can verify a robots.txt block without parsing raw logs.
Common mistakes
- Assuming a block removes brand mentions Brandwatch gathers from social platforms — it only stops direct crawling.
- Blocking Brandwatch and expecting it to also block Meltwater or Talkwalker — each uses its own token.
- Counting social-listening crawler hits as human traffic.
Privacy and accuracy notes
Blocking Brandwatch relies only on the request user-agent token. No human identity is involved. WebmasterID records the crawl as a bot event, separate from human analytics, and never attaches it to a visitor profile.
Related pages
- Brandwatch and social-monitoring crawlers
Brandwatch is a social-listening and consumer-intelligence platform that gathers public web and social content to track brand mentions, sentiment, and trends. It and similar tools crawl or fetch public pages to feed mention analysis, not to index content for search ranking. Their fetches appear in logs as monitoring traffic from the platform's infrastructure.
- Social-listening crawlers overview
Social-listening and media-monitoring platforms collect public web and social content to track brand mentions, sentiment, and trends for their customers. They are monitoring tools, not search crawlers: they analyse public conversation rather than indexing your pages to rank them. Much of their data also comes from platform APIs and licensed feeds, not only direct crawling.
- How to block the SimilarWeb crawler
SimilarWeb operates a crawler that gathers public web data for its market-intelligence and traffic-estimation products. It is a declared crawler with a documented robots.txt token, so operators who do not want their pages crawled for competitive-analytics datasets can disallow it. This page shows the token to target and the rule to use.
- Bot intelligence
Deterministic categorisation of crawlers, social-listening bots, and automation.
Sources and verification notes
- Brandwatch — Help & documentationBrandwatch documents its data collection; confirm the exact crawler token against current docs.
- Robots Exclusion Protocol (RFC 9309)Defines user-agent group and Disallow semantics.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.