How to block Barkrowler
Barkrowler is the crawler that feeds Babbar's link-graph and SEO datasets. This page shows how to disallow Barkrowler in robots.txt, how to slow it with crawl-delay, and how to confirm it has stopped.
robots.txt rule
Babbar documents that its crawler uses the Barkrowler token with a self-identifying URL in the user agent. To disallow it site-wide:
User-agent: Barkrowler Disallow: /
Babbar states Barkrowler honours robots.txt, so a correctly targeted Disallow is the supported way to opt out. Match the Barkrowler token rather than a version string.
- robots.txt token: Barkrowler
- Self-identifying URL in the user agent
- Honours robots.txt per Babbar's documentation
Throttle instead of block
To keep Barkrowler crawling but more gently, Babbar documents crawl-delay support:
User-agent: Barkrowler Crawl-delay: 10
That asks the crawler to wait between requests. Crawl-delay applies only to crawlers that honour it, so keep this directive in the Barkrowler group rather than a shared catch-all.
How it appears in analytics and logs
Barkrowler hits mean Babbar's crawler is mapping links to and from your pages for its dataset. It is not a search engine and provides no referral traffic.
Diagnostic use case
Stop Barkrowler from crawling your site for third-party link-graph data when you have no need to appear in Babbar's index.
What WebmasterID can help detect
WebmasterID records Barkrowler as an SEO/link crawler, letting you confirm a block is respected and keep its hits out of human analytics.
Common mistakes
- Misspelling the Barkrowler token so the rule never applies.
- Expecting crawl-delay to affect Googlebot — it does not.
- Treating link-graph crawler hits as human traffic.
Privacy and accuracy notes
The rule matches the Barkrowler token only. No visitor data is involved; robots.txt requests compliance from cooperative crawlers and does not enforce it.
Related pages
- Barkrowler — Babbar's web crawler
Barkrowler is the web crawler operated by Babbar (eXensa) to build the link graph and authority metrics behind the Babbar SEO platform. It is a third-party SEO/link-data crawler, not a search engine. Babbar documents Barkrowler and provides robots.txt guidance for operators who want to identify or restrict it.
- How to block AhrefsBot in robots.txt
AhrefsBot is the crawler Ahrefs uses to build its backlink and SEO index. This page gives the robots.txt rule to disallow it and notes that Ahrefs documents support for both robots.txt rules and the crawl-delay directive, so you can slow rather than fully block it.
- The crawl-delay directive in robots.txt
Crawl-delay is a non-standard robots.txt directive that asks a crawler to wait between requests. Support is uneven: Google does not use it and points to Search Console instead, while Bing and Yandex have historically honoured it. This page explains the directive and the safer alternatives.
- Bot intelligence
Tell Barkrowler apart from real visitors in your data.
Sources and verification notes
- Babbar — Barkrowler crawler informationOfficial Barkrowler page: token, robots.txt and crawl-delay support.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.