Barkrowler — Babbar's web crawler
Barkrowler is the web crawler operated by Babbar (eXensa) to build the link graph and authority metrics behind the Babbar SEO platform. It is a third-party SEO/link-data crawler, not a search engine. Babbar documents Barkrowler and provides robots.txt guidance for operators who want to identify or restrict it.
What this means
Barkrowler is operated by Babbar (the SEO platform from eXensa) to crawl the public web and build the link graph and authority signals it sells. It is a link-data crawler, comparable in role to AhrefsBot or MJ12bot, and is not a search-engine indexer.
Because Barkrowler crawls broadly to map links, you may see it regardless of whether you use Babbar. That is normal for link-intelligence crawlers.
How Barkrowler identifies itself
Barkrowler uses the robots.txt user-agent token Barkrowler. Its user-agent string contains that token together with a self-identifying URL pointing at Babbar's Barkrowler information page. Match on the stable token rather than a full version string.
The user agent is a claim that can be copied. For requests where authenticity matters, corroborate against Babbar's published Barkrowler documentation rather than trusting the string alone.
- robots.txt token: Barkrowler
- User agent contains the Barkrowler token plus a Babbar URL
- Operated by Babbar (eXensa) for link/authority data
robots.txt control
Barkrowler honours robots.txt. To disallow it site-wide:
User-agent: Barkrowler Disallow: /
Babbar documents crawl-delay support so you can throttle Barkrowler instead of blocking it if load is the only concern. robots.txt is a request honoured by compliant crawlers, not an access-control boundary.
How it appears in analytics and logs
A request carrying the Barkrowler token is Babbar's crawler fetching a URL to expand its link graph — a bot event, not a human visit. It reflects link-data collection across the web and should be counted as crawl coverage rather than audience.
Diagnostic use case
Identify Barkrowler in logs, understand it is building link/authority data, and restrict or slow it via robots.txt if its crawl load is unwanted.
What WebmasterID can help detect
WebmasterID classifies Barkrowler server-side as an SEO crawler and surfaces its activity on the bot-intelligence surface, separate from human analytics, so you can see Babbar link-data hits without parsing server logs.
Common mistakes
- Counting Barkrowler link-graph hits as human sessions in analytics.
- Assuming Barkrowler is a search-engine crawler — it builds link-intelligence data.
- Blocking it outright when a crawl-delay would have reduced load enough.
Privacy and accuracy notes
Barkrowler detection uses only the request user-agent. No human identity is involved. WebmasterID records the crawl as a bot event, separate from human analytics, and never attaches it to a visitor profile.
Related pages
- AhrefsBot — Ahrefs SEO crawler
AhrefsBot is the crawler operated by Ahrefs to build its SEO and backlink index. It is a third-party crawler, not a search engine, so it does not affect Google or Bing rankings directly. It uses the AhrefsBot robots.txt token and is documented as respecting robots.txt and crawl-delay.
- MJ12bot — Majestic's web crawler
MJ12bot is the crawler behind Majestic's backlink index. It is notable for being distributed — run across many independent operators — rather than a single central crawl. It uses the MJ12bot robots.txt token and is documented as honouring robots.txt.
- Search crawlers vs SEO crawlers
Search-engine crawlers like Googlebot and Bingbot build the indexes that determine search visibility. Third-party SEO crawlers like AhrefsBot and SemrushBot feed analysis tools and do not affect rankings directly. Distinguishing them matters for crawl-budget reasoning and for deciding what to allow or limit.
- Bot intelligence
Deterministic categorisation of crawlers, search bots, and link-data tools.
Sources and verification notes
- Babbar — Barkrowler crawler informationDocuments the Barkrowler token and robots.txt handling.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.