How to block DotBot in robots.txt
DotBot is the crawler operated by Moz to build the link index behind its SEO tools. This page gives the robots.txt rule to disallow the DotBot token and notes that Moz documents the crawler and a Crawl-delay option for throttling it.
What DotBot is
DotBot is the crawler operated by Moz to gather link data for the index behind Moz's SEO tools. It is distinct from rogerbot, which Moz uses for site-audit crawling — the two are separate tokens with separate purposes. Blocking DotBot removes you from Moz's link index but does not affect search-engine indexing.
Moz publishes documentation describing DotBot, its robots.txt token, and supported throttling. Use that documentation rather than guessing the user-agent string.
The rule
To disallow DotBot site-wide, target its token:
User-agent: DotBot Disallow: /
Moz also documents Crawl-delay support, so to throttle rather than block:
User-agent: DotBot Crawl-delay: 10
Match the stable token. If you also want to stop Moz's audit crawler, target rogerbot separately. robots.txt is a request, not enforcement.
- Token: DotBot
- Operator: Moz (link index)
- Separate from rogerbot (Moz site-audit crawler)
How it appears in analytics and logs
A request carrying the DotBot token is Moz's link-index crawler fetching a URL. It is a bot event that feeds an SEO dataset, not a human visit or referral.
Diagnostic use case
Disallow DotBot to keep your pages out of Moz's link index, or use Crawl-delay to slow it instead of blocking it entirely.
What WebmasterID can help detect
WebmasterID classifies DotBot by its token as an SEO crawler, separate from human analytics, so you can confirm whether a disallow rule reduced its requests.
Common mistakes
- Assuming a DotBot rule also blocks Moz's rogerbot — they are separate tokens.
- Misspelling the token — it must be exactly DotBot.
- Treating Crawl-delay as something Google honours.
Privacy and accuracy notes
Blocking DotBot is a publishing-policy choice in a public file. It involves no visitor data and is not an access-control boundary.
Related pages
- DotBot — Moz's web crawler
DotBot is the crawler operated by Moz to build its link index, which powers Moz's link-analysis tools. It is a third-party crawler, not a search engine. It uses the DotBot robots.txt token and is documented as honouring robots.txt.
- How to block rogerbot in robots.txt
rogerbot is the crawler Moz uses for site-audit crawling in Moz Pro campaigns. This page gives the robots.txt rule to disallow the rogerbot token and explains that it is separate from DotBot, Moz's link-index crawler, so blocking one does not affect the other.
- The crawl-delay directive in robots.txt
Crawl-delay is a non-standard robots.txt directive that asks a crawler to wait between requests. Support is uneven: Google does not use it and points to Search Console instead, while Bing and Yandex have historically honoured it. This page explains the directive and the safer alternatives.
- Bot intelligence
See whether a DotBot disallow changed its activity.
Sources and verification notes
- Moz — DotBot crawler documentationDocuments the DotBot token, robots.txt support, and Crawl-delay.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.