DotBot — Moz's web crawler
DotBot is the crawler operated by Moz to build its link index, which powers Moz's link-analysis tools. It is a third-party crawler, not a search engine. It uses the DotBot robots.txt token and is documented as honouring robots.txt.
What this means
DotBot is the crawler Moz uses to build its link index, which underpins Moz's link-analysis and SEO tooling. It is a third-party crawler, distinct from search-engine crawlers, and its visits do not directly change your rankings.
DotBot's user agent contains the DotBot token together with a self-identifying URL pointing at Moz's documentation, following the common crawler pattern.
robots.txt considerations
Moz documents that DotBot honours robots.txt. To control it, target the DotBot token with a Disallow rule, or apply a crawl-delay where supported. The user agent is spoofable, so rely on Moz's documented guidance where authenticity matters.
- robots.txt token: DotBot
- Moz link-index crawler (not a search engine)
- Documented to honour robots.txt
How it appears in analytics and logs
A request carrying the DotBot token is Moz's link-index crawler fetching a URL — a bot event from a third-party tool, not a search engine and not a human visit. It does not influence search rankings by itself.
Diagnostic use case
Recognise DotBot as a third-party link-index crawler and control it via robots.txt using its token.
What WebmasterID can help detect
WebmasterID classifies DotBot server-side as an SEO crawler and shows its activity separately from human traffic and search-engine crawlers, so third-party crawling is visible without log parsing.
Common mistakes
- Assuming DotBot crawling affects search rankings — it does not.
- Treating link-index crawl hits as human visits.
- Blocking DotBot without realising it removes you from Moz's link index.
Privacy and accuracy notes
Identification uses the user agent — no human identity. WebmasterID records DotBot as a bot event, separate from human analytics.
Related pages
- MJ12bot — Majestic's web crawler
MJ12bot is the crawler behind Majestic's backlink index. It is notable for being distributed — run across many independent operators — rather than a single central crawl. It uses the MJ12bot robots.txt token and is documented as honouring robots.txt.
- SemrushBot — Semrush SEO crawler
SemrushBot is the crawler operated by Semrush to gather data for its SEO toolset. It is a third-party crawler, not a search engine, so it does not affect search rankings directly. It uses the SemrushBot robots.txt token and is documented as respecting robots.txt.
- Bot vs human
Separate automated crawlers from real human visits.
Sources and verification notes
- Moz — DotBot documentationDocuments DotBot, its purpose, and robots.txt handling.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.