How to block rogerbot in robots.txt
rogerbot is the crawler Moz uses for site-audit crawling in Moz Pro campaigns. This page gives the robots.txt rule to disallow the rogerbot token and explains that it is separate from DotBot, Moz's link-index crawler, so blocking one does not affect the other.
What rogerbot is
rogerbot is the crawler Moz uses for its site-audit (Site Crawl) feature inside Moz Pro campaigns. It crawls a site to surface technical SEO issues for the campaign owner. It is distinct from DotBot, which Moz uses to build its broader link index — the two are separate tokens for separate jobs.
Moz publishes documentation for rogerbot, including the robots.txt token and a Crawl-delay option. Because rogerbot is campaign-driven, you may see it only when a Moz Pro campaign targets your domain.
The rule
To disallow rogerbot site-wide, target its token:
User-agent: rogerbot Disallow: /
Moz documents Crawl-delay support, so to throttle instead of block:
User-agent: rogerbot Crawl-delay: 10
Match the stable token. To also stop Moz's link-index crawler, target DotBot separately. robots.txt is a request, not enforcement.
- Token: rogerbot
- Operator: Moz (Moz Pro Site Crawl audits)
- Separate from DotBot (Moz link index)
How it appears in analytics and logs
A request carrying the rogerbot token is Moz's site-audit crawler fetching a URL, typically driven by a Moz Pro campaign. It is a bot event tied to an audit, not a human visit.
Diagnostic use case
Disallow rogerbot when you do not run Moz Pro audits on a site, or to stop a Moz audit crawler that someone else has pointed at your domain.
What WebmasterID can help detect
WebmasterID classifies rogerbot by its token as an SEO crawler, separate from human analytics, so you can confirm whether a disallow rule stopped its audit crawling.
Common mistakes
- Assuming a rogerbot rule also blocks DotBot — they are separate tokens.
- Misspelling the token — it must be exactly rogerbot.
- Expecting Crawl-delay to be honoured by search engines like Google.
Privacy and accuracy notes
Blocking rogerbot is a publishing-policy choice in a public file. It involves no visitor data and is not an access-control boundary.
Related pages
- How to block DotBot in robots.txt
DotBot is the crawler operated by Moz to build the link index behind its SEO tools. This page gives the robots.txt rule to disallow the DotBot token and notes that Moz documents the crawler and a Crawl-delay option for throttling it.
- The crawl-delay directive in robots.txt
Crawl-delay is a non-standard robots.txt directive that asks a crawler to wait between requests. Support is uneven: Google does not use it and points to Search Console instead, while Bing and Yandex have historically honoured it. This page explains the directive and the safer alternatives.
- How to block AhrefsBot in robots.txt
AhrefsBot is the crawler Ahrefs uses to build its backlink and SEO index. This page gives the robots.txt rule to disallow it and notes that Ahrefs documents support for both robots.txt rules and the crawl-delay directive, so you can slow rather than fully block it.
- Bot intelligence
See whether a rogerbot disallow stopped its audit crawling.
Sources and verification notes
- Moz — rogerbot crawler documentationDocuments the rogerbot token, robots.txt support, and Crawl-delay.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.