How to block MJ12bot in robots.txt
MJ12bot is the distributed crawler operated by Majestic-12 to build the Majestic backlink index. This page gives the robots.txt rule to disallow the MJ12bot token and notes that Majestic documents both a disallow and a Crawl-delay option for slowing it down.
What MJ12bot is
MJ12bot is the crawler operated by Majestic-12 (Majestic) to build a large backlink index used by its SEO tools. Unlike most crawlers, it runs as a distributed effort, so its requests can originate from many independent participants rather than a single data center.
Majestic publishes documentation for MJ12bot, including the robots.txt token and a supported Crawl-delay directive for operators who want to throttle rather than fully block it.
The rule
To disallow MJ12bot site-wide, target its token:
User-agent: MJ12bot Disallow: /
To slow it instead of blocking it, Majestic documents support for Crawl-delay:
User-agent: MJ12bot Crawl-delay: 10
Match the stable token. Because the crawler is distributed, do not expect to verify it by a single IP range. robots.txt is a request to compliant crawlers, not enforcement.
- Token: MJ12bot
- Operator: Majestic-12 (Majestic backlink index)
- Distributed crawler — supports Crawl-delay per Majestic docs
How it appears in analytics and logs
A request carrying the MJ12bot token is a node of the distributed Majestic-12 crawler fetching a URL. Because it is distributed, requests can arrive from many different sources, which is normal for this crawler.
Diagnostic use case
Disallow MJ12bot to keep your pages out of the Majestic backlink index, or use Crawl-delay to slow it rather than block it outright.
What WebmasterID can help detect
WebmasterID classifies MJ12bot by its token as an SEO crawler, separate from human analytics, so you can confirm whether a disallow or Crawl-delay rule changed its request volume.
Common mistakes
- Trying to verify MJ12bot by a single IP range despite it being distributed.
- Misspelling the token — it must be exactly MJ12bot.
- Using Crawl-delay and expecting Google or Bing to also honour it.
Privacy and accuracy notes
Blocking MJ12bot is a publishing-policy choice in a public file. It involves no visitor data and is not an access-control boundary.
Related pages
- MJ12bot — Majestic's web crawler
MJ12bot is the crawler behind Majestic's backlink index. It is notable for being distributed — run across many independent operators — rather than a single central crawl. It uses the MJ12bot robots.txt token and is documented as honouring robots.txt.
- The crawl-delay directive in robots.txt
Crawl-delay is a non-standard robots.txt directive that asks a crawler to wait between requests. Support is uneven: Google does not use it and points to Search Console instead, while Bing and Yandex have historically honoured it. This page explains the directive and the safer alternatives.
- How to block DotBot in robots.txt
DotBot is the crawler operated by Moz to build the link index behind its SEO tools. This page gives the robots.txt rule to disallow the DotBot token and notes that Moz documents the crawler and a Crawl-delay option for throttling it.
- Bot intelligence
See whether an MJ12bot disallow or Crawl-delay changed its activity.
Sources and verification notes
- Majestic — MJ12bot crawler documentationDocuments the MJ12bot token, robots.txt support, and Crawl-delay.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.