How to block Mail.RU_Bot
Mail.RU_Bot is the crawler for Mail.ru's search service. This page shows how to disallow it in robots.txt, when blocking is sensible, and the trade-off for sites that get traffic from the Russian Mail.ru search market.
robots.txt rule
Mail.ru's crawler identifies with the Mail.RU_Bot token. To disallow it site-wide:
User-agent: Mail.RU_Bot Disallow: /
Confirm the exact token from your logs, since Mail.ru may operate additional product-specific tokens. Match the documented token rather than a version string, and place the rule in its own user-agent group.
- robots.txt token: Mail.RU_Bot
- Additional Mail.ru product tokens may appear
- Confirm the token set from access logs
Market trade-off and limits
Mail.ru search has a user base in Russia and nearby regions. Blocking Mail.RU_Bot is a deliberate choice that removes the route to Mail.ru search visibility, so weigh it against any referral value before applying a blanket Disallow.
As always, a Disallow does not de-index already-indexed pages, and the user agent can be spoofed. Use noindex on a fetchable URL to drop a page from results, and treat lingering hits as possible non-compliant clients.
How it appears in analytics and logs
Mail.RU_Bot hits mean Mail.ru is crawling for its index. Continued hits after a Disallow point to a cached robots.txt, a token mismatch, or a spoofed user agent.
Diagnostic use case
Trim crawl load from Mail.RU_Bot for sites with no Mail.ru audience, while keeping it allowed where Mail.ru search referrals are valuable.
What WebmasterID can help detect
WebmasterID records Mail.RU_Bot as a search bot so you can verify a block is honoured and confirm you are not cutting off a market that actually sends referrals.
Common mistakes
- Blocking Mail.RU_Bot for a Russian-market site and losing Mail.ru visibility.
- Assuming a single token covers every Mail.ru crawler.
- Expecting the block to remove pages already indexed.
Privacy and accuracy notes
The rule matches the Mail.RU_Bot crawler token. No visitor data is involved, and robots.txt requests compliance rather than enforcing it.
Related pages
- Mail.ru search crawler — Mail.RU_Bot
Mail.ru is a major Russian internet portal and search provider, and it operates its own crawler to index the web for its search results. The crawler self-identifies with a Mail.RU_Bot token. It is a genuine regional search-engine indexer, so allowing it can help your pages appear in Mail.ru search.
- How to control YandexBot in robots.txt
YandexBot is the crawler for Yandex, a major search engine in Russia and nearby markets. You can target it in robots.txt with the YandexBot token. Yandex documents its robots.txt handling, has historically honoured crawl-delay, and provides additional crawl controls in Yandex.Webmaster.
- User-agent groups and matching in robots.txt
robots.txt rules are organised into user-agent groups. A crawler does not combine every group — it selects the single most specific group whose token matches its name, falling back to the * group only when no named group matches. Understanding this prevents rules that never apply.
- Web crawler encyclopedia
Reference for regional search crawlers like Mail.RU_Bot.
Sources and verification notes
- Google — How Google interprets robots.txtGeneral robots.txt syntax; Mail.RU_Bot token confirmed from logs.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.