How to block DataForSeoBot in robots.txt
DataForSeoBot is the crawler operated by DataForSEO to gather data for its SEO and SERP data APIs. This page gives the robots.txt rule to disallow the DataForSeoBot token and notes that DataForSEO publishes crawler documentation describing how it respects robots.txt.
What DataForSeoBot is
DataForSeoBot is the crawler operated by DataForSEO, a company that sells SEO and SERP data through APIs. Its crawl feeds backlink, ranking, and on-page datasets that DataForSEO's customers query. Blocking it removes your pages from that sampling but has no effect on Google or Bing indexing.
DataForSEO documents the crawler and the robots.txt token on its help pages, including guidance for operators who want to limit or block it.
The rule
To disallow DataForSeoBot site-wide, target its token:
User-agent: DataForSeoBot Disallow: /
The user-agent string contains the DataForSeoBot token plus a self-identifying URL pointing at DataForSEO's crawler page. Match the stable token and confirm in logs that activity drops. robots.txt is a request to compliant crawlers, not enforcement.
- Token: DataForSeoBot
- Operator: DataForSEO (SEO data APIs)
- Not a search engine — blocking it does not affect Google or Bing
How it appears in analytics and logs
A request carrying the DataForSeoBot token is DataForSEO's data crawler fetching a URL to populate its SEO datasets. It is a bot event, not a human visit, and brings no audience by itself.
Diagnostic use case
Disallow DataForSeoBot when you do not want your site sampled into third-party SEO datasets, or to reduce crawl load from a data-vendor crawler that brings you no direct visitors.
What WebmasterID can help detect
WebmasterID classifies DataForSeoBot by its token as an SEO/data crawler, separate from human analytics, so you can confirm whether a disallow rule reduced its requests.
Common mistakes
- Confusing DataForSeoBot with a search-engine crawler that affects indexing.
- Misspelling the token — it must be exactly DataForSeoBot.
- Inventing IP ranges instead of confirming the effect in logs.
Privacy and accuracy notes
Blocking DataForSeoBot is a publishing-policy choice in a public file. It involves no visitor data and is not an access-control boundary.
Related pages
- DataForSeoBot — SEO data crawler
DataForSeoBot is the crawler operated by DataForSEO to gather data exposed through its SEO data APIs. It is a third-party crawler, not a search engine, so it does not affect rankings directly. Its robots.txt token is DataForSeoBot; DataForSEO documents it, and some specifics are partially verified.
- How to block AhrefsBot in robots.txt
AhrefsBot is the crawler Ahrefs uses to build its backlink and SEO index. This page gives the robots.txt rule to disallow it and notes that Ahrefs documents support for both robots.txt rules and the crawl-delay directive, so you can slow rather than fully block it.
- How to block SemrushBot in robots.txt
SemrushBot is the crawler Semrush uses to build its SEO datasets. Semrush documents several specialised sub-bots under related tokens, so this page covers the base disallow rule and explains why you may need to target multiple tokens to cover the activity you care about.
- Bot intelligence
See whether a DataForSeoBot disallow changed its activity.
Sources and verification notes
- DataForSEO — DataForSeoBot crawler documentationDocuments the DataForSeoBot token and robots.txt handling.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.