How to control Baiduspider in robots.txt
Baiduspider is the crawler for Baidu, the dominant search engine in China. You can target it with the Baiduspider token in robots.txt. Blocking it removes you from Baidu over time, which chiefly matters for sites serving Chinese-language or China-based audiences.
What Baiduspider is
Baiduspider is the primary crawler for Baidu, which holds the largest share of web search in China. If your audience does not include Chinese-language or China-based users, Baiduspider crawling may add load without visibility benefit; if it does, blocking it reduces your Baidu search presence.
The rule
To restrict a path for Baiduspider:
User-agent: Baiduspider Disallow: /private/
To disallow it entirely:
User-agent: Baiduspider Disallow: /
Baidu documents robots.txt handling for Baiduspider. As with any search crawler, a full block trades away search visibility in Baidu's market, and robots.txt remains a request to the compliant crawler rather than enforcement.
- Token: Baiduspider
- Baidu leads the China search market
- A full Disallow trades away Baidu visibility
How it appears in analytics and logs
A request with the Baiduspider token is Baidu's search crawler. Disallowing paths reduces their presence in Baidu search, the leading engine in China, so the trade-off is mainly about that market.
Diagnostic use case
Decide whether to allow or restrict Baiduspider based on whether you want visibility in Baidu's China-focused search market.
What WebmasterID can help detect
WebmasterID classifies Baiduspider as a search crawler separate from human traffic, so you can confirm a robots.txt change affected it as intended.
Common mistakes
- Blocking Baiduspider while still wanting visibility in Baidu's China market.
- Assuming a Googlebot rule covers Baiduspider — it has its own token.
- Misspelling the token — it must be exactly Baiduspider.
Privacy and accuracy notes
Managing Baiduspider is a crawl and search-visibility choice in a public file. It involves no visitor data.
Related pages
- Baiduspider — Baidu's web crawler
Baiduspider is the main crawler for Baidu, a leading search engine for Chinese-language search. It uses the Baiduspider robots.txt token. Baidu's documentation is primarily in Chinese and verification options are more limited than Google's, so treat verification with care.
- User-agent groups and matching in robots.txt
robots.txt rules are organised into user-agent groups. A crawler does not combine every group — it selects the single most specific group whose token matches its name, falling back to the * group only when no named group matches. Understanding this prevents rules that never apply.
- How to control Bingbot in robots.txt
Bingbot is Microsoft's search crawler. You can target it in robots.txt with the bingbot token, but fully disallowing it typically removes your pages from Bing search over time. For load concerns, Bing offers crawl-control settings in Bing Webmaster Tools rather than relying on a blanket block.
- Bot intelligence
See Baiduspider crawl activity separate from human traffic.
Sources and verification notes
- Baidu — Baiduspider and robots.txt helpDocuments Baiduspider and its robots.txt handling.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.