Mediapartners-Google (AdSense crawler)
Mediapartners-Google is the crawler Google AdSense uses to fetch and analyse the content of pages that show AdSense ads, so it can choose relevant ads. It is documented in Google's crawler list as a special-case crawler tied to the AdSense product, distinct from the Googlebot search crawler, and it has its own robots.txt token.
What this means
Mediapartners-Google is the AdSense crawler. When a page runs AdSense, this crawler fetches the page so Google can analyse its content and serve more relevant ads. It is a product crawler tied to AdSense, not the search index.
If you block this token, AdSense may be less able to match ads to that page's content, which can reduce ad relevance. That trade-off is different from blocking Googlebot, which affects search indexing.
How Mediapartners-Google identifies itself
It uses the robots.txt user-agent token Mediapartners-Google. Google documents it in the crawler list as the AdSense crawler. Match on the stable token rather than a version string.
As with any Google crawler, the user agent is a claim; verify the source IP against Google's published Googlebot/special-crawler ranges when authenticity matters.
- robots.txt token: Mediapartners-Google
- Purpose: AdSense content analysis for ad matching
- Separate from Googlebot search indexing
How it appears in analytics and logs
A request carrying the Mediapartners-Google token is the AdSense crawler reading a page to match ads, not Googlebot building the search index. It is bot traffic tied to ad serving; it does not by itself indicate search-ranking activity.
Diagnostic use case
Separate AdSense content-analysis crawling from search indexing in your logs, and understand why blocking the Mediapartners-Google token can reduce AdSense ad relevance on a page.
What WebmasterID can help detect
WebmasterID classifies Mediapartners-Google as the AdSense crawler separate from Googlebot, so ad-matching fetches do not inflate search-crawl coverage or human analytics in your bot-intelligence view.
Common mistakes
- Assuming Mediapartners-Google crawl activity reflects search indexing or ranking.
- Blocking the token and then seeing less relevant AdSense ads, without realising the cause.
- Counting AdSense crawl hits as human page views.
Privacy and accuracy notes
Mediapartners-Google is identified by its user-agent token and verifiable against Google's published ranges. It is a crawler, not a person; WebmasterID records it as a bot event and never links it to a visitor identity.
Related pages
- AdsBot-Google — Google Ads landing-page checker
AdsBot-Google is the Google crawler that checks the quality of web pages used as Google Ads landing pages. Google documents that AdsBot crawlers may ignore the global robots.txt wildcard group, so to control them you target the AdsBot-Google token explicitly.
- APIs-Google fetcher
APIs-Google is a Google fetcher user agent used when Google products send push notifications or other API-driven requests to a developer's server — for example a PubSubHubbub (WebSub) delivery. It is not the search crawler and is not used to build the search index. Google documents it in the list of Google crawlers and fetchers, and it is verifiable against Google's published crawler IP ranges.
- Googlebot Desktop — Google's secondary crawler
Googlebot Desktop is the desktop user-agent variant of Googlebot. Under mobile-first indexing it is secondary to Googlebot Smartphone for most sites. It shares the Googlebot robots.txt token and is verified the same way: reverse DNS into googlebot.com or google.com, or matching Google's published crawler IP ranges.
- Bot vs human
How automated fetches are separated from real visitors.
Sources and verification notes
- Google Search Central — Google crawlers and fetchers overviewMediapartners-Google documented as the AdSense crawler.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.