Googlebot Smartphone — Google's mobile-first crawler
Googlebot Smartphone is the mobile user-agent variant of Googlebot and, under mobile-first indexing, Google's primary crawler for most sites. It uses the Googlebot robots.txt token and can be verified through reverse DNS and Google's published crawler IP ranges.
What this means
Under mobile-first indexing, Google primarily crawls with Googlebot Smartphone — a mobile user agent. The desktop variant (Googlebot Desktop) still exists but is secondary for most sites. Both share the robots.txt token Googlebot, so robots rules written for Googlebot apply to both.
How to verify Googlebot
Because the Googlebot user agent is widely spoofed, verify it rather than trusting the string. Google supports two methods: a reverse DNS lookup that should resolve into googlebot.com or google.com (then a forward lookup back to the same IP), and a published list of Googlebot IP ranges you can match against.
- robots.txt token: Googlebot (covers smartphone + desktop variants)
- Verify via reverse DNS to googlebot.com / google.com
- Or match the source IP against Google's published crawler ranges
What it means for indexing
Regular Googlebot Smartphone crawling is the normal, healthy state. Use Search Console's URL Inspection and Crawl Stats for authoritative data — server logs show the crawl, but Search Console shows how Google processed it. A page that is crawled but not indexed is a content/quality question, not a crawler one.
How it appears in analytics and logs
Requests from Googlebot Smartphone indicate Google is crawling the mobile rendering of your pages. Healthy, regular Googlebot crawling is expected; a sudden drop can signal an indexing or availability problem worth investigating.
Diagnostic use case
Confirm Google is crawling your site mobile-first, and verify a request claiming to be Googlebot before trusting it — fake Googlebot traffic is common.
What WebmasterID can help detect
WebmasterID classifies Googlebot server-side as a search crawler and shows its crawl activity separately from human traffic, so you can see Google crawl coverage without raw log access.
Common mistakes
- Trusting the Googlebot user agent without reverse-DNS or IP verification.
- Serving different content to Googlebot than to users (cloaking) — a policy violation.
- Assuming desktop rendering is what Google sees; mobile-first means the mobile render matters most.
Privacy and accuracy notes
Crawler identification uses the user-agent plus reverse-DNS/IP verification — no human identity. WebmasterID records Googlebot activity as bot events kept out of human analytics.
Related pages
- Bingbot — Microsoft Bing's web crawler
Bingbot is the crawler Microsoft Bing uses to discover and index web pages. It uses the bingbot robots.txt token and can be verified through Bing's reverse-DNS method and published IP ranges. Bing also powers results for other surfaces, so Bingbot coverage has reach beyond Bing.com.
- Spoofed and fake user agents: what to watch for
Spoofing a user agent is trivial — any client can claim to be Googlebot or a normal browser. This page explains why spoofing happens, the common fake-crawler patterns, and the verification methods that turn a claimed identity into a confirmed one.
- HTTP 404 Not Found: what it means for crawlers
404 Not Found means the server has no resource at that URL. It is the correct, healthy response for genuinely missing pages — crawlers expect some 404s. Problems arise when important pages 404 by accident, when removed pages should signal 410, or when 'not found' pages wrongly return 200.
- Bot intelligence
Deterministic categorisation of crawlers and automation.
Sources and verification notes
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.