User agents and dynamic serving for SEO
Dynamic serving means returning different HTML to different user agents at the same URL — for example a mobile variant to phones. For SEO this is allowed in principle but risky: if crawlers receive materially different content from users, it edges toward cloaking. Treating search bots the same as the human they emulate is the safe rule.
What this means
Dynamic serving (sometimes called dynamic rendering in a related context) returns different markup based on the requesting user agent while keeping a single URL. A common motivation is delivering a tailored mobile or prerendered variant.
Search engines permit responding to different devices, but they care that the crawler and the equivalent user get consistent content. The risk is that serving search bots something materially different from humans crosses into cloaking, which search guidelines treat as a violation.
Staying on the safe side
The durable rule is: show a search crawler the same content you would show the human it is standing in for. If you serve a mobile variant to phones, the mobile search crawler should get that same mobile variant — not a special bot-only page.
Because user-agent-based branching is fragile and crawler user agents can be emulated, verify behaviour rather than trusting strings, and keep crawler and human content aligned. Where possible, prefer responsive design or client hints over UA-keyed HTML swaps.
- Allowed in principle, but divergent bot vs human content risks cloaking
- Serve crawlers the same variant their human equivalent gets
- Prefer responsive design / client hints over UA-keyed HTML swaps
How it appears in analytics and logs
When HTML varies by user agent at one URL, a crawler and a human can receive different content. In logs this appears as UA-correlated differences in the response body for the same path.
Diagnostic use case
Decide whether to serve content dynamically by user agent, understand the cloaking boundary, and ensure search crawlers see what equivalent human users see.
What WebmasterID can help detect
WebmasterID records each request's user agent server-side, so you can confirm which variant search crawlers received versus human visitors and catch accidental divergence between them.
Common mistakes
- Serving search bots a different page than humans at the same URL.
- Relying on UA branching that breaks as strings change.
- Assuming dynamic serving is automatically safe without checking parity.
Privacy and accuracy notes
Dynamic serving keys on the request user agent, a client hint, not a person. Any grouping it produces is coarse, and search-bot user agents are crawler claims, not human identities.
Related pages
- User agents and adaptive serving
Adaptive serving means returning different markup, assets, or experiences based on the requesting client, often keyed on the user agent. It can deliver lighter pages to constrained devices, but UA-based branching is brittle: strings drift, edge cases proliferate, and a wrong guess degrades real users. Client hints and feature detection are the more durable approach.
- Mobile vs desktop UA detection
Distinguishing mobile from desktop clients is one of the few user-agent checks still broadly reliable: mobile browsers include a Mobile token and platform markers. But user-agent reduction trims finer detail, so for form-factor decisions the modern, robust approach is the Sec-CH-UA-Mobile Client Hint and responsive design rather than deep UA parsing.
- Googlebot Smartphone — Google's mobile-first crawler
Googlebot Smartphone is the mobile user-agent variant of Googlebot and, under mobile-first indexing, Google's primary crawler for most sites. It uses the Googlebot robots.txt token and can be verified through reverse DNS and Google's published crawler IP ranges.
- Website observability
Confirm crawlers and humans receive matching content per URL.
Sources and verification notes
- MDN — Browser detection using the user agentDocuments the fragility of UA-based serving; basis for the cloaking-avoidance guidance.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.