GoogleOther — Google non-Search crawler
GoogleOther is a generic crawler Google uses for fetches not tied to building the Search index — for example internal research and development crawls. Google documents it as separate from Googlebot, with its own robots.txt token. Controlling GoogleOther does not affect Googlebot's Search crawling, and vice versa.
What this means
GoogleOther is a generic crawler Google introduced for fetches that are not about building the Search index — Google describes it as used by various teams for purposes such as research and development crawling. It shares Google's crawler infrastructure but serves different, non-Search purposes.
Because it is separate from Googlebot, a policy decision for GoogleOther does not change how Googlebot crawls for Search. Google documents the two as independent tokens.
How GoogleOther identifies itself
GoogleOther uses the robots.txt user-agent token GoogleOther. Its user-agent string contains that token. Match on the stable token rather than a full version string.
Like all Google crawlers, GoogleOther can be verified against Google's published methods (such as reverse-DNS into googlebot.com / google.com domains and Google's published IP ranges). The user agent alone is a claim, so verify where authenticity matters and do not invent ranges.
- robots.txt token: GoogleOther
- Non-Search uses (e.g. research and development crawling)
- Distinct from Googlebot; verify via Google's published methods
robots.txt considerations
GoogleOther honours robots.txt. To disallow it site-wide:
User-agent: GoogleOther Disallow: /
This affects only GoogleOther, not Googlebot, so your Search crawling is unchanged. robots.txt is a request honoured by compliant crawlers, not an access-control boundary.
How it appears in analytics and logs
A request carrying the GoogleOther token is Google fetching for a non-Search purpose, not building the Search index. Treat it as a distinct bot event; allowing or blocking it does not change Googlebot's Search behaviour.
Diagnostic use case
Identify GoogleOther in logs by its token and set robots.txt policy for Google's non-Search crawling separately from Googlebot.
What WebmasterID can help detect
WebmasterID classifies GoogleOther server-side by its token and surfaces it separately from Googlebot on the bot-intelligence surface, so you can distinguish Google's non-Search crawling from Search crawling.
Common mistakes
- Assuming a GoogleOther rule changes Googlebot Search crawling — it does not.
- Treating GoogleOther as the Search crawler.
- Matching a full version string instead of the stable GoogleOther token.
- Counting GoogleOther crawl hits as human traffic.
Privacy and accuracy notes
Detection uses only the request user-agent token. No human identity is involved — a crawler is not a person. WebmasterID records the crawl as a bot event, separate from human analytics.
Frequently asked questions
- Is GoogleOther the same as Googlebot?
- No. Googlebot crawls to build the Search index. GoogleOther is a generic crawler for non-Search purposes such as research and development. They are separate tokens and are controlled independently in robots.txt.
Related pages
- Google-Extended — Google AI training control
Google-Extended is not a crawler or a user-agent string. It is a robots.txt token that lets site owners control whether their content is used to improve Google's generative AI models such as Gemini and Vertex AI. Googlebot continues to crawl for Search normally regardless of the Google-Extended setting.
- Separating AI crawler and search-bot traffic
AI crawlers and classic search bots arrive together but serve different purposes, honour different controls, and deserve different policies. Separating them in logs — by token, not by a generic bot flag — lets you allow Googlebot for Search while setting independent rules for GPTBot, ClaudeBot, and others. Mixing them produces misleading totals and the wrong policy decisions.
- Web crawlers reference
Reference for Google and other crawler tokens and how they appear.
Sources and verification notes
- Google — overview of Google crawlersDocuments GoogleOther as a generic, non-Search crawler separate from Googlebot.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.