Google CloudVertexBot
CloudVertexBot is a Google crawler that fetches website content for Vertex AI Agents when a site owner sets up that integration. Google documents it as a separate token, distinct from Googlebot's Search crawling and from Google-Extended's training-control token. It is owner-directed: it crawls sites at the request of the party building the agent.
What this means
CloudVertexBot is a Google crawler used to fetch website content for Vertex AI Agents. Google documents it among its crawlers as a user-triggered fetcher that crawls sites at the request of the owner building an agent — so it is tied to an integration rather than to general indexing.
It is separate from Googlebot, which crawls for Search, and from Google-Extended, which is a control token for Gemini-related training rather than a crawler that fetches pages itself. Treat all three as independent.
How CloudVertexBot identifies itself
CloudVertexBot uses the robots.txt user-agent token CloudVertexBot. Its user-agent string contains that token. Match on the stable token rather than a full version string.
Like other Google crawlers, it can be verified against Google's published methods such as reverse-DNS into Google's crawler domains and Google's published IP ranges. The user agent alone is a claim; verify where authenticity matters and do not invent ranges.
- robots.txt token: CloudVertexBot
- Fetches content for Vertex AI Agents at an owner's request
- Distinct from Googlebot and Google-Extended; verify via Google's methods
robots.txt considerations
CloudVertexBot honours robots.txt. To disallow it site-wide:
User-agent: CloudVertexBot Disallow: /
This affects only CloudVertexBot, not Googlebot's Search crawling and not the Google-Extended control. robots.txt is a request honoured by compliant crawlers, not an access boundary.
How it appears in analytics and logs
A request carrying the CloudVertexBot token is Google fetching content for a Vertex AI Agents integration, not building the Search index. Treat it as a distinct bot event; it does not change Googlebot's Search behaviour.
Diagnostic use case
Identify CloudVertexBot in logs by its token and understand that it crawls on behalf of a Vertex AI Agents setup, separate from Search and from Google-Extended controls.
What WebmasterID can help detect
WebmasterID classifies CloudVertexBot server-side by its token and surfaces it separately from Googlebot and Google-Extended on the bot-intelligence surface, so Vertex AI agent crawling is distinguishable from Search.
Common mistakes
- Confusing CloudVertexBot with Googlebot — it does not crawl for Search.
- Assuming a CloudVertexBot rule changes Google-Extended training control.
- Matching a version string instead of the stable CloudVertexBot token.
- Counting CloudVertexBot crawl hits as human traffic.
Privacy and accuracy notes
Detection uses only the request user-agent token. No human identity is involved — a crawler is not a person. WebmasterID records the crawl as a bot event, separate from human analytics.
Frequently asked questions
- Does CloudVertexBot affect my Google Search ranking?
- No. CloudVertexBot fetches content for Vertex AI Agents, not the Search index. Search crawling is handled by Googlebot under a separate token, and the two are controlled independently in robots.txt.
Related pages
- Google-Extended — Google AI training control
Google-Extended is not a crawler or a user-agent string. It is a robots.txt token that lets site owners control whether their content is used to improve Google's generative AI models such as Gemini and Vertex AI. Googlebot continues to crawl for Search normally regardless of the Google-Extended setting.
- GoogleOther — Google non-Search crawler
GoogleOther is a generic crawler Google uses for fetches not tied to building the Search index — for example internal research and development crawls. Google documents it as separate from Googlebot, with its own robots.txt token. Controlling GoogleOther does not affect Googlebot's Search crawling, and vice versa.
- Bedrockbot — Amazon Bedrock crawler
Bedrockbot is a crawler Amazon documents in association with Amazon Bedrock, used to retrieve web content for Bedrock features. Amazon lists it among its crawlers with a robots.txt token. It is distinct from Amazonbot; identify each by its own token and set policy separately.
- Web crawlers reference
Reference for Google and other crawler tokens and how they appear.
Sources and verification notes
- Google — overview of Google crawlers and fetchersDocuments CloudVertexBot as a user-triggered fetcher for Vertex AI Agents.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.