Google-Safety — abuse/malware crawler
Google-Safety is the user agent Google uses for abuse-related crawling, such as malware detection tied to specific products. Google documents that, because of its safety purpose, it is not bound by robots.txt rules in the way ordinary crawlers are.
What this means
Google-Safety is the user agent Google associates with abuse-focused crawling, such as malware scanning connected to specific Google products. Its purpose is protective rather than for building the Search index.
The important distinction is its relationship to robots.txt: Google documents that, because Google-Safety serves a safety function, it is not bound by the usual robots.txt rules. That design exists so that a malicious site cannot hide from safety scanning by disallowing crawlers.
Why robots.txt does not stop it
For ordinary crawlers, robots.txt is a request that compliant bots honour. Google documents Google-Safety as an exception tied to abuse protection, so a Disallow rule will not necessarily stop it. This is intentional: safety scanning that could be switched off via robots.txt would be ineffective. Treat Google-Safety hits as security activity, not as indexing pressure on your crawl budget.
- User-agent token: Google-Safety
- Purpose: abuse and malware-related scanning
- Documented to ignore robots.txt for safety reasons
How it appears in analytics and logs
A request carrying the Google-Safety token is Google performing abuse or malware-related scanning, not indexing. Google documents that this user agent can ignore robots.txt rules because of its safety function, so seeing it despite a Disallow is expected, not a malfunction.
Diagnostic use case
Recognise Google-Safety hits as security/abuse scans rather than indexing, and understand why a robots.txt Disallow does not stop them.
What WebmasterID can help detect
WebmasterID classifies Google-Safety server-side as search-infrastructure activity and shows it separately from human traffic, so safety-scan fetches are visible without log parsing.
Common mistakes
- Expecting a robots.txt Disallow to stop Google-Safety — Google documents it does not for safety scans.
- Mistaking safety scans for indexing crawl that consumes crawl budget.
- Trusting the user agent without verifying against Google's methods.
Privacy and accuracy notes
Identification uses the user agent plus Google's verification methods — no human identity. WebmasterID records Google-Safety as a bot event, separate from human analytics.
Related pages
- Googlebot Smartphone — Google's mobile-first crawler
Googlebot Smartphone is the mobile user-agent variant of Googlebot and, under mobile-first indexing, Google's primary crawler for most sites. It uses the Googlebot robots.txt token and can be verified through reverse DNS and Google's published crawler IP ranges.
- How to verify Googlebot
The Googlebot user agent is widely spoofed, so a request claiming to be Googlebot should be verified, not trusted. Google documents two methods: a reverse-DNS check that resolves into googlebot.com or google.com confirmed by a matching forward lookup, and matching the source IP against Google's published crawler IP ranges.
- Website observability
See unusual and infrastructure crawler activity in context.
Sources and verification notes
- Google — Google crawlers (user agents) overviewDocuments Google-Safety and that it can ignore robots.txt for safety reasons.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.