curl, wget and script user agents
Command-line and library HTTP clients send a default user agent that names the tool: curl/x.y, Wget, python-requests, Go-http-client, and similar. These are scripts, not browsers, and seeing them is normal. This page explains the patterns and how to treat them without over- or under-reacting.
Default tool user agents
Command-line tools and HTTP libraries ship with a default user agent that names the tool and its version: curl uses a curl/x.y form, Wget names itself, the Python requests library sends a python-requests token, and Go's HTTP client sends a Go-http-client token.
These are not browsers and make no claim to be. They are the honest default of a scripted client, which is why a clearly named tool UA is easy to categorise.
- curl/x.y, Wget — command-line clients
- python-requests, Go-http-client — library defaults
- Names the tool and version, makes no browser claim
Benign or not is about behaviour
Seeing tool user agents is normal: deploy scripts, webhooks, health checks, integrations, and your own code all use them. A tool UA is not inherently malicious.
Whether scripted traffic is acceptable depends on what it does — request rate, paths, and whether it honours your rules — not on the user-agent string. Note that a default tool UA can also be overridden, so a custom string does not rule out a script.
How it appears in analytics and logs
A user agent naming a tool like curl, Wget, python-requests, or Go-http-client is a scripted client, not a browser. It is automation; whether it is benign depends on behaviour, not the string itself.
Diagnostic use case
Recognise scripted and library HTTP clients by their default tool user agents so they are counted as automation rather than as browser visits.
What WebmasterID can help detect
WebmasterID recognises common tool and library user agents server-side and classifies them as automation, separate from human analytics, with unknown clients kept in an honest 'other' bucket.
Common mistakes
- Treating every curl or python-requests hit as an attack.
- Counting scripted-client requests as human browser visits.
- Assuming a script must keep its default UA — it can be changed.
Privacy and accuracy notes
Tool user agents describe software, not a person. WebmasterID records scripted-client requests as bot events, never as human profiles.
Related pages
- Empty or missing user-agent strings
The User-Agent header is not mandatory, so some requests arrive with an empty string or no header at all. This usually points to a script, a misconfigured client, or an old device — not a specific identity. This page explains what a missing UA means and how to handle it without over-blocking.
- Headless browser user agents
Headless browsers run a real browser engine without a visible window, and are widely used for testing and scraping. Headless Chrome historically exposed a HeadlessChrome token, though automation can also wear an ordinary browser string. This page explains the patterns and why headless traffic is automation.
- Bot vs human traffic
Separate scripted and library clients from real browser visits.
Sources and verification notes
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.