HTTP 304 Not Modified and crawl efficiency
304 Not Modified is the response to a conditional request when the resource has not changed since the client last fetched it. The server returns no body, so the crawler reuses its cached copy. Correct conditional-request support with ETag or Last-Modified saves bandwidth and crawl budget.
What 304 means
304 Not Modified is returned for a conditional GET when the resource has not changed. A client includes validators — an If-None-Match header carrying a previously received ETag, or an If-Modified-Since header with a timestamp — and if nothing changed the server responds 304 with no message body.
The client then serves its cached copy. No content is re-transferred.
Why it helps crawling
Crawlers refetch pages to keep their index fresh. If every refetch transferred the full page, large sites would burn bandwidth and crawl budget on unchanged content. Conditional requests let a crawler confirm freshness cheaply: a 304 is small and fast, so the crawler can spend its budget discovering and refreshing pages that actually changed.
To enable this, serve accurate ETag and/or Last-Modified headers and honour If-None-Match / If-Modified-Since.
- ETag + If-None-Match — content fingerprint validation
- Last-Modified + If-Modified-Since — timestamp validation
- 304 has no body — the cached copy is reused
Operator checklist
Make sure your server emits stable ETag or Last-Modified headers and returns 304 when validators match. Avoid changing ETags when content has not changed. Check that unchanged pages produce 304s rather than full 200 responses on recrawl.
How it appears in analytics and logs
A 304 means a crawler asked 'has this changed since my last copy?' and the server answered 'no'. Healthy 304 rates indicate efficient caching; the crawler avoids re-downloading unchanged resources.
Diagnostic use case
Reduce wasted crawling by serving 304 when content is unchanged, and confirm crawlers are sending and honouring conditional requests.
What WebmasterID can help detect
WebmasterID can surface the mix of 200 vs 304 responses crawlers receive, helping you see whether caching headers are letting crawlers skip unchanged pages.
Common mistakes
- Never emitting validators, so every recrawl is a full 200 download.
- Returning a body with a 304 (it must be empty).
- Generating a new ETag on every request even when content is identical.
Privacy and accuracy notes
Conditional-request status codes carry no personal data. WebmasterID reports these patterns for crawler traffic without exposing individual visitors.
Related pages
- HTTP 200 OK: what it means for crawlers
200 OK means the request succeeded and the server returned the resource. For crawlers it is the green light to process and potentially index a page. The subtle trap is the soft 404 — an error or empty page served with a 200 status, which wastes crawl budget and pollutes the index.
- Crawl budget waste: causes and fixes
Crawl budget is the finite attention a search engine spends on your site. It is wasted when crawlers spend it on low-value URLs — endless faceted combinations, parameter variants, soft 404s, and redirect chains — instead of your important pages. Reducing that waste helps key content get crawled.
- Website observability
See the status-code mix crawlers receive, including 304s.
Sources and verification notes
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.