Dead-letter queue for events
When an event cannot be processed — it fails validation, throws in a transform, or repeatedly errors — a dead-letter queue (DLQ) holds it instead of discarding it. The DLQ preserves the data for inspection and replay, and its depth is a live signal that something upstream broke. This page explains how a dead-letter queue protects analytics completeness and surfaces failures.
Catch instead of drop
Pipelines must decide what to do with an event they cannot process. Dropping it loses data silently and hides the failure. A dead-letter queue instead routes the failed event to a side store, with enough context (the error, the original payload) to diagnose it. The data is preserved and the failure becomes observable.
The DLQ turns an invisible loss into a visible, actionable backlog.
- Failed events go to a side store, not the void
- Context (error + payload) supports diagnosis
- Depth is a live health signal
Working the queue
Alert on DLQ depth and growth rate, because a rising queue is often the earliest sign of a schema change or upstream bug. Inspect samples to find the root cause, fix it, then replay the dead-lettered events through the corrected pipeline — using idempotency keys so replay does not double-count. Set retention on the DLQ so failed events are not held indefinitely against your privacy rules.
The DLQ is where validation failures land and where backfill-style replay puts them back.
How it appears in analytics and logs
A growing dead-letter queue means events are failing processing — a schema change, bad data, or a bug — before any report shows the gap.
Diagnostic use case
Avoid silently losing unprocessable events by routing them to a dead-letter queue for inspection and later replay.
What WebmasterID can help detect
WebmasterID can quarantine events that fail validation so they are inspectable rather than lost, keeping the human total trustworthy.
Common mistakes
- Dropping unprocessable events instead of dead-lettering them.
- Not alerting on dead-letter queue growth.
- Replaying without idempotency keys, causing double counts.
Privacy and accuracy notes
Dead-lettered events may contain data subject to retention and deletion rules; govern the DLQ accordingly. This page is educational, not legal advice.
Related pages
- Server-side event validation
Server-side collection gives one place to validate every event before it is stored or forwarded. Checks fall into shape (does it match the tracking plan), type (are values the right kind), and plausibility (is the sequence possible). Rejecting or quarantining failures keeps malformed and fabricated data out of downstream tables. This page describes how server-side event validation gates an analytics pipeline.
- Idempotency and dedup keys
Distributed pipelines deliver at least once, so the same event can arrive twice from retries, replays, or backfills. An idempotency key — a stable, unique identifier per event — lets the pipeline recognize a repeat and keep exactly one copy, so re-processing does not inflate counts. This page explains idempotency and de-duplication keys and how to choose one that survives the whole pipeline.
- ETL and pipeline failures
Analytics often flows through an ETL/ELT pipeline that extracts events, transforms them, and loads them into reporting tables. A failure at any stage — a timed-out extract, a transform exception, a half-written load — leaves data partial or stale, and if the failure is silent it reads as a genuine traffic dip. This page explains ETL failure modes and how to tell a pipeline gap from a real one.
- Website observability
Alert on dead-letter queue depth and growth.
Sources and verification notes
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.