Holdout-based attribution
Holdout-based attribution uses a randomized holdout — a group deliberately excluded from a campaign or channel — to estimate how much of a channel's credited conversions are genuinely incremental. By comparing the treated population against the holdout, it grounds attribution in a counterfactual rather than relying solely on observed click paths, which tend to over-credit channels that intercept already-converting users.
What this means
Touch-based attribution credits whatever interactions it observes, but it cannot see the counterfactual: would those conversions have happened without the channel? Holdout-based attribution adds that missing piece by randomly excluding a portion of users (or regions) from a channel and measuring the resulting change in conversions.
The measured incremental share can then be used to adjust the credit a rules- or data-driven model assigns. A channel that loses little when held out was largely harvesting demand; a channel whose holdout drops sharply was genuinely generating conversions.
Where it helps and its limits
Holdouts are especially valuable for channels prone to over-attribution — brand search, retargeting, and any tactic that re-engages users already on a path to convert. Combining holdout results with a touch model gives credit that is both granular (per touch) and grounded (calibrated to incrementality).
Limits: holdouts require giving up some media on purpose, enough scale to detect change, and a clean test window; they yield aggregate incremental shares, not per-impression truth. Treat holdout-based attribution as a calibration layer over a model, not a replacement for sound experimental design.
- Randomly excludes a channel/campaign to read incrementality
- Calibrates touch-model credit toward measured incremental share
- Best for over-credited channels like brand search or retargeting
How it appears in analytics and logs
If holding out a channel barely changes conversions, much of its attributed credit was non-incremental; a large drop confirms the channel was genuinely driving conversions.
Diagnostic use case
Apply a holdout to a channel you suspect is harvesting existing demand (such as brand search), then scale its attributed credit toward the measured incremental share.
What WebmasterID can help detect
WebmasterID's first-party conversion events can provide the outcome series for treated and held-out populations used to recalibrate attributed credit.
Common mistakes
- Assuming attributed credit equals incremental contribution.
- Holding out at too small a scale to see a real change.
- Using one holdout result as a permanent multiplier forever.
Privacy and accuracy notes
Holdouts compare aggregate group outcomes, not individual identities. This page is educational and not statistical or legal advice.
Related pages
- Incrementality testing: what attribution cannot tell you
Incrementality testing measures the lift a channel actually causes by withholding it from a control group and comparing outcomes. It answers the question every attribution model dodges: would this conversion have happened anyway? It is causal where attribution is merely correlational, but it requires deliberate experiment design.
- Geo experiments for measurement
A geo experiment divides geographic regions into a treatment group (which sees a media change) and a control group (which does not), then compares outcomes between them. Because assignment is at the region level rather than the user level, geo experiments measure incremental effect without needing cookies, device IDs, or per-person attribution — making them a privacy-resilient complement to touch-based models.
- Attribution vs incrementality vs MMM
Attribution, incrementality testing, and marketing-mix modeling (MMM) are three distinct measurement approaches often confused. Attribution distributes credit across observed touches; incrementality experiments measure causal lift versus a control; MMM uses aggregate, often top-down regression on spend and outcomes. They answer different questions and should be used together, not treated as interchangeable.
- Attribution analytics
Calibrate attributed credit with holdout outcomes.
Sources and verification notes
- Google Ads Help — About Conversion lift (holdout experiments)Describes randomized holdout groups used to estimate incremental conversions.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.