Confounding variables in conversion
A confounding variable is a third factor that affects both the thing you changed and the outcome you measured, producing a spurious association. Confounders are why 'we shipped X and conversions rose' is weak evidence — a campaign, a season, or a price change could be the real cause. Randomised experiments neutralise confounders by design. This page explains the concept and the defence.
What a confounder is
A confounder sits upstream of both the intervention and the outcome. If a holiday boosts both the likelihood of seeing your new banner and the likelihood of buying, the banner and sales correlate even if the banner did nothing. The confounder manufactures the link.
Why randomisation is the fix
Random assignment makes the treated and untreated groups equivalent on average across all confounders — known and unknown — so any remaining difference is attributable to the change. This is the core reason A/B tests beat observational before/after analysis, where confounders run free.
- Confounder affects both cause and outcome
- Creates correlation without causation
- Randomisation balances confounders across arms
When you can't randomise
Sometimes a true experiment isn't possible. Then you must reason explicitly about confounders — adjust for the ones you can measure and stay humble about the ones you can't. But adjustment is never as clean as randomisation, and unmeasured confounders remain a standing threat to any observational conclusion.
How it appears in analytics and logs
A correlation between a change and an outcome can be entirely a confounder's doing. Only a comparison that holds other factors equal — usually randomisation — isolates the change.
Diagnostic use case
Before crediting a change for a metric move, list what else changed at the same time; a confounder can fully explain the result without the change doing anything.
What WebmasterID can help detect
WebmasterID's first-party context (campaign, referrer, time) helps you spot co-occurring factors that could confound a before/after claim and push you toward a controlled test.
Common mistakes
- Crediting a change while a campaign or season also shifted.
- Treating observational before/after as if it were an experiment.
- Assuming you have adjusted for confounders you never measured.
Privacy and accuracy notes
Identifying confounders is a reasoning exercise over aggregate context, not a need for personal data. Coarse first-party context is enough to spot likely confounders.
Related pages
- Simpson’s paradox in experiments
Simpson's paradox is when an effect that holds within every subgroup reverses or vanishes once the subgroups are pooled. In experiments it appears when the mix of traffic differs between arms — so the aggregate is driven by composition, not the change. It is a vivid reason to check segments and to ensure arms are comparable. This page explains how it arises and how to avoid being fooled.
- Control and variant in experiments
In an experiment the control is the existing version that acts as the baseline, and the variant is the version carrying the one change you are testing. Comparing the two only yields a clean answer when assignment is random and the variant differs from the control in exactly one way. Multiple variants are possible but each must be isolated.
- Interaction effects between changes
An interaction effect occurs when the combined impact of two changes is not simply the sum of their individual impacts — one change alters how the other performs. Interactions matter when several experiments run on the same page at once, and they are the core reason multivariate testing exists. This page explains interactions and how concurrent tests can collide.
- Regression to the mean in tests
Regression to the mean is the statistical tendency for an extreme measurement to be closer to the average on the next observation. In experimentation it explains why a page picked because it converted unusually well often 'declines' afterward, and why early test readings overstate effects. Recognising it prevents crediting a change for a return to normal. This page explains the mechanism.
Sources and verification notes
- Wikipedia — ConfoundingDefinition and the role of randomisation.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.