Minimum detectable effect (MDE)
The minimum detectable effect (MDE) is the smallest change in your metric that an experiment is set up to detect reliably. It is an input you choose, not an output: a smaller MDE demands more traffic. Setting the MDE to the smallest difference that would actually matter to the business keeps experiments honestly sized.
What this means
The MDE is the smallest effect size — for example, a given relative lift in conversion rate — that you design the experiment to be able to detect with your chosen power and significance. It is decided before the test. A larger MDE (you only care about big changes) needs less traffic; a smaller MDE (you want to catch subtle changes) needs much more.
Setting it honestly
Choose the MDE from what would actually change a decision, not from wishful thinking. Setting an unrealistically large MDE makes the test cheap but blind to the modest improvements most changes produce. Setting a tiny MDE may demand more traffic than you can ever gather. The MDE, the baseline rate, the significance level, and the power together determine the sample size.
Report the MDE alongside any null result: 'no difference' really means 'no difference as large as our MDE'.
- MDE is chosen up front, not measured
- Smaller MDE means much larger sample size
- A null result is relative to the MDE you set
How it appears in analytics and logs
The MDE bounds what an experiment can find. If the true effect is smaller than your MDE, the test will usually report 'no significant difference' even though a small effect exists.
Diagnostic use case
Set the MDE to the smallest improvement worth acting on, then size the experiment to detect it — rather than hoping to catch any difference at all.
What WebmasterID can help detect
WebmasterID's first-party baseline conversion rate is the anchor you set an MDE relative to.
Common mistakes
- Setting an unrealistically large MDE that blinds the test to real change.
- Treating the MDE as an output instead of a design choice.
- Reading a null result without stating the MDE behind it.
Privacy and accuracy notes
The MDE is a planning parameter over aggregate rates; no personal data is involved. WebmasterID provides the baseline rate it builds on.
Related pages
- Sample size in experiments
Sample size is the number of subjects per arm an experiment needs to detect a chosen effect with acceptable error rates. It is computed in advance from the baseline rate, the minimum effect worth detecting, and the false-positive and false-negative rates you accept. Too small and you miss real effects; running until 'it looks good' inflates false positives.
- Statistical significance and p-values
A result is 'statistically significant' when it would be unlikely if there were really no effect. The p-value is the probability of seeing data at least as extreme as yours assuming the null hypothesis is true — it is not the probability the variant is better, and not a measure of how big the effect is. Significance and practical importance are different questions.
- A/B testing fundamentals
An A/B test randomly assigns visitors to a control (A) or a variant (B), shows each group one version, and compares a pre-chosen metric. Random assignment is what lets you attribute a difference to the change rather than to who happened to see it. The discipline is in deciding the metric and sample size before you start, not after you peek at the numbers.
- WebmasterID docs
Anchor an MDE to your baseline rate.
Sources and verification notes
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.