Incrementality Testing

Attribution Shows Correlation. Incrementality Shows Causation.

Most marketing measurement tells you which touchpoints were present before a conversion. Incrementality testing tells you which touchpoints actually caused the conversion. That distinction is the difference between knowing that paid search appeared in 60% of closed deals and knowing whether those deals would have closed without paid search. The first is a correlation. The second is a causal answer — and it is the only answer that should drive budget decisions.

A significant portion of marketing budget goes to channels that capture demand that would have converted anyway. Incrementality testing reveals exactly how much of your spend is truly incremental versus simply taking credit for organic demand.

How Incrementality Testing Works

The methodology is borrowed from clinical trials: expose one group, hold back another, and measure the difference. In practice, you split a target audience into a test group (sees the campaign) and a holdout group (does not). If the test group converts at a meaningfully higher rate, the campaign is driving incremental outcomes. If both groups convert at similar rates, your spend is capturing demand — not creating it.

Component	What It Looks Like	Why It Matters
Test group	Receives the marketing treatment	Shows total conversions with the campaign
Holdout group	Receives no treatment	Shows baseline conversions without the campaign
Incremental lift	Difference between groups	The true causal impact of the campaign
Statistical significance	Confidence that the lift is real	Prevents acting on noise

The key requirement is a sufficiently large sample size and a clean holdout. If your holdout group is exposed to the campaign through other channels, the test is contaminated and the results are unreliable.

Where Incrementality Testing Fits in the Measurement Stack

The strongest measurement programs use three methods together: multi-touch attribution for tactical optimization, marketing mix modeling for strategic allocation, and incrementality testing for causal validation. Each answers a different question. MTA tells you which touchpoints are associated with conversions. MMM tells you how to allocate budget across channels at the portfolio level. Incrementality tells you whether a specific investment is actually driving net-new outcomes.

19.9% of US marketers say incrementality testing best identifies business value drivers, trailing MMM at 30.1% but ahead of third-party MTA at 11.7% (EMARKETER/Snap Inc., 2024). Adoption is growing because the method answers the question CMOs care about most: "What would happen if we stopped spending on this channel?"

When to Run Incrementality Tests

Run them when the stakes are high enough to justify the holdout cost. Holding back a segment of your audience means you are deliberately not marketing to them — which means potential lost revenue during the test period. That trade-off makes sense for high-spend channels where even a small efficiency improvement translates to significant budget savings. It does not make sense for low-spend experiments where the marketing ROI answer matters less than the learning speed. Start with your largest channel or the one where you have the most uncertainty about true incremental impact.