In 2019 Zalando conducted an experiment that has put into crisis many established beliefs in digital marketing. They stopped showing retargeting ads to a group of loyal customers for six weeks, discovering that conversions have dropped only by 2%. This shows that those customers would still buy, regardless of ads. Yet, platform reports showed an apparent ROAS over 8x, a result that turned out to be an accounting illusion: 98% of turnover was basic, non-increative.
Why your dashboard ROAS can trick you
Standard attribution models, such as last-click, assign all the merit of the purchase to the last touchpoint, ignoring whether that campaign really caused the sale or was only present in the path. This leads to overestimate the effectiveness of campaigns such as retargeting or brand search.
| Model allocation | Operation | Main limit |
|---|---|---|
| Last-click | 100% of the credit at the last touchpoint | Does not distinguish cause from correlation, favors retargeting |
| First-click | 100% to the first touchpoint | Ignore jurturing and subsequent decisions |
| Linear | Fairly distributed credit | It takes equal weight for each interaction, rarely true |
| Data-driven (Google) | ML distributes pattern based credit | Measure correlation, non-causal |
Nobody answers the key question: “If the campaign had not existed, would those sales have taken place anyway?” That’s why we need an experimental approach.
Increality: definition and formula
Incrementality measures the causal effect of a campaign:
Incremental sales = Sales in the exposed group, Sales in the control group
The control group is identical to the one shown, except for the fact that it did not receive the campaign. The operating metric is the iROAS (incremental Return on Ad Spend):
.
If the platform ROAS is 6x but the iROAS is 1.8x, only 30% of the turnover attributed is really causal.
Three methods of measuring real increase
1. geo-lift test
It divides the market into two comparable geographical areas: one receives the campaign, the other does not. Compare sales growth to estimate the causal effect.
Example: TV campaign in Lombardy (test) vs Piedmont (control). After 4 weeks, Lombardy grows by 14%, Piedmont by 6%, therefore the estimated increase is 8%.
The Python code attached shows how to analyze data using a difference-in-difference approach, normalizing the series and calculating the iROAS.
Limits: The choice of regions must guarantee parallel historical trends; local events can distort results.
2. conversion lift study (Meta and google)
The platforms randomly divide the audience into testing and control. The test group sees ads, control does not. Conversions are compared to estimate the incrementality.
Advantages: user randomization, gold standard for causality.
Disadvantages: conflict of interest, often optimistic results, more reliable for prospecting than for retargeting.
3. probability score analysis (PSA)
Use machine learning to calculate the probability that a user has been exposed to the campaign. Compare similar users exposed and not exposed to estimate the effect.
Advantages: It works with observational data, useful when you can’t randomize.
Disadvantages: requires all relevant variables; hidden variables distort estimates.
How to calculate and interpret the iROAS
A simplified example shows how a platform ROAS of 6x can translate into a real 1.4x iROAS, with 77% of the turnover allocated that would come anyway. This radically changes the perception of effectiveness and guides decisions of budget reallocation.
Expected increase by channel
| Channel | Typical Increality | Reasons | ROAS platform |
|---|---|---|---|
| Brand Search | 10-30% | Users already in mind, organic conversion | 15-30x |
| Retargeting | 20-50% | High contamination by ready-made users | 4-8x |
| Social Ads Prospecting | 50-75% | New demand generated | 2-4x |
| Influencer & Awareness | 60-85% | Ex-novo question | 1-3x |
| Email to existing customers | 30-60% | Varies depending on the type of countryside | 3-15x |
Brand search: a case to review
Many believe that the search brand with ROAS 25x is pure gold. Actually, removing ad, organic traffic almost completely compensates. eBay has shown that the increase is often below 5%. Relocating budget from brand search to prospecting can improve results.
Increality and statistical significance
For reliable tests, appropriate samples are required: at least 50 regions or 4-6 weeks per geo-lift, 10,000 conversions per conversion lift, 1,000 users exposed and not for PSA.
Case study: italian retailer discovers 40% of the budget goal without real increases
A retailer found that 41% of the Meta budget was spent on segments with iROAS close to 1x, i.e. without generating incremental sales. By relocating the budget to new users and look-alike, it maintained its turnover and improved the acquisition cost and lifetime value.
Building a culture of incrementality
The real obstacle is cultural: Easy KPIs like ROAS look more performing. To change:
- Starts with a critical channel with rigorous tests, Presents budget reallocation as optimization, not critical, Integrates iROAS into regular dashboards
Incrementality and HADI cycles
Incrementality testing is an advanced component of the HADI framework to test methodically marketing assumptions.
The point measuring only ROAS without incrementality is how to evaluate a drug without control group: you don’t know what is causal. only knowing the iROAS you can pay for what you really caused, avoiding wasting budgets on sales that would happen anyway.
The best marketing managers do not stop at ROAS, but try to understand the iROAS. This is the discipline of making decisions under uncertainty, distinguishing between correlation and causality.
To learn more, please consult our form on statistical significance and testing.