What is an incrementality experiment?
An incrementality experiment aims to measure the true value or effectiveness of a specific marketing channel or tactic. Its purpose is to determine the amount of conversions or revenue driven solely by a specific marketing campaign, independent of unreliable and sometimes biased 3rd party tracking solutions. For this reason, incrementality experiments are often referred to as the gold standard of marketing measurement.
Do you need help future-proofing your marketing measurement in a privacy-safe way?
Geo-based vs user-level incrementality experiment
All incrementality experiments are based on the same core concept, where the target audiences are divided into test and control groups. The test group receives marketing, while the control group does not. The conversions of these two groups are then compared to understand the incremental or true value of the specific marketing tactic.
For Geo Experiments (also referred to as Geo Lift Tests), these two groups are created based on geographical location like region, city or zip code. For user-level experiments, the two groups are created by randomizing the audience on an individual level (like traditional A/B tests).
Geo Experiments: Pros & cons
Pros
Measure cross-channel incrementality (like incrementality of Meta + Google Search) Measure incrementality of “untrackable” channels like TV, Out-of-home or Audio. Future proof: no reliance on sensitive IDs or other trackers Fully transparent and replicable test results Low setup cost compared to other incrementality measurements (e.g. MMM)
Cons
Requires expertise to properly set-up and execute Usually takes 2-4 weeks before results can be interpreted
User-level experiments: Pros & cons
Pros
Can detect incrementality in smaller audiences Results can sometimes be available within 1 week
Cons
Dependence on ad networks. No way to validate or replicate results Tracking limitations makes this type of measurement harder execute and trust Unable to measurement cross-channel lift Unable to measure “untrackable” channels
Geo Experiments has increased in popularity over the last few years as a result of the privacy changes enforced by Apple and regulators across the world which has increased the complexity of executing user-level experiments properly.
Do you need help future-proofing your marketing measurement in a privacy-safe way?
Not all Geo Experiments are created equal
One of the critical components in executing a trustworthy Geo Experiment is the test design. How do we create two unbiased and similar groups that account for different user-behavior across different parts of the test market? At Twigeo, we use a unique approach based on combining modern machine learning, Bayesian framework and Synthetic Control Method (SCM) to build a synthetic control unit based on the control group.
The use of SCMs (originally designed for evaluating policy changes) in marketing measurement was pioneered roughly 5 years ago by PhD researchers. At Twigeo, our Data Science team has been building on top of these innovations to create an experiment framework fully tailored to the modern marketing landscape. In 2022, we partnered up with Meta to collaborate with their Marketing Science team to further develop our Geo Experiment toolkit, and have since become one of their selected measurement partners.
Over the last few years, our approach to Geo experiments has helped clients:
- Figure out the true incrementality of their Meta and TikTok campaigns
- Understand whether or not their Google Search brand terms are cannibalizing on organic traffic
- Understand how effective online and offline branding campaigns are at driving up demand that leads to bottomline conversions and revenue
Why you should use Twigeo incrementality studies
At Twigeo we have vast experience in the field of incrementality within marketing. We are pioneers who constantly seek new ways to evolve, challenge and explore this area. Thanks to that we are a trusted and certified Meta partner who execute experiments for their selected clients. Do you need help future-proofing your marketing measurement in a privacy-safe way? Get in touch below!
