/ 

Measure True Marketing Impact with Incrementality Experiments

Measure True Marketing Impact with Incrementality Experiments

Incrementality experiments is the gold standard of marketing measurement. Learn about the pros and cons of geo-based vs. user-level incrementality experiments from Twigeo's seasoned data science team, and start future-proofing your measurement.

What is an incrementality experiment?

An incrementality experiment aims to measure the true value or effectiveness of a specific marketing channel or tactic. Its purpose is to determine the amount of conversions or revenue driven solely by a specific marketing campaign, independent of unreliable and sometimes biased 3rd party tracking solutions. For this reason, incrementality experiments are often referred to as the gold standard of marketing measurement.

Do you need help future-proofing your marketing measurement in a privacy-safe way?

Geo-based vs user-level incrementality experiment

All incrementality experiments are based on the same core concept, where the target audiences are divided into test and control groups. The test group receives marketing, while the control group does not. The conversions of these two groups are then compared to understand the incremental or true value of the specific marketing tactic.

For Geo Experiments (also referred to as Geo Lift Tests), these two groups are created based on geographical location like region, city or zip code. For user-level experiments, the two groups are created by randomizing the audience on an individual level (like traditional A/B tests).

Incrementality Experiments

Pros & cons Geo Experiments

Pros

✅ Measure cross-channel incrementality (like incrementality of Meta + Google Search)
✅ Measure incrementality of “untrackable” channels like TV, Out-of-home or Audio.
✅ Future proof: no reliance on sensitive IDs or other trackers
✅ Fully transparent and replicable test results
✅ Low setup cost compared to other incrementality measurements (e.g. MMM)

Cons

❌ Requires expertise to properly set-up and execute
❌ Usually takes 2-4 weeks before results can be interpreted

Pros & cons user-level experiment

Pros

✅ Can detect incrementality in smaller audiences
✅ Results can sometimes be available within 1 week

Cons

❌ Dependence on ad networks. No way to validate or replicate results
❌ Tracking limitations makes this type of measurement harder execute and trust
❌ Unable to measurement cross-channel lift
❌ Unable to measure “untrackable” channels

Geo Experiments has increased in popularity over the last few years as a result of the privacy changes enforced by Apple and regulators across the world which has increased the complexity of executing user-level experiments properly.

Do you need help future-proofing your marketing measurement in a privacy-safe way?

Not all GEO Experiments are created equal

One of the critical components in executing a trustworthy Geo Experiment is the test design. How do we create two unbiased and similar groups that account for different user-behavior across different parts of the test market? At Twigeo, we use a unique approach based on combining modern machine learning, Bayesian framework and Synthetic Control Method (SCM) to build a synthetic control unit based on the control group.

The use of SCMs (originally designed for evaluating policy changes) in marketing measurement was pioneered roughly 5 years ago by PhD researchers. At Twigeo, our Data Science team has been building on top of these innovations to create an experiment framework fully tailored to the modern marketing landscape. In 2022, we partnered up with Meta to collaborate with their Marketing Science team to further develop our Geo Experiment toolkit, and have since become one of their selected measurement partners.

Over the last few years, our approach to Geo experiments has helped clients:

  1. Figure out the true incrementality of their Meta and TikTok campaigns
  2. Understand whether or not their Google Search brand terms are cannibalizing on organic traffic
  3. Understand how effective online and offline branding campaigns are at driving up demand that leads to bottomline conversions and revenue

Why you should use Twigeo incrementality studies

At Twigeo we have vast experience in the field of incrementality within marketing. We are pioneers who constantly seek new ways to evolve, challenge and explore this area. Thanks to that we are a trusted and certified Meta partner who execute experiments for their selected clients. Do you need help future-proofing your marketing measurement in a privacy-safe way? Get in touch below!

Let’s talk about how we can grow your business

Our team of experts are ready to chat and walk you through what we do.

Stay relevant!

We’ll send you the latest trends in growth marketing, analytics and creatives — but never spam.