How to Measure Incrementality in Audio
Synthetic Control to Randomized User-Level Holdout Groups
Podcast advertising is growing, and so is the pressure to prove that it is working. With brands running campaigns across channels like paid social, search, CTV, radio, and podcasting, the question isn't just "are people converting?" It's "did our podcast ad actually cause that conversion?".
In advertising, incrementality measures how many people performed a desired action such as, visiting a website, installing an app, signing up for a trial, making a purchase, or completing another key conversion event, because they were exposed to an ad, and would not have otherwise.
At Podscribe, we understand that the preferences for advertisers are unique when it comes to measuring incrementality. To ensure the preferred methodology can be used in audio, Podscribe can measure incrementality through synthetic control groups, and randomized user-level holdout groups (both via ghost & PSA holdouts).
Need help selecting a methodology that works with your goals? Reach out to us!
Podscribe Incrementality Capabilities
Podscribe offers multiple approaches for measuring incrementality depending on campaign structure, inventory type, and advertiser goals.
1. Synthetic Control Groups
Instead of withholding ads from listeners, Podscribe creates a synthetic control group of similar listeners to the exposed group to estimate what conversions would have looked like without an ad exposure. Podscribe constructs these similarity groups based on audience demographics, geography, past listening behavior, and past purchase affinity.
Synthetic control group testing provides several advantages:
Requires no setup from advertisers across various campaigns regardless of buy type. It can be enabled with the flip of a switch.
No opportunity costs as it does not hold out any listeners from ad exposure
No need to pay for PSA ads
No need to work with a publisher to create a hold-out group
Synthetic control groups are the default method within Podscribe for computing control groups if other options (listed below) are not selected.
2. Randomized User-Level Holdout Groups
SmartServe, Podscribe's ad-serving layer, now enables randomized user-level holdout groups by plugging into your existing media buys via a simple VAST tag. By sitting between the buy and the publisher’s ad delivery in real time, SmartServe enables controls that traditional podcast ad serving often can’t, including the ability to create randomized listener holdout groups, experiment-safe audience partitioning, and real-time creative selection.
Podscribe SmartServe supports user-level randomized holdout groups for dynamically inserted ads, including impression-based buys and episodic buys that use dynamic insertion.
Randomized holdout groups are created directly within the ad server, allowing exposed and control audiences to be selected from the same eligible listener pool in real time. Advertisers can be highly confident that control listeners are as similar to exposed listeners as possible, helping produce more accurate and trustworthy incrementality results.
Listeners are assigned to:
An exposed (test) group that hears/sees the campaign
A holdout (control) group that does not
This methodology creates scientifically valid experiments that isolate the causal effect of podcast advertising. This approach also aligns closely with emerging industry standards around:
Randomized selection
Balanced audience composition
Frequency matching
Contamination prevention
Privacy-safe listener partitioning
While randomized user-level holdouts are generally preferred by data scientists, the main reasons a buyer would not always use SmartServe for such groups are:
Podscribe offers two types of randomized holdout testing:
2a. Ghost Holdouts
By randomly assigning listeners to a holdout group at the time of ad serving (or “treatment”), SmartServe can create a “ghost holdout group”, as the control group is constructed without any ad being served to these listeners. Because no replacement ad is served, ghost holdouts eliminate media cost while still enabling incrementality measurement.
Depending on the publisher ad server or DSP SmartServe runs on, there may be slight additional targeting filters applied to exposed listeners that ghost holdout listeners are not exposed to. This means ghost holdout testing can occasionally create slight differences between exposed and control groups depending on campaign targeting or filtering criteria applied during ad serving. This may reduce how closely the holdout audience mirrors the exposed audience, although in most cases this difference is minimal, and easily viewable in the Podscribe dashboard for each campaign.
For campaigns where audience similarity between test and control groups is critical, PSA-based holdouts offer a stronger match, which is why some advertisers choose to invest in PSA testing despite the added cost.
2b. PSA Control Groups
Instead of delivering no ad experience at all, holdout listeners hear a neutral PSA (public service announcement) or a non-converting filler creative. Incremental lift is then measured through an A/B test comparing performance between listeners exposed to the PSA and those exposed to the actual ad.
PSA-based incrementality testing can also help maintain pacing consistency, normalize ad load and reduce listener experience differences, making the two groups as comparable as possible.
You can now upload and designate a PSA creative for holdout audiences within Podscribe SmartServe.
3. Publisher-created Holdout Groups
Podscribe also supports incrementality measurement for publisher-created holdout groups.
The publisher can independently create exposed and control audiences, while Podscribe measures conversion behavior through pixel-based attribution and incrementality analysis.
Podscribe can then compare attribution and conversion performance between the exposed and control groups to estimate incremental lift and campaign impact. To help improve test integrity, Podscribe can also analyze audience overlap between groups and may be able to identify and remove overlapping listeners.
Which methodology should you use?
While randomized holdouts remain the strongest causal methodology, synthetic controls can provide valuable incremental insights when audience/campaign budget constraints exist. Not sure what methodology would work best for your goals? We can help — book a demo and we’ll walk you through it.
The Future of Podcast Measurement
As podcast advertising continues to mature, advertisers are demanding the same level of accountability and rigor they expect from every other major media channel.
“Podcast advertising has reached a point where advertisers expect the same level of accountability they get from other channels like Meta and Google. Our goal with Podscribe Incrementality is to help advertisers move beyond directional attribution and better understand true causal lift by bringing better measurement and transparency to podcasting.” — Pete Birsinger, CEO/Founder, Podscribe
Podscribe is proud to help lead that shift toward more transparent, standardized, and actionable podcast measurement.
Additional relevant resources:
Need help getting started or want a quick walkthrough?
Reach out anytime at partnerships@podscribe.com or request a demo.
