Is your retargeting incremental?

Retargeting is the easiest channel in a UA stack to talk yourself into, because the attribution model almost flatters it by design. A lapsed player who was always going to reopen the app sees a retargeting ad first, taps it out of habit or coincidence, and the MMP credits the reopen to the campaign. The ratio looks excellent. Nothing incremental necessarily happened. That is the whole problem in four lines, and the rest of this piece is about telling the two apart before the budget renews.

Why retargeting is structurally prone to this

Retargeting campaigns, by definition, go after people who already have a relationship with the app, which means the baseline probability of them returning without any ad at all sits far above anything a cold acquisition audience would show. Standard attribution cannot see that baseline; it measures only what happened after exposure, never what would have happened without it.

A channel with a naturally high organic-return rate will show strong last-touch performance whether or not the ads do anything, which is precisely the scenario incrementality testing exists to catch.

Holdout design for re-engagement

Geo-lift does not transfer cleanly here. The standard design withholds an entire region from all spend, but a retargeting audience is not a place. Behaviour defines it: lapsed players, high-value dormant accounts, cohorts no map draws a line around. Use a user-level holdout instead. Randomly assign a share of the eligible retargeting audience, typically ten to twenty percent depending on audience size, into a suppressed group that sees no retargeting ads for the test duration, while the rest of the audience runs as normal. Then compare reopen rates and subsequent revenue between the two groups over a matched window.

Lock the audience definition before randomisation and apply it identically to both groups, because any leakage will quietly contaminate the test: one suppressed user pulled back in through a lookalike audience built on the same behavioural signal muddies the read. Run it long enough to capture the platform's natural reopen tail. Retargeting audiences carry players who come back on their own after weeks of dormancy whether or not an ad ever reached them, so a short window inflates the campaign's apparent effect by closing before that natural return would have shown up on both sides anyway.

The ghost-ads approach

Where a network supports them, ghost ads offer something cleaner than a fully suppressed group. Some networks call them public service announcement holdouts. Rather than withholding ads entirely, the platform runs the auction and the bidding exactly as normal for the holdout group but serves a neutral placeholder in place of the actual retargeting creative, so the holdout sees the same ad load and the same delivery mechanics as the test group, which isolates the creative's effect rather than tangling it up with a difference in exposure frequency. Incrementality tests you can afford covered this method for acquisition-side holdouts, and the logic transfers straight across to retargeting, with more value here than there: re-engagement is exactly the setting where exposure-frequency confounds inflate an apparent effect.

Compare reopen rate and per-user revenue across the targeted and suppressed groups once the test period ends, but read the confidence interval before you draw any conclusion, not just the headline gap. Retargeting audiences run smaller than acquisition audiences, which means a user-level holdout on a modest re-engagement segment has the same power problem a geo-lift test does, and produces a result too noisy to act on even when the point estimate looks encouraging. If the audience cannot power a confident read inside a reasonable window, pool several similar campaigns into one holdout design rather than testing each in isolation.

Where retargeting genuinely lifts outcomes

The cohorts where retargeting shows a real, defensible lift share a pattern. Those players have a specific, addressable reason to have lapsed rather than simply drifting away: a stalled progression point, an expired offer, a content update they never saw. The ad's job there is to tell the player something they did not already know, not to remind them the app exists. High-value dormant accounts behave the same way. Players who spent meaningfully before going quiet tend to show genuine incremental lift when the creative points at a concrete re-entry hook rather than a generic come-back message, because that segment's return decision responds to a reason in a way a low-value lapsed player's does not.

Broad, low-value lapsed audiences on generic creative are the segment most likely to deliver attribution-flattering, incrementality-negative results. If your retargeting spend sits there rather than in a narrower, hook-driven segment, point the first holdout test at it before the budget gets renewed on the strength of a last-touch number that was never measuring what it looked like it was measuring.

Related archive reading

These articles provide related context and remain subject to their stated review status.

• Incrementality tests you can afford: geo holdouts, ghost ads, and what breaks
• Re-engaging lapsed players: a campaign design