Hook-level holdout for SLG war creatives during a season peak
Analysis
By UA Ledger staff — 1 min read
Labelled synthetic holdout lab for SLG war hooks during a season peak. Scales holdout rate to treatment size; does not invent lift.
Season peaks confound creative tests: every hook looks good when war week fills the funnel. This lab applies a scaled holdout contrast with labelled synthetic counts. It is not a measured campaign lift.
Define the calculation before using it
Use eligible assigned people in each arm and one shared conversion definition (for example first IAP or alliance join). Divide holdout conversions by holdout population, multiply that rate by treatment population, then subtract from treatment conversions. Keep the rate difference visible beside the count difference. Do not mix reached users in one arm with assigned users in the other.
AppsFlyer’s attribution model reminds operators that install timestamps follow first launch under MMP rules. NIST discusses confidence intervals for a proportion; this sheet reports descriptive excess only.
Work through the synthetic example
| Input | Value | Status |
|---|---|---|
| Treated assignments | 2,000 | labelled synthetic |
| Treated conversions | 160 | labelled synthetic |
| Holdout assignments | 500 | labelled synthetic |
| Holdout conversions | 30 | labelled synthetic |
| Output | Result |
|---|---|
| Holdout rate | 0.0600 |
| Scaled holdout conversions at treatment size | 120.0000 |
| Descriptive excess conversions | 40.0000 |
| Rate difference (pp) | 2.0000 |
Facts versus limits
| Field | Evidence |
|---|---|
| Formula | excess = treated_conv - holdout_rate * treated_n |
| Causal lift | Not established by this sheet alone |
| Season-peak confound | Why the holdout is required; not quantified as a market average |
Download the lab CSV.
Draft prepared 20 September 2026. Synthetic teaching example; human editorial review pending.
Featured
Related posts
creative strategy
measurement
·1 min read
Cohort quality: D1 retention vs D7 payer rate divergence lab
creative strategy
measurement
·1 min read
Creative concept half-life by F2P genre (operator method, labelled synthetic)
creative strategy
measurement
·1 min read
Country mix shift: standardising F2P ROAS before declaring creative win
creative strategy
measurement
·2 min read
Creator code program launch checklist for F2P (MMP, finance, legal)
More from the Creative Strategy desk
creative strategy
·1 min read
UK ASA loot-box disclosure in ads and store listings
creative strategy
media buying
·1 min read
India real-money adjacency: what a casual F2P ad must not imply
creative strategy
media buying
·1 min read
Publish the 2026 festival calendar to UA 60 days out
creative strategy
media buying
·1 min read