Hook-level holdout for SLG war creatives during a season peak
Illustrative holdout lab for SLG war hooks during a season peak. Scales holdout rate to treatment size; does not invent lift.
Season peaks confound creative tests: every hook looks good when war week fills the funnel. This lab applies a scaled holdout contrast with illustrative counts. It is not a measured campaign lift.
Define the calculation before using it
Use eligible assigned people in each arm and one shared conversion definition (for example first IAP or alliance join). Divide holdout conversions by holdout population, multiply that rate by treatment population, then subtract from treatment conversions. Keep the rate difference visible beside the count difference. Do not mix reached users in one arm with assigned users in the other.
AppsFlyer’s attribution model reminds operators that install timestamps follow first launch under MMP rules. NIST discusses confidence intervals for a proportion; this sheet reports descriptive excess only.
Work through the synthetic example
| Input | Value | Status |
|---|---|---|
| Treated assignments | 2,000 | illustrative |
| Treated conversions | 160 | illustrative |
| Holdout assignments | 500 | illustrative |
| Holdout conversions | 30 | illustrative |
| Output | Result |
|---|---|
| Holdout rate | 0.0600 |
| Scaled holdout conversions at treatment size | 120.0000 |
| Descriptive excess conversions | 40.0000 |
| Rate difference (pp) | 2.0000 |
Facts versus limits
| Field | Evidence |
|---|---|
| Formula | excess = treated_conv - holdout_rate * treated_n |
| Causal lift | Not established by this sheet alone |
| Season-peak confound | Why the holdout is required; not quantified as a market average |
Download the lab CSV.
