Put a Wilson interval beside a small retention estimate

A retention percentage hides how much information produced it. This lab adds a two-sided Wilson interval to a binomial proportion using a fixed 1.96 normal quantile. The interval is a statistical teaching aid, not a benchmark and not a substitute for checking how the cohort was sampled.

Define the calculation before using it

Count one eligible return outcome per independent install and preserve the total sample. The score-interval formula adjusts the centre and width using the sample size, observed proportion and squared quantile. The supplied artifact shows the arithmetic rather than rounding a small cohort to a confident whole percentage. For clustered or dependent observations, this model is not the correct uncertainty calculation.

Work through the synthetic example

In the synthetic example, 20 of 100 eligible installs return, giving a 20% point estimate. The displayed lower and upper bounds are approximately 13.34% and 28.88%. They describe uncertainty under the binomial model, not the probability that a future campaign will land in the interval. The sensitivity view changes the denominator while preserving the entered return count.

Use the artifact and preserve its assumptions

Open the editable calculator to change the inputs and inspect the sensitivity view. The CSV records synthetic inputs and expected outputs; the JSON fixture keeps the equations available for reproduction. These calculations have been checked against the stated example. No measured campaign data is included.

The sensitivity rows vary only n by 20% below and above the entered value. They are scenarios, not confidence limits or a forecast distribution. A row outside the model’s constraints is labelled rather than turned into a plausible-looking result. Save the chosen inputs with the decision so another reader can distinguish a changed assumption from a changed formula.

Evidence and limits

NIST discusses confidence intervals for a binomial proportion and cautions about approximation with small samples. Our worksheet states its assumptions; a descriptive rate or count contrast alone does not establish statistical certainty. See Confidence intervals for a proportion, especially “Confidence intervals; small-sample exact intervals”.

Use a pre-specified analysis plan for comparisons or repeated monitoring. Overlapping or non-overlapping intervals alone are not a complete test of the difference between two campaign treatments.

Background: UA metrics explained: CPI, ROAS, LTV and payback and Reading an MMP dashboard without fooling yourself. These existing articles provide context; the present calculation does not verify every archived claim.