Portfolio Theory for Creative: How Many Concepts to Run

Two creative concepts a month is under-testing. Twenty a month is usually not testing at all, just producing volume and crowning whatever performed best this week, no matter how thin the sample behind that result. Both mistakes share a root. Nobody ever set an explicit number for how many concepts a given budget can support with a statistically meaningful read, so the number defaults to whatever the production team happened to make.

Portfolio theory, borrowed loosely from how an investor sizes a basket of assets against a risk budget, beats either instinct as a frame. A creative portfolio has a budget; it has a minimum sample size each concept needs before its signal means anything; and it has a diminishing return on adding concepts once the budget can no longer give each one enough spend to clear that threshold. The job is to find the number of concepts that maximises the odds of turning up a genuine winner without diluting spend below the point where anyone could spot one.

The maths most teams skip

A concept needs enough impressions and installs to tell a real performance difference from noise, and that threshold doesn't shrink just because a team wants to test more concepts. Say a channel needs roughly 300 to 500 installs per concept to detect a meaningful difference in D7 retention with reasonable confidence. A $20,000 weekly test budget producing 2,000 installs at a blended CPI across the test set therefore supports four to six concepts a week at a genuinely readable sample. Not the twelve a production team might be capable of generating.

Running twelve concepts against that same $20,000 doesn't test twelve ideas. It tests twelve ideas badly, each one getting roughly 165 installs, well short of what it takes to separate a real winner from a lucky week. A team that picks a winner out of that set and scales it isn't scaling a proven concept; it's scaling whichever concept got the best luck that week, and there's a meaningful chance the actual best performer sat buried in noise at that sample size.

Sizing the portfolio to the budget, not the production pipeline

The correct order of operations inverts what most teams do by default. Most teams decide how many concepts the creative team can produce this sprint, then split the test budget across all of them. Invert it. Work out the minimum viable sample size for your primary success metric, divide the available test budget by the cost of reaching that sample per concept, and let the answer cap the concepts tested that cycle. Surplus concepts go into a queue for the following cycle instead of diluting the current one.

A worked example

Take a hypothetical hybrid-casual title with a $15,000 weekly test budget and a blended CPI of $3.00 across test placements, producing 5,000 installs a week. If the team's minimum viable sample per concept is 400 installs for a readable D3 retention signal, the budget supports 12 concepts a week at full statistical weight. So a creative team capable of 20 concepts a week should hold 8 back rather than force all 20 into the current cycle at 250 installs each, which lands below the threshold where anyone should believe the result.

The queue isn't wasted work.

A concept held back one cycle moves up the following week, refined against whatever early signal the current cycle throws off about hooks and pacing, which often makes for a stronger second-cycle test than firing everything at once.

When to widen the portfolio

The ceiling above isn't fixed forever. It moves with budget. It moves with CPI, and with how aggressive a minimum sample a team will accept for early-stage signal as against a confirmatory read. A team running a two-stage process, a lightweight first-pass filter at a lower sample threshold to cut a large concept set down to a shortlist, then a full statistical test on the shortlist alone, can responsibly test more raw concepts per cycle than the single-stage maths suggests, because that first stage never pretends to produce a final answer.

Applying it across channels, not just within one

The example above sizes a portfolio for one channel. Most UA teams run creative across three or four channels at once, each with its own CPI and audience mix. Run the sizing calculation per channel rather than once across the whole test budget, because a concept that clears a readable sample on a cheap channel can still fall short on an expensive one at the same allocation, and a team watching only the blended number will miss that dilution entirely. A portfolio that looks properly sized in aggregate can sit quietly under-tested on the one channel a studio most wants to scale.

The discipline this replaces

What portfolio sizing removes from a UA team's workflow is the temptation to treat every concept the creative team produces as equally deserving of budget. Some ideas earn more spend for a faster, more confident read. Others are worth a smaller sample as a cheap filter before anyone commits real money. Set the portfolio size explicitly, before the sprint starts rather than after the concepts already exist, and creative testing stops being a volume exercise and becomes an actual experiment, with a result a media buyer can act on rather than a number that happens to be the highest on a spreadsheet this week.

Related archive reading

These articles provide related context and remain subject to their stated review status.

• Concepts vs Variants: Sizing a Live Creative Slate
• Creative fatigue is a portfolio problem