ASO experiments
ASO experiment and sample-size planner
Plan an app listing A/B test with baseline conversion, detectable uplift and traffic. Export the sample size, timing and experiment assumptions.
Your working inputs stay in this browser. Anonymous site analytics may record tool actions, never your values.
Plan before testing your listing
Estimate the sample for two independently randomized variants with 50:50 allocation and a binary conversion outcome. The estimate helps assess feasibility; it does not declare a winner or reproduce a store console’s statistical method.
Target conversion
12.00%
Visitors per variant
3,841
Total visitors
7,682
Minimum traffic days
8
Days count traffic collection only. Allow full business cycles and conversion maturation. Low traffic may make this experiment impractical; reduce the number of variants or test a larger meaningful change.
How to read the result
Estimate a fixed-horizon sample for two equally allocated variants, then export a plan with the hypothesis and conversion metric.
Worked example
At 10% baseline conversion, a 20% relative improvement means a 12% target rate: a two-percentage-point increase. A two-sided 5% significance level and 80% power require approximately 3,841 visitors per variant with this normal approximation. Traffic collection time divides the total sample by participating daily traffic.
Definitions and method
For p0 and p1 and p̄ = (p0 + p1) / 2, sample per variant is rounded up from [z(1−α/2)√(2p̄(1−p̄)) + z(power)√(p0(1−p0)+p1(1−p1))]² / (p1−p0)². This is an equal-allocation normal approximation without continuity correction, assuming independent binary outcomes. It is a planning model, not the store console’s inference engine.
Common questions
- Why does a smaller detectable improvement need more visitors?
- Smaller differences are harder to distinguish from sampling variation. Sample size grows quickly as the effect shrinks. Choose the smallest change that would matter to your decision.
- Can I stop when the result first looks significant?
- Not under a fixed-horizon design. Set the sample and observation window before starting and allow delayed conversions to mature. Repeated early stopping needs a different statistical method.
- Can I test three screenshots at once?
- This planner supports two randomized variants. More variants, unequal allocation and multiple comparisons need an appropriately adjusted design.
Sources and review date
Method and reference links reviewed 15 September 2026. Recheck provider requirements before submission.