A/B Test Sample Size Calculator

Plan your experiment before you run it. This free A/B test sample size calculator tells you how many visitors you need per variant to reliably detect the improvement you care about — so you do not stop too early and chase noise, or run forever.

How to use the sample size calculator

Enter your current (baseline) conversion rate and the minimum relative improvement you want to be able to detect. Pick your confidence level and statistical power, and optionally your daily traffic to estimate the test duration. The calculator returns the visitors needed per variant. This pairs perfectly with our A/B test significance calculator, which you use once the data is in.

Why tests get called too early

The most common way to waste a test is to watch the dashboard and stop as soon as the lines separate. Early on, conversion rates swing widely because each individual conversion is a large share of a small total. A variant can sit well ahead on day two and finish level on day fourteen. Checking repeatedly and stopping at the first significant-looking moment inflates the false positive rate far beyond the 5 percent the test was designed around, a problem usually called peeking.

The remedy is to fix the sample size before the test starts and then run to it. Settle on the baseline conversion rate, the smallest lift that would actually change a decision, and the confidence and power you want. Calculate how many visitors each variant needs, write that number down, and do not evaluate significance until you reach it.

Preview of the A/B Test Sample Size Calculator with Variant A and Variant B fields and a Confidence result

What the calculator cannot account for

A sample size is a floor, not a guarantee. These factors sit outside the arithmetic:

  • Weekly and seasonal cycles. Tuesday traffic does not behave like Saturday traffic, so run for whole weeks even when the required sample arrives mid-week.
  • Novelty effects. Returning visitors sometimes respond to a change simply because it is new, and that lift fades once it stops being unfamiliar.
  • Multiple variants. Testing four arms at once multiplies the chance of a false positive, so the sample needed per arm goes up rather than down.
  • Segment differences. A flat overall result can conceal a clear win on mobile and a loss on desktop. Decide which segments you will examine before the test runs, not afterwards.
  • Downstream effects. A variant can raise sign-ups and lower retention. If the metric you measured is not the one you care about, the sample size is beside the point.
  • Traffic that is not independent. The same person arriving across several devices, or unfiltered bot traffic, breaks the assumptions the calculation rests on.
Illustration: a scan beam auditing a web page, with a checklist of passes, warnings and a ready-made fix

Frequently asked questions

Why does sample size matter in A/B testing?

Too small a sample and your test cannot tell a real effect from random noise, leading to false winners. Calculating the required sample size up front keeps your results trustworthy.

What is minimum detectable effect?

It is the smallest improvement you want the test to be able to detect, expressed as a relative percentage of your baseline. Smaller effects require much larger samples to detect reliably.

What power and confidence should I use?

The common defaults are 95% confidence and 80% power. Raising either makes your test more rigorous but requires a larger sample size.

Can I stop a test early if the result looks significant?

Not with a fixed-horizon calculation like this one. The sample size assumes a single evaluation, made at the end. Every additional look is another opportunity to catch a random swing, and repeatedly testing against a 95 percent threshold can push the true false positive rate above 20 percent. If stopping early matters to you, use a method built for it, such as sequential testing or a Bayesian approach with a stopping rule declared in advance, rather than applying a fixed-horizon threshold over and over.

See your site’s full SEO score — free.Instant audit, 10 checks, no signup to see it.
Run free audit →
Free tools →My toolkit →Locations →Comparisons →
Verified Agent-Ready by DigiJaws
See who AI recommends instead of you — free AEO audit, 90-day plan you keepGet my free audit