Free tool
A/B test calculator: is your result real?
Enter visitors and conversions for both versions. You get the conversion rates, the uplift, a p-value and a plain answer. A second box shows how many visitors you need.
Check your test result.
How many visitors do you need?
The numbers use a pooled two-proportion z-test. The visitor estimate assumes 80% power and uses the confidence level and test type chosen above. It is a planning figure, and very small counts make the maths rough. Everything is worked out in your browser, and nothing you type is sent anywhere.
How to read the result.
What significance means
The p-value answers one question: if both versions were really the same, how often would luck alone give you a gap this big? At 95% confidence you want that to happen fewer than 5 times in 100. It does not say how big the gain is, and it is not the chance that B is better. A significant result tells you the difference is probably real. Whether it is worth having is up to you.
Why small sites need big changes
With few visitors, your conversion rate jumps around from week to week. To pick a small lift out of that noise you need thousands of visitors for each version. Try 3% with a 10% lift in the second box and you will see why a site with a few hundred visitors a month cannot test button colours. Test bigger changes instead, such as a new offer, a shorter form or a different page layout, where the lift could be 30% or more and far fewer visitors will do. You can also measure a step that more people reach, such as clicks on Add to cart.
Common mistakes
- Stopping as soon as the result looks good. Checking every day and stopping at the first p-value under 0.05 gives false wins much more often than 5 times in 100. Pick the visitor count first, then read the result once.
- Testing many things at once. Compare five versions or ten metrics and one of them will look significant by chance.
- Ending on a Wednesday. Run whole weeks, so weekdays and weekends count in both versions.
- Reading “not significant” as “no difference”. It means unproven. There may be a gap too small for your traffic to show.
- Choosing one-tailed after you see the data. Decide before the test starts.
To run tests that hold up, see our conversion rate optimization services or the wider conversion and testing services. For the practical side, read how to A/B test a landing page and what a good ecommerce conversion rate looks like.
Want tests run for you?
We set up tracking, pick what to test and read the results honestly. We tell you first if your traffic is too low for a fair test.
No calls needed, and no commitment.