A/B test calculator
Two ads, two landing pages or two offers. Enter visitors and conversions for each and see whether the winner is a real difference or just random noise.
Runs in your browser. Nothing you type is sent anywhere.
How the significance test works
The calculator uses a two-proportion z-test, a standard way to compare two rates such as conversion rate or click-through rate:
- It calculates the rate for each version: conversions ÷ visitors.
- It estimates how much the rates could differ by chance, based on sample size.
- It returns a p-value: the probability of seeing a difference at least this big if both versions were really the same.
If the p-value is below your threshold (0.05 for 95% confidence), the difference is called statistically significant.
Example
Version A: 40 conversions from 2,000 visitors (2.0%). Version B: 62 from 2,000 (3.1%). B is 55% better and the p-value is about 0.028, so B wins at 95% confidence.
Good testing habits
- Decide the sample size or test length before you start, and avoid stopping the moment one version looks ahead.
- Test one clear change at a time.
- Run tests over full weeks to include weekday and weekend behaviour.
- On Meta, budget may not be split evenly between ads. Compare rates, not totals.
See how we structure tests: Meta Ads creative testing on a small budget.
FAQ
Can I use this for CTR?
Yes. Enter impressions as visitors and clicks as conversions.
What confidence level should I use?
95% is the common standard. 90% is acceptable for low-risk decisions; use 99% for big, expensive changes.
Why is my clear winner not significant?
With small numbers, large differences can happen by chance. Collect more visitors and conversions before deciding.
Keep calculating.
Ad metrics calculator
CPM, CTR, CPC, conversion rate, CPA and ROAS from one set of numbers.
Open the toolUTM link builder
Build clean tracking links for Meta, Google, TikTok, WhatsApp and email in seconds.
Open the toolROAS & profit calculator
Turn ad spend and revenue into ROAS, profit after ads and a clear verdict.
Open the tool