Get a free audit
Free tool · Tracking & testing

A/B test calculator

Two ads, two landing pages or two offers. Enter visitors and conversions for each and see whether the winner is a real difference or just random noise.

Your numbers
People who saw version A.
People who converted on version A.
People who saw version B.
People who converted on version B.
95% is the usual standard.
Result—
Rate A—
Rate B—
Difference (B vs A)—
p-value—
Get a free audit

Runs in your browser. Nothing you type is sent anywhere.

How the significance test works

The calculator uses a two-proportion z-test, a standard way to compare two rates such as conversion rate or click-through rate:

  1. It calculates the rate for each version: conversions ÷ visitors.
  2. It estimates how much the rates could differ by chance, based on sample size.
  3. It returns a p-value: the probability of seeing a difference at least this big if both versions were really the same.

If the p-value is below your threshold (0.05 for 95% confidence), the difference is called statistically significant.

Example

Version A: 40 conversions from 2,000 visitors (2.0%). Version B: 62 from 2,000 (3.1%). B is 55% better and the p-value is about 0.028, so B wins at 95% confidence.

Good testing habits

  • Decide the sample size or test length before you start, and avoid stopping the moment one version looks ahead.
  • Test one clear change at a time.
  • Run tests over full weeks to include weekday and weekend behaviour.
  • On Meta, budget may not be split evenly between ads. Compare rates, not totals.

See how we structure tests: Meta Ads creative testing on a small budget.

FAQ

Can I use this for CTR?

Yes. Enter impressions as visitors and clicks as conversions.

What confidence level should I use?

95% is the common standard. 90% is acceptable for low-risk decisions; use 99% for big, expensive changes.

Why is my clear winner not significant?

With small numbers, large differences can happen by chance. Collect more visitors and conversions before deciding.

Ready to grow? Let's talk.