Guide

THE TOOLBOX / SCHOOL & STATISTICS UTILITIES

Compare outcomes,
plan the sample.

Calculate a two-proportion A/B test p-value and approximate sample size locally.

Enter visitors and conversions to run a two-proportion Z-test or plan your sample size.

Scenarios
1. Experiment Data & Power PlannerTwo-Sided Z-Test

Analyze Observed Results

Plan Required Sample Size

2. Statistical Verdict & Breakdownα = 0.05
Not Yet Significant at 95% Confidence
p = 0.1529
Observed lift could still be due to random variance (p ≥ 0.05).
Control A
10.00%
100 / 1000 conversions
Variant B
12.00%
120 / 1000 conversions
Relative Lift
+20.0%
Absolute Diff
+2.00 pp
Z-Score
1.429
95% Confidence Interval for Difference (B − A)
[-0.74 pp, 4.74 pp]
Click “Analyze results” or “Calculate sample size” to generate a text summary.

Assumes two independent groups with fixed sample size and sufficient expected counts (≥ 5 per cell). Repeated peeking without sequential correction inflates false positives.

THE DETAILS

Two-proportion comparisons

The test uses a large-sample approximation described in the NIST two-proportion test reference.

What test is used?

A two-sided pooled two-proportion z-test using a normal approximation.

What assumptions apply?

Groups should be independent with a fixed sample and sufficient expected counts. Repeated peeking and multiple comparisons affect results.

How is sample size estimated?

A normal approximation estimates equal group sizes from baseline and target rates, confidence and power.

Is the result definitive?

No. P-values and sample sizes depend on assumptions and do not measure practical significance.

Are inputs uploaded?

No. Calculations run locally.