How this calculator works

Statistical parameters can be quite confusing for those who are new to AB testing, and MDE in particular is often misunderstood by even experienced CROs - check out this article on the Convert.com blog for more on this topic.

This calculator is designed to take the guesswork out of choosing these parameters. It will start with some reasonable defaults for significance, power, and MDE, and then make adjustments depending on your traffic and conversion volume.

It will try to keep the test duration within a window of 1 to 8 weeks. If you have very high volume and it has maxed out the error control, it will allow test durations of less than 1 week, or if you have very low volume and error control has been reduced as much as possible (without being reckless) it will allow up to 12 weeks.

The initial defaults are 95% significance, 80% power, and 5% target MDE. The hard limits are:

• significance between 90% and 99%,
• power level between 80% and 95%, and
• a target MDE between 1% and 10%.

If you want to override the recomendations and/or test duration and MDE limits you can click the toggle or link to customize test parameters.

This is for AB tests with just one variant tested against the control, using a one-tailed superiority test, meaning that you want to prove that the variant is better than the control in order to justify shipping it. These defaults, ranges, and assumptions should be appropriate for the majority of online AB testing scenarios.