A/B Testing & CRO Glossary

Plain-English definitions of the terms behind A/B testing and conversion optimization.

A/B testing

A method of comparing two versions of a page or element by showing each to a random half of your traffic at the same time and measuring which produces more conversions. Also called split testing.

Split testing

Another name for A/B testing — splitting traffic between two or more variants to see which performs best. "Split URL testing" specifically means testing separate URLs rather than on-page changes.

Statistical significance

The likelihood that the difference between your variants is real rather than random chance. A result at 95% confidence means there is only a 5% probability the difference is a fluke. Most teams call a winner at 95%+.

Conversion rate

The percentage of visitors who complete your goal (a form submit, sign-up, or purchase). Calculated as conversions divided by unique visitors, times 100.

Conversion rate optimization (CRO)

The practice of systematically increasing the percentage of visitors who convert, usually through A/B testing, landing page improvements, and removing friction from the funnel.

Control and variant

The control (or "A") is your existing version; the variant (or "B", the challenger) is the new version you are testing against it. The control is the baseline you measure improvement from.

Sample size

The number of visitors (and conversions) a test needs to detect a given difference with confidence. Lower baseline conversion rates and smaller expected lifts require larger samples. Estimate it before you start.

Statistical power

The probability that a test will detect a real difference when one exists. Underpowered tests (too little traffic) often miss real winners or report false results.

Multivariate testing

Testing multiple elements and their combinations at once (e.g., two headlines × two images) to find the best combination. It needs far more traffic than a simple A/B test.

Landing page

A standalone page built for a single goal — usually the destination for an ad or campaign — designed to convert visitors into leads or customers.