A/B testing a landing page means showing two (or more) versions to different visitors at the same time and measuring which one converts better. Done right, it turns “I think this headline is stronger” into a decision backed by data. Here is the exact process.
1. Choose one conversion goal
Every test needs a single primary metric: form submissions, button clicks, sign-ups, purchases, or a thank-you-page view. If you track five things and one moves, you will fool yourself. Pick the one that maps to revenue and make it the goal.
2. Form a hypothesis
A good hypothesis is specific and falsifiable: “Changing the headline from a feature statement to an outcome statement will increase demo requests, because visitors care about the result, not the mechanism.”This forces you to change something meaningful and gives you a reason to believe it will work.
3. Change one meaningful thing
Isolate the variable so you know what caused the change. The elements worth testing first, in order of impact:
- Headline — the single biggest lever on most pages.
- Primary CTA — the copy, and where it sits.
- Hero offer / value proposition.
- Form length — fewer fields usually means more submissions.
- Social proof placement — testimonials, logos, numbers.
Leave colors and button shades for later; they rarely move the needle compared to the above.
4. Split traffic evenly and randomly
Send 50/50 (or weighted) traffic to each variant, and make the assignment sticky so a returning visitor always sees the same version. Both variants must run at the same time — never “this week vs. last week,” because seasonality and traffic sources would contaminate the result.
5. Run to statistical significance
This is where most tests go wrong. A variant that is “winning” on day two is usually just noise. Keep running until:
- You reach 95%+ statistical confidence (a chi-square test on conversions vs. unique visitors), and
- The test has run at least one full business cycle (typically 1–2 weeks), so weekday and weekend behavior are both represented.
Decide the required sample size before you start — if your traffic can never reach it, the test cannot conclude, and you should test a bigger, bolder change instead of a subtle one.
6. Ship the winner (or learn from the loss)
If a variant wins with confidence, roll it out to 100% of traffic. If it loses or ties, that is still a result: you learned that lever does not matter for this audience. Either way you now know more than you did.
7. Repeat — testing is a program, not an event
The compounding gains come from a steady cadence: one test after another, each building on the last. Teams that test continuously pull far ahead of teams that run one test a quarter.
Common mistakes to avoid
- Stopping early. Calling a winner before significance is the #1 error.
- Changing several things at once. You will not know what worked.
- Testing tiny changes on low traffic. You will never reach significance.
- Ignoring segments. A variant can win on mobile and lose on desktop — check both.
- Not running long enough. Always cover a full business cycle.
Doing this in SplitLab
SplitLab handles the mechanics of the above automatically: it splits traffic deterministically by weight on your own custom domain, keeps assignments sticky per visitor, tracks pageviews and conversions, and calculates statistical significance for you so you know exactly when you have a winner. You can even build the challenger variant with AI and publish it as a test in a few minutes.