Most websites are built on opinion. A/B testing replaces opinion with evidence. You run two versions of a page, a subject line or a button, send real visitors to both, and let the numbers tell you which one earns more clicks, signups or sales. No more arguing about whether green converts better than orange. You actually find out.
This matters because small changes compound. A 6% lift on a homepage CTA, a 12% increase in checkout completion, and a 30% jump in email open rates. None of those is flashy, but stacked across a year of traffic, they shift real revenue.
What Is A/B Testing?
A/B testing is a method for comparing two versions of something, a webpage, an email, a button or a headline, to see which one performs better with real users. It works by splitting your audience randomly. Half see version A. Half see version B. After enough visits, you can tell which version produced more of whatever outcome you cared about.
The principle is simple. The discipline is harder. You only change one element at a time, otherwise you can’t tell what caused the result. Then, run the test long enough to get a statistically meaningful sample. And you decide the goal upfront, before you start. Clicks, signups, sales, time on page. Pick one. Measure it.
A/B testing is sometimes called split testing. Multivariate testing is a related approach that compares more elements at once, but it requires far more traffic to produce reliable results, so most businesses start with A/B.
Why Does A/B Testing Work?
A/B testing works because human behaviour on websites is rarely what we expect it to be. Your team’s intuition about what converts is shaped by what you like, what you’ve seen elsewhere, and the assumptions you’ve never questioned. Real visitors don’t share your assumptions.
Three things make A/B testing different from a redesign or a gut-feel update. First, it’s measurable. You’re not guessing whether the change helped. You have numbers. Second, the risk is low. If version B loses, you go back to A. Nothing is permanent. Lastly, you learn either way. A losing test still tells you something about what your customers don’t respond to. No test is wasted.
What Can You A/B Test?
Almost anything you put in front of a user. Some elements move the needle more than others. Here are the ones worth starting with.
Headlines and value propositions. The first thing a visitor reads decides whether they stay. Test wording, length and specificity.
Calls to action. Button text, colour, size, position. “Get a quote” versus “See pricing” can change conversion by double digits.
Forms. Number of fields, layout, single-step versus multi-step. Shorter forms usually convert better, but not always. Sometimes, more fields signal seriousness and filter out time-wasters.
Email subject lines. The easiest place to start. Subject lines have an immediate, measurable effect on open rates.
Page layout. Where the content sits, what’s above the fold, how much scrolling is needed. Layout changes can produce some of the biggest lifts.
Images and video. Hero images, product photos, autoplay versus click-to-play.
Pricing presentation. How tiers are ordered, whether you anchor with a premium option, monthly versus annual framing.
You don’t need to test everything. Start with the page that gets the most traffic and matters most to your business. Usually, that’s a homepage, a key landing page or a checkout flow.
A Real Example: Ubisoft’s “For Honor” Page
In 2019, video game publisher Ubisoft tested a redesign of the purchase page for its game For Honor. The original page used a single-column layout that required visitors to scroll through multiple steps to complete a purchase. Analytics suggested users were dropping off along the way.
Ubisoft built a variant with a two-column layout that kept all the purchase steps visible at once, reducing the need to scroll. They ran the test for three months against the original. According to VWO, which ran the experiment, conversions increased from 38% to 50%. That’s a 12 percentage point absolute gain. Overall lead generation across the page increased by 12%.
The change wasn’t dramatic. It wasn’t a rebrand or a new product. It was a layout adjustment, validated with evidence rather than opinion. Across Ubisoft’s traffic volume, the revenue implications were significant.
How to Run Your First A/B Test
Five steps. Don’t skip any.
1. Pick One Goal
Decide what success looks like before you do anything else. More form submissions. Higher email open rate. More clicks on a specific button. One metric. If you try to measure three things at once, you’ll end up unable to call a winner.
2. Form a Hypothesis
Don’t test for the sake of testing. Start with a theory grounded in evidence. “Visitors aren’t clicking the CTA because the wording is too generic. Replacing ‘Submit’ with ‘Get my free audit’ should increase clicks.” That’s a hypothesis. It can be wrong, but it’s specific enough to be useful.
3. Change One Thing
Only one variable per test. Change the button text and the colour and the position together, and you can’t tell which one caused the result. Keep the rest of the page identical between the control and the variant.
4. Run Long Enough
This is where most small business tests fall apart. A test needs enough visitors to produce a statistically reliable result. The exact number depends on your current conversion rate and the size of the lift you’re trying to detect, but as a rough rule of thumb, you want at least a thousand visitors per variant on the page being tested, and a minimum run of two weeks to smooth out day-of-week effects. Calling a winner after 48 hours and 200 visits is how you fool yourself.
5. Implement the Winner, Then Test Again
If version B wins, it becomes the new control. Then you find something else to test. A/B testing is a loop, not a project. The businesses that get compounding returns are the ones that run it continuously.
Why Does A/B Testing Fail for So Many Businesses?
Most A/B testing programs fail for the same handful of reasons.
Tests are stopped too early. The numbers look encouraging after three days, so the team calls a winner. A few weeks later, the “winning” change isn’t actually performing better.
Too many variables change at once. The team redesigns half the page and labels it an A/B test. It isn’t. It’s a relaunch with a control group.
The wrong thing is being measured. A change might increase clicks but reduce conversions further down the funnel. Measure only the click and you’ll declare a winner that’s actually hurting revenue.
There’s no hypothesis. Testing random ideas without a reason is a slow way to discover that most changes don’t matter.
Tests are run on pages with too little traffic. If a page gets 50 visitors a week, you’ll never get a statistically meaningful result. Test high-traffic pages first.
A/B testing isn’t a tool problem. It’s a discipline problem. The tools are easy. Doing it properly is what separates programs that compound from programs that waste everyone’s time.
What Tools Should You Use for A/B Testing?
You probably already have access to A/B testing tools without realising it. Mailchimp, Campaign Monitor and most modern email platforms have split testing built in for subject lines and email content. WordPress has plugins that handle on-page tests.
For websites, options include VWO, Optimizely, Convert and AB Tasty, which range from free tiers up to enterprise pricing.
Start with what’s already in your stack. If you’re running email campaigns, test subject lines first. If you’re running paid advertising, test ad copy variants. Get a few wins under your belt before investing in dedicated testing platforms.
A/B testing also works hand in hand with good website statistics practice. Tests tell you what you won. Analytics tell you where to test next.
Where to Start
A/B testing isn’t complicated, but it does require honesty. You have to be willing to find out your favourite headline doesn’t work. You have to run tests long enough to trust the result. And you have to keep doing it after the first win, because one test isn’t a strategy.
If you’re not testing yet, pick one thing this week. It could be your email subject line, your homepage button or your contact form layout. Set up a simple A/B test in whatever tool you already have. Run it for two weeks. See what happens.
Mettro has been helping Queensland businesses make data-led decisions about their websites and marketing for 25+ years. If you want to talk about where testing could move the needle for your business, or you’ve been running tests and the results aren’t stacking up, get in touch.
Talk to us about your marketing.
