Landing pages are an important part of websites, often used to promote a feature, a product or a specific sale. These pages have one purpose and one purpose only: to convert. So how can you use A/B Testing to improve conversions through data-driven design? This blog post will aim to show you examples of how to A/B test landing pages with real world examples that we’ve experienced.
Understanding A/B Testing
At its core, A/B testing is a methodical way of comparing two versions of a web page to determine which one performs better. The process involves splitting the incoming traffic between the two versions and analysing user behavior and interactions to draw insights. In the context of landing pages, this technique helps fine-tune elements to enhance user engagement, conversion rates, and overall effectiveness.
The test candidate
A website we’ll be running tests on is Inboxes App, a simple disposable email website. It makes use of a Chrome Extension to allow you to view your emails within your browser and requires that users install the extension via the Chrome Store.
Because of this, it’s important that the landing page sends users to the Chrome Store so that they can download the extension, and because of this we want to A/B test the download button to see if we can convert even more users.
Hypothesis
Before we start testing random parts of the website, we first need to form a hypothesis. The Inboxes website has a simple and clean design which clearly explains what the extension does, however the download button isn’t converting very well.
As you can see, the "Get the extension!" Call To Action (CTA) isn’t very appealing and doesn’t really tell me what I’m signing up for.
We can hypothesise that if we improve the content within this button we may yield a higher conversion rate.
The test variant
If the default "Get the extension!" button is our control group or “A” variant, then we need a catchy “B” variant which might (or might not!) out-perform the base case. For that, we’ve used the following text: "Add it to Chrome (It's free!)".
And now we wait…
As this was initially a fairly low traffic website which subsequently picked up traffic over time, it was important to capture those early website visitors. However, low traffic means getting to statistical significance would take longer.
This test was left to run over the period of a month and a half until the results became fairly conclusive. Overall there were ~1500 views to this test, which while small was able to give us 94% confidence in our test results.
Variant B is the winner!
After a month and a half the results weren’t changing much, and we had 94% confidence in our tests. While we didn’t reach complete statistical significance, the results have overall been positive and resulted in a 24% increase in downloads.
Maybe it was the more direct "Add it to Chrome", or perhaps it was letting people know that "(It's free!)". These are both things we could test for, however for now a 24% uplift in downloads from running an A/B test on a landing page isn’t a bad result!
Other possible avenues to test
While the above example shows a simple button text change, when you A/B test landing pages you can test many other parts of the website.
We could for example:
- Test the h2 heading to give a different description of the app.
- Change the graphic on the right with a video demo of the browser extension.
- Animate the download button to grab the user’s attention.
- Change the button colour.
- Make the “Get the extension” nav link a button which would make it stand out more.
The options are endless, and while A/B Testing is only a small part of your marketing mix, it would be a shame to lose out on users you’ve fought so hard to get to your website.