A/B & Multivariate testing

Things you must know before you start A/B & Multivariate testing

Every website visitor matters. Every click is a chance of potential lost or gained revenue. Testing gives your visitor a voice in your design process. It allows these visitors to tell you what worked on your website and what did not.

Testing is an important component of the Conversion Framework. And over the last three years, our clients consistently averaged a 20% improvement in conversion rates and seen the following benefits as a result of testing:

  • Increased website sales and lower customer acquisition costs through improving overall conversion rates
  • Removed the guesswork from design and optimization process. Gaining the confidence of knowing what works and what doesn't’t work on the website
  • Provided site visitors with better experience and reduced bounce and exit rates on the site

While the benefits of testing are obvious, less than half of the top 500 e-retailers deployed any testing on their website as of 2007.

Increasing conversion rates through A/B & Multivariate testing

Do we want you to test? Of course we do. But there are few things you must keep in mind before you deploy any type of testing on your website.

1. Testing is one of many steps in the conversion optimization process

Testing is an important component of any optimization project. However, it should not and the only component. Testing should take place after other equally important stages of optimization work such as persona development, site analysis, design and copy creation are completed first. Each of these elements provides a building block towards a highly optimized website that converts visitors into clients.

2. Poorly designed experiments are a waste of time and money

Tools are useless without the people who properly run them. Testing software such as Google Website Optimizer allows you to create and start testing is less than 10 minutes. But that is the easy part. Designing successful test scenarios, assessing results, and creating meaningful follow up tests are ultimately where many companies fail.

Poorly designed experiments can take years to complete. Even worse, they might not provide concrete insights to what elements will convert more visitors into clients.
Imagine a case where you plan test different headlines on a page. You start by coming up with 10 different possible variations to the headlines. What criteria are you going to use to determine which of the headlines you should test? Why not test all of the 10 different headlines? You will most likely find yourself relying on guesswork to determine which of the versions to include in the test.

The same logic of course applies to all elements you want to test on a page. Without being judicious with test scenarios, we have seen clients attempting to test millions of combinations.

3. Length of time to complete an experiment

When conducting online tests, your goal is to arrive at statically valid results in the shortest possible time. If you are not careful designing your test experiments, you will end up having to run test scenarios for years to come. 

4. Traffic matters

Sometimes companies are too eager to deploy testing without analyzing their current traffic trends. The number of visitors your page receives plays an essential role in determining how long an experiment runs for. But the quality of these visitors matters too. Pages that receive less than 30 or 40 visitors per day are not ideal with multivariate testing. Analyze the quality of traffic your page is receiving. What keywords are visitors using to get to your page? Are they looking for relevant terms to the topic of the page? When it comes to testing, quality and quantity of visitors are equally important.

8 elements to successfully design a test scenario

Designing a good test scenario requires time and preparation. Successful tests are conducted in short periods of time yet yield greatest possible improvements in conversion rates. To achieve successful testing, you need to 

  1. Define clear and measurable conversion criteria for the test. Not all tests are about increasing sales or leads captured on a page. You might test difference combinations to lower the bounce or exit rates on a page. 
  2. Determine whether A/B or multivariate testing is more appropriate for your situation.
  3. Assess current traffic trends, current conversion rates, target conversion goals and determine a reasonable number of scenarios to test in a relatively short time span.
  4. Determine which page elements will have the most impact on your particular website visitors through the persona created for your website
  5. For each of the target elements create possible variations that will appeal to different website personas. Consider caring vs. logical appeal, impulsive vs. aggressive types of visitors.
  6. Install and deploy test scenarios by using an appropriate testing tool
  7. Track and evaluate results and improvements
  8. Utilize test results and findings to conduct testing follow-ups and further improve conversion rates

What elements should you test on a page?

There are many elements you can test on any page. We find that starting with the following elements provides the highest impact on conversion rates:

  • The headline of the page
  • The hero image of the page
  • The list of benefits on the page
  • Calls to action on the page
  • Incentives used on the page
  • Page layout

There are many variations to test for each of the above elements. You should resist the temptation of randomly selecting variations for these elements and hoping that visitors will react positively. Deploying test variations based on customer preferences through the guidance of personas is a lot more effective.

A/B & Multivariate testing tools

There are several tools available in the market place to conduct optimization testing. Our favorite tool by far is Google Website Optimizer. Not only is the tool available for free, but the insights it provides are valuable. With most clients, Google website optimizer serves their needs effectively.

Designing test scenarios and deploying is an integral part of our conversion optimization services  including full website optimization, conversion recommendations report, or landing page optimization. Contact us today to get started with optimization work.