fbpx

A/B And Multivariate Testing In Plain English

When in comes to conversion rate optimization there is no failure only feedback. Testing is the key to getting to what works best for your prospects and visitors. All too often someone within your organization thinks they know what is best. Perhaps it is your design team, chief marketing officer or even your CEO. The brutally honest truth is that none of them know. They may be following best practices or concepts that have worked in case studies they’ve read but until you test you will never know for sure what truly works for your customers.

With free testing tools like Google Website Optimizer there is no reason not to test. Every marketer should always be in the middle of at least one test on their website at any given time. There just aren’t any excuses for not testing. Testing is how we improve.

Two of the most common online testing methods are the A/B test and the multivariate test. Here’s a plain English explanation of each that you can take to your boss to get the testing ball in your organization moving in the right direction.

A/B Testing

In an A/B test there are two versions of a web page, version A and version B. Version A is usually the control page, or the existing page and version B is the alternative page. The winning version becomes the control page in a follow-up test against another alternative.

Half of the traffic sees version A and the other half sees version B until each version has enough traffic and enough conversions (that’s the goal of the particular page you are testing and the goal should be the same for both versions) has been gathered for the test to be statistically valid for a wining version to be declared.

An A/B test is the most common and easiest type of test. It’s a great place to start with testing and it gives you the largest range of possibility. If you have a specific landing page that you’re driving traffic to from a channel like pay per click adwords start with an A/B test.

The advantages of an A/B test is that it allows you to complete freedom to come up with a totally different page version to test against the control. Which means you can try wild solutions in order to get the biggest gains. You’ll also get the fastest results with an A/B test since there are only two versions. Version A and version B, even though version B can be completely different than version A.The more versions of a page the longer a test needs to run and the more traffic it needs to be declared a statistically valid winner.

Multivariate Testing

In a multivariate test there are several versions of one or more elements on a single page. The combination of theses elements are what’s being tested. An element can be anything, an image, a headline, the words on the page, a call to action button, etc. For example let’s say you have 3 different headlines you thought might increase the conversion rate of the page along with 2 different images and 4 different call to action buttons.

So you have 3 different elements in this test, the headline, the image and the call to action button. Your testing tool such as Google Website Optimizer would pull in each element to make up every combination. In this example there would be 24 different combinations or versions of this test page. 3 x 2 x 4 = 24 (3 headlines x 2 images x 4 buttons) With each different version a visitor would see only one combination of elements. This is what’s known as a full factorial experiment.

The advantages of a multivariate test is that it’s much more granular with different combinations of elements you can really get specific, not only with each element but also the combination of elements. The disadvantage is that it requires more traffic to achieve statistically valid results since there are more than 2 versions and it can easily get out of hand with the amount of versions as in the example we had 24 different versions of the page.

In conclusion, each type of test is dependent on the needs of the business and the goals of the page. The important thing to remember is that testing is not a one-time thing. It’s an ongoing effort for continuous improvement. Remember this is just a basic overview of these two types of tests, we’re just scratching the surface of this very broad and deep topic.

Discover the 3 funnels that can help your health supplement business succeed.

Health Supplement Business Mastery Podcast

Listen to the Health Supplement Business Mastery Podcast for for dietary supplement entrepreneurs and marketers.

Listen to the health supplement business mastery podcast on Spotify
Listen to the health supplement business mastery podcast on Google Podcasts

By Bobby Hewitt

Bobby Hewitt is the founder of Creative Thirst. A conversion rate optimization agency for health and wellness companies with a specialized focus in dietary supplements. We’ve helped health clients profitably scale using our four framework growth model validated through A/B testing. Bobby has over 17 years of experience in web design and Internet marketing and holds a bachelors degree in Marketing from Rutgers University. He is also certified in Online Testing and Landing Page Optimization and won the Jim Novo Award of Academic Excellence for Web Analytics. As well as a public speaker and contributing author to “Google Analytics Breakthrough: From Zero to Business Impact, published by Wiley.