When improving website conversion rates through testing and optimization, understanding the why is critical. The why is at the top level of insight that we as designers and marketers need to continuously strive to get to. True insight from any testing and optimization comes from an understanding of what specific variable in your test made the difference. What was it that caused the lift in conversion rate, or the dip? Was it the headline, the button, the product image? What was it that you can point to and bring to your boss and say. This did that and the impact was this in real money to the bottom line.
Knowing the why in any conversion rate optimization test for both positive gains and decreases means we can then begin to get closer to predictability. We can make changes in the future to generate consistent results. This is the holy grail of conversion rate optimization, but it is not always what we should be striving for. Often times when it comes to testing and improving conversion rates, businesses do not have the luxury of time.
Sometimes a business needs results and they need them fast. In such a situation it may be less important to know exactly which particular test variable contributed to the impact of improvement and it may be more important to just simply increase results. This is not the dogma of the scientific method but from a business perspective it makes sense.
Sometimes the business need outweighs the needs of science. We’re not living in a lab with white coats and precise measurement tools. We’re building e-commerce sites and living in the trenches of online conversion rates. To get closer to understanding the why we need to conduct systematic tests, changing only one element at a time or conduct very strict multivariate tests, which require a considerable amount of traffic in order to be statistically valid and a disciplined thought process.
For such a situation, where traffic in terms of visits is not all that high to conduct anything other than an A/B test. The business pain is high and results are required fast. Or upper management buy-in to the idea of conversion rate optimization or testing is low and you need a quick win. It’s OK to enter the world of testing and improvement without solving for the why. In such a situation you’re strictly solving for improvement. Your conversion rate optimization philosophy needs to shift in order to focus on the biggest gains possible. The best chance of reaching those big double, and triple digit improvements is to approach your testing in a radically different way. Changing one element at a time will beyond a shadow of a doubt tell you if that particular element improved your goal or not but you’ll most likely experience minor gains at best, in the range of a few percentage points. Taking a different approach however can lift your conversion rates in the triple digit range and beyond. To get the 100+% improvements your test page needs to be drastically different from your control page. Think of your test variable in this case as your entire page rather than simply 2 versions of the headline. There is a time and place in your testing plan to get more granular and bring it back to understanding the why but in the situation described previously your best chance for wind improvements is going to be found in wildly different test pages.
If you’d like to further discuss conversion optimization testing philosophy please contact me, (bobby @ creativethirst dot com) I’d be more than happy to chat.
Great Post!