How to Design a Split Test
Scott Miller,
Vertster.com, Salt Lake City, UT
Jan 18, 2007, 13:15
The ultimate success or failure of your split testing will largely be determined by three W's:
- Which pages should you test?
- What page elements should you test? and
- Why test these things?
Let's look at each of these to see how they can help you sell more on your website.
Which Pages Should You Test?
The specific page to be tested is the very first decision that a testing marketer will make. Unfortunately, many folks decide to test their home pages first, which immediately leads to frustration and lack of conclusive results. The home page, by its very nature, is the hardest page on your site to test. Because of the many goals the page attempts to serve, isolating a single goal and "success action" is very difficult.
It is much better to focus on specific landing pages, which usually have only one goal and purpose.
What Page Elements Should You Test?
The next decision you'll be faced with involves deciding which portions of a web page to include in your experiment. Effective test design means looking at your web pages and visually breaking them down into individual ingredients, called components. A typical landing page is made up of several common ingredients. Here are some examples:
- Headline
- Sub-headline
- Greeting
- Main image (sometimes called a hero shot)
- Body copy
- Testimonials
- Signature
Focus on components that appear near the top of the web page. Some of the first things to look at when designing a split test are the headline, hero shot, any promotional messaging (such as "free shipping" or "30% off"), and any scarcity messaging (such as "limited time only"). When designing variations, select substantially different options, as these tend to yield clearer data. Be as specific as possible, because "Save 20% if you order by Feb 19" is often better than "Limited time offer, save 20%."
Why Test These Things?
As you select pages and components to test, keep one word in the back of your mind: Why?
I once had a customer who was set on testing whether a footer should appear or not. His answer was sheepish: "Because I wonder if it will help my sales or not." What do you think? With an average bounce rate greater than 50%, that is, people who immediately hit their back button, it's safe to say most of his users weren't even staying long enough to scroll down to the footer.
Focus on the top portion of the page, but also remember why you are testing in the first place: to increase your sales. Use tests that are likely to improve your business, not to answer some inner curiosity about the relative importance of various page elements.
Scott Miller is the CEO of Vertster.com and the author of the book, The ConversionLab.com: How to Experiment Your Way to Increased Web Sales Using Split Test and Taguchi Optimization. This past year he has overseen more than 2,500 web experiments.
