Conversion / Testing video

A/B Split Testing vs. Multivariate Testing with Lance Loveday (5:16) Dr. Ralph F. Wilson , Web Marketing Today - Aug 10, 2010

In this video interview, conversion expert Lance Loveday explains the pros and cons of simple A/B split-testing vs. the much more complex multivariate testing.

We've found, says, Loveday, that A/B testing tends to deliver the largest gain. Of course, he adds, that this could be ABCD testing, that is 4 variations of a single factor, as opposed to testing a large number of factors at one time.

The advantage of A/B testing is that it tends to cause you to rethink the entire user experience as you look for elements to test -- layout, visual hierarchy, messaging, etc. Multivariate testing, on the other hand, often leads to testing smaller incremental variations that don't make as much difference. A/B testing forces you to be more creative about what you're testing.

Loveday explains that his firm's process is to start with some A/B testing to find an effective layout for a client. Once that's established, they'll do multivariate testing to get some additional incremental gains.

A/B testing suits small businesses well because clear results often don't require as many visitors as are needed with multivariate testing. And, A/B testing is a lot easier to execute than multivariate.

With a small business with some in-house tech savvy, a company could do A/B testing itself with a tool like Google Website Optimizer -- or hire a firm to do testing for it.

Loveday encourages viewers to consider Internet sites that allow "crowdsourcing" of designs relatively inexpensively. For a couple of designs that might cost under $1,000, you could test them against your baseline and potentially make some big gains.

 

Lance Loveday is the president of Closed Loop Marketing, which assists clients through A/B testing, multivariate testing, end-to-end optimization, and coaching and consulting on testing, as well as search engine marketing. This interview was recorded at Conversion Conference West Coast in San Jose, California, May 2010.