I have a love/hate relationship with A/B Split Testing, especially when it comes to Web copy. Love that A/B testing can deliver significantly improved response, but hate that many brands may base all future copy decisions on a single test that delivered or over delivered on expectations.
Relying on a singular result, creates missed opportunity to refocus or edit content for other circumstances, site users, time periods or changing business factors. This is why it is crucial to have a sound content strategy to help determine variables, governance and success metrics for copy based on the user personas that were developed for your Web site.
If we can agree that content is your Web site’s greatest asset, the user persona should be the guidepost you’re using to increase its value any time we change messaging, and we can validate this premise through A/B copy testing.
And only through repeated and frequent testing will we be able to make changes that help us:
What factors should be considered in A/B Split Copy Testing?
1. Start with a metric in mind.
What are you trying to accomplish with the test? Are you after more subscribers, conversion rate increase, or a greater return on investment? Just like wanting to know what we want our users to do helps us define content strategy, goals for testing will determine parameters, which in turn will determine the potential success of our efforts.
2. Establish a control copy page/persona
Think back to your elementary school science class friends. Establishing a control persona will help us to establish the copy that we will test all varitions against, always keeping step one in mind as we develop considerations for variables.
If you are just getting started with A/B testing, your control page will be your current copy that is underperforming before any variation is served. When new copy outperforms the existing control copy, consider it your new benchmark (control persona) in any subsequent testing.
3. Determine a reasonable interval for the test
Determine how you’ll gather the data and for how long you need to gather it. This time period will vary from site to site, but should allow for the gathering sufficient data to gauge real insight about your A/B tests. If your site has a lower number of daily unique visitors, the test may run significantly longer to determine a clear copy winner.
4. Significantly vary your copy
Go big or go home. Slight word changes won’t necessarily give us enough of a true variable. Be radical with copy changes. If we’re spending the time and money to test differences, be sure they’re clear enough to users to determine if the change should really be made. If two to three radical variations can be tested against the control, make it happen!
5. Test, refine and test again
Test the alternate copy against the control (there are lots of different software suites and services that you can use to do A/B testing or you can do it yourself through something as simple as CGI Scripting). Ideally, each copy/persona will be tested against every other variation, but if you don’t have the funds or it becomes impractical to run multiple tests, test two pages at a time and keep the best as your control for subsequent tests as mentioned above.
In a perfect world, our brands, bloggers and friends have the time and the resources to follow a process like this and perform true split testing, but even if we have neither we can still create sequential A/B testing through throwing up one version of our site with one version of copy for a given period and then test alternative versions for the same time period after gathering data. Results may not be as reliable as true A/B split testing, but we can still gather incredibly valuable information from the exercise.
Copy testing will help us maximize conversion rates, solve site problems, and challenge our assumptions. If you’ve got a fussy client, who continually wants to beat his chest about a product claim, good A/B testing might just show that all the user really cares about is what color it may be or the fact that it fits into their back pocket. And if we can start showing wins on this level, we can open the door for HUGE opportunities when we get beyond testing small changes.
Once initial factors and bugs in content are worked out, we can do bigger things, like designing and writing radically different versions of our pages, for brand new personas, where almost everything is different. And when we can test dramatic changes for new audiences, we’re most likely to achieve breakthrough improvements in conversion rates and potentially that all-important ROI.