On The Importance Of A/B Copy Testing In Content Strategy

Written by Daniel Eizans on 03/22/2010 – 9:05 am -

I have a love/hate relationship with A/B Split Testing, especially when it comes to Web copy. Love that A/B testing can deliver significantly improved response, but hate that many brands may base all future copy decisions on a single test that delivered or over delivered on expectations.

Relying on a singular result, creates missed opportunity to refocus or edit content for other circumstances, site users, time periods or changing business factors. This is why it is crucial to have a sound content strategy to help determine variables, governance and success metrics for copy based on the user personas that were developed for your Web site.

A/B Testing

If we can agree that content is your Web site’s greatest asset, the user persona should be the guidepost you’re using to increase its value any time we change messaging, and we can validate this premise through A/B copy testing.

And only through repeated and frequent testing will we be able to make changes that help us:

  • Understand visitor behaviors and priorities when they visit our sites
  • Solve specific copy problems (e.g. poor performing calls to action) we have with individual pages
  • Dramatically challenge assumptions we have made about a persona or content consumers
  • What factors should be considered in A/B Split Copy Testing?

    1. Start with a metric in mind.
    What are you trying to accomplish with the test? Are you after more subscribers, conversion rate increase, or a greater return on investment? Just like wanting to know what we want our users to do helps us define content strategy, goals for testing will determine parameters, which in turn will determine the potential success of our efforts.

    2. Establish a control copy page/persona
    Think back to your elementary school science class friends. Establishing a control persona will help us to establish the copy that we will test all varitions against, always keeping step one in mind as we develop considerations for variables.

    If you are just getting started with A/B testing, your control page will be your current copy that is underperforming before any variation is served. When new copy outperforms the existing control copy, consider it your new benchmark (control persona) in any subsequent testing.

    3. Determine a reasonable interval for the test
    Determine how you’ll gather the data and for how long you need to gather it. This time period will vary from site to site, but should allow for the gathering sufficient data to gauge real insight about your A/B tests. If your site has a lower number of daily unique visitors, the test may run significantly longer to determine a clear copy winner.

    4. Significantly vary your copy
    Go big or go home. Slight word changes won’t necessarily give us enough of a true variable. Be radical with copy changes. If we’re spending the time and money to test differences, be sure they’re clear enough to users to determine if the change should really be made. If two to three radical variations can be tested against the control, make it happen!

    5. Test, refine and test again
    Test the alternate copy against the control (there are lots of different software suites and services that you can use to do A/B testing or you can do it yourself through something as simple as CGI Scripting). Ideally, each copy/persona will be tested against every other variation, but if you don’t have the funds or it becomes impractical to run multiple tests, test two pages at a time and keep the best as your control for subsequent tests as mentioned above.

    In a perfect world, our brands, bloggers and friends have the time and the resources to follow a process like this and perform true split testing, but even if we have neither we can still create sequential A/B testing through throwing up one version of our site with one version of copy for a given period and then test alternative versions for the same time period after gathering data. Results may not be as reliable as true A/B split testing, but we can still gather incredibly valuable information from the exercise.

    In Conclusion
    Copy testing will help us maximize conversion rates, solve site problems, and challenge our assumptions. If you’ve got a fussy client, who continually wants to beat his chest about a product claim, good A/B testing might just show that all the user really cares about is what color it may be or the fact that it fits into their back pocket. And if we can start showing wins on this level, we can open the door for HUGE opportunities when we get beyond testing small changes.

    Once initial factors and bugs in content are worked out, we can do bigger things, like designing and writing radically different versions of our pages, for brand new personas, where almost everything is different. And when we can test dramatic changes for new audiences, we’re most likely to achieve breakthrough improvements in conversion rates and potentially that all-important ROI.


    Tags: , , , , , , ,
    Posted in Content Strategy | No Comments »

    Why Your Web Content Strategy Should Be Driving Enterprise Content Strategy

    Written by Daniel Eizans on 02/23/2010 – 8:37 am -

    Pssst … Hey you. Yeah, you with your marketer hat on. Is your Web content becoming more and more important to your client? Is your client upping her spend on content creation for the Web?

    Web Content Strategy Drives Enterprise Content Strategy
    If the answer is yes, and your Web content strategy hasn’t started influencing (if not shaping) your enterprise content strategy, you could be in for a rude awakening.

    For good reason, content strategy and user experience design have a nice long process in place to assist in content and site creation. Through persona development, messaging mix and the creation of a content governance plan, we should be able to create content that can be delivered as scattershot as we’d like or as precisely as the bullet from the Army’s best snipers. We already know a lot of folks don’t follow these processes when they develop new content for the web, but if you do, what’s stopping you from flipping the process on its head and applying it to your enterprise work?

    Here are just four (and there are a lot more) reasons why you should allow a well-established Web content strategy to start influencing the enterprise stuff:

    1.Good Content Strategy starts with personas, which essentially means you know the individual needs, drivers and habits of the various potential content consumers (customers, etc.)

    Who wouldn’t want to start delivering different enterprise materials to their audience based on their habits? It has the potential to prevent creep on project scope, reduce printing costs or highlight deficiencies in your enterprise messaging mix. If we have a better understanding of who the content consumer is, chances are we may even be able to eliminate certain messaging mediums altogether. Does 89-year-old Uncle Morty really need a big glossy photo in the local city magazine for a sale on his bed sore cream? Probably not.

    In bed-stricken Uncle Morty’s case, should we really be messaging to his caregivers? What are the best ways to do that in a non-digital way? Personas and a content plan based on those personas help determine exactly what we should be producing.

    2. Governance Plans keep messaging fresh and on target.

    How many times do you see the exact same asset, or message or mailer from a company on a weekly or monthly basis if you even notice it in the first place? It doesn’t take too long for some messages or assets to get incredibly stale. Governance Plans are in place on the Web to combat this and they can certainly be used for non-digital communication plans. We don’t have to do look any further than our daily stack of mail and the custom publications we receive from brands we’re affiliated with to see whether or not the content we’re being served is still being effective.

    Good Governance Plans put expiration dates on content.

    3. Adoption Of The Web Approach For Enterprise Solutions Leads To Accelerated Release Of Content To The Market

    Marrying your Web content strategy to your enterprise approach allows for content that is rooted from a proven process. Through synthesis of strong consumer, social and product insights that lead to several personas, content creators have shorter creation cycles due to laser focused content goals and reduced maintenance cycles.

    Our creators spend far less time repeatedly authoring new content because they reuse existing content wherever possible, supplementing it with modified content where appropriate. (Uncle Morty only needs you to change some sentence structure to speak from a position of authority about bed sore cream as opposed to trying to educate him about its potential benefits).

    With a unified process in the driver’s seat, Editors also spend less time reviewing content across channels because they only have to review the content that is new or changed; existing content has already been reviewed and signed off. With more work done on defining needs up front, we can drastically affect the publishing and production cycles by reducing the scope for our content creation teams.

    4. Content rooted in strategy is BETTER CONTENT!

    This one sort of seems like a no brainer, but it’s worth pointing out. We’re not guessing or doing creative solutions for the sake of doing creative solutions because our reasons for doing print, web, broadcast, mobile apps or any other medium are clearly defined. Content that is clearly modeled for consistent structure; improves things like readability and usability. Most importantly, content is accurate and consistent wherever it appears. Issues of inaccurate content, inconsistent content, or missing content are reduced or eliminated because it all has a reason to be there.

    This process is incredibly repeatable and can be applied to a variety of organizational situations. Think about marketing materials, advertising, internal Web sites, corporate communications or even your news releases. It’s time to stop thinking of the Web content strategy as a separate monster and start thinking of it as the driver for a more unified publishing vision.


    Tags: , , , ,
    Posted in Content Strategy | 6 Comments »

    Content Rule of Thumb: If You Feel Full Reading, Stop Feeding

    Written by Daniel Eizans on 01/08/2010 – 2:47 pm -

    There is probably no SEO misconception that I despise more than the “more is better” argument. A good rule of thumb when it comes to content on your web site? If you’re feeling tired (and full) when you’re reading your content, stop feeding it to your visitors.

    Full

    A very wise woman recently wrote:

    “Online, when it comes to informational, marketing, or promotional content, more is almost never more.”

    We need look no further than a pair of popular cleaning product makers to see this process in action. When we examine the homepage for Lysol and compare it to Clorox, a few clicks will show just how much more content is jammed into one versus the other.

    As you can see below, Lysol’s site is jam packed with Did you Knows, Germ Information, PDFs, menus, sub-menus and enough cookie crumbed pages to choke a small child.

    Lysol.com - Home & Family Tab

    Conversely, the Clorox page features a great deal more whitespace, has fewer downloadables, and no submenus linking into deep pages. It also segments users based on their cleaning locales of interest (someone did their personas!).

    Clorox.com - Healthier Living Tab

    So, who has the better SERP ranking? Clorox beats the crap out of Lysol with far less content.

    Clorox.com - Cleaner Home Tab

    A Google search of the keywords “Cleaning Products” put Clorox.com number one in the sponsored links and the top result of actual products (7th in the organic list). Lysol didn’t appear in the organic search on the first page, nor did it on the next 20. While it appeared in the sponsored links underneath Clorox, I didn’t have the heart to keep digging to see if they made the organic SERPs within 40 clicks.

    Not only are the Clorox.com search results better, it’s also far easier to find the information you’re looking for. The fonts are larger, the headlines are shorter, written specifically for a web audience and sub headlines provide an accurate description of what we’ll find within a link or article page. Product information is weaved in nicely with informational articles, there are graphics that engage users while informing and useful tools that help decide the best way to treat stains.

    Bottom line, good content helps improve search ranking, but it has to be useful, tagged and relevant.

    So before you go off and publish a mammoth Web site with tons of pages, articles and information, with high hopes of a higher SERP ranking, ask yourself a few questions.

    • “Does this information add to my brand story, user experience or increase engagement?”
    • “Would I lose anything if this content wasn’t here?”
    • “Can this information be better organized, bulleted or edited down?”
    • “By including this content, am I preventing my users from getting content they need?” (THIS ONE IS MY FAVORITE!)

    Useful, usable Web sites are not not about providing every single piece of information that anyone could ever think of. It’s about providing solid content that is on strategy, that means something to your visitors and that is properly tagged optimized and placed to be useable for machines and humans. Tag better, write better, and spend more time optimizing and editing less content to get more of a result.

    If you get overwhelmed and feeling too full looking at your site, chances are your users feel exactly the same way.

    Photo: Sandeep Nair


    Tags: , , , , , ,
    Posted in Content Strategy | 1 Comment »

    Who’s Responsible For Content That Expires?

    Written by Daniel Eizans on 12/17/2009 – 1:14 pm -

    When it comes to your Web site content, who is responsible for removing it after its lifecycle? For many of you publishers out there, I’d wager that you treat your expired content the same way you treat your recently deceased and always distant Aunt Mildred … Whoever has the bandwidth to get the job done gets stuck with figuring out what to do next, and more people than necessary will ultimately have to become involved.

    We owe expired content and our expired Aunt Mildreds a lot more respect than that my friends. Had we only had taken the time to plan for the end of their lives ahead of time, numerous people’s time and productivity wouldn’t be taxed by these untimely deaths.

    Enter the Content Strategy, enter the Governance Plan!

    As I stated in a post from a few weeks ago, I’m of the belief that good content strategists forge fruitful relationships with content managers. One of the most important facets to this relationship is related to the governance of content. As content strategists, we’re essentially serving as the funeral director for our content managers tasked with dealing with expired content on our sites. Through various touchpoints and reviews, gap anlaysis and a complete content inventory that maps the site, a good content strategist should be able to establish all of the things that need to be accounted for when a piece of content expires. The strategist will be able to answer all of the manager’s funeral questions. Some examples you ask?

    • Does the content link to other critical content on the site? (Does it serve as a gate to non-expired content)
    • Has the content been embedded on or linked to from partner or vendor sites?
    • Will the removal of this content create a gap in our messaging plan?
    • Does this content simply need a refresh as opposed to complete removal?
    • Will the content need to be archived?

    Those are just a few things that come to mind. Believe me, there’s a lot more. Just like when dear old aunt Mildred died, there are tons of things you never seem to think about until content is gone.

    So, why haven’t we been preparing for the death of content and the death of aunt Mildred?

    I believe most organizations fail at governance because they expect that their content management systems or the manager of said system to do all of the lifting. They overlook the need for a governance model prior to CMS selection or workflow development, opting instead to (prematurely) dive into technical design and development, hoping the creative skinning and product capabilities will magically solve operational requirements.

    In other words, strategy for the content that goes in your CONTENT management system comes after you select the system. How incredibly backwards is this!!

    There are many schools of thought out there about how to establish a governance model, and I’m sure that all of them have some merit, but without the strong collaboration between content strategists and content managers little will be done to solve the headaches that come when content finally expires.

    If you’re interesting in reading up more on the governance process, check out Navigation Arts has some pretty insightful research, as does Razorfish (PDF).

    The moral of the story really is to plan ahead, and determine all of the factors that will occur if your content or your aunt Mildred suddenly expire.

    Have you implemented a Governance Model? I’m curious to hear the details. Comment below!

    Photo by Dave Shea (a.k.a Mezzoblue)


    Tags: , , ,
    Posted in Content Strategy | 1 Comment »

    Why Strategists Need Content Managers

    Written by Daniel Eizans on 12/04/2009 – 3:05 pm -

    Every Wednesday morning, I wander upstairs to cheat on my colleagues in the content strategy and user experience departments to broaden my knowledge base and talk about things just as geeky as our collective practices.

    You see, Wednesdays are my weekly trysts with my agency’s Content Management and Content Integration groups. I very much enjoy these sessions, not only because it provides me the opportunity to fully nerd it out on everything from asset grids to meta data, but it also affords me the opportunity to vent shared frustrations, create new efficiences and discover insights as to how we make this whole content strategy/content management thing play nice. I do this because ultimately, I’d like to better serve our clients and assist our content developers in producing more relevant and compelling creative.

    Let me start by disclaiming that I’m of the opinion that content strategy is most certainly NOT content management. As strategists, we have input on how the content is produced, managed and governed, but our goal is ultimately to aid in the creation of a strategic set of best practicies and personas to be sure that content developers are creating the most appropriate content for machines and humans.

    So, if content management is concerned with the cataloging, re-purposing and proper tagging of assets so they’re readily available and relevant, and content integration is concerned with making these assets usable for a variety of media channels, how do these guys get along with and provide insights into the content strategy role?

    The answer that I’ve slowly been coming to after a few months of getting inside the heads of various content integration and management team members is that there is an absolutely crucial need to design a common set of systems, routines and nomenclature for an integrated content development process – something myself and two colleagues are working very hard to shape.

    It seems that our separate languages can certainly be understood by one another, but somehow can become twisted in interpretations when they’re translated to those outside our happy content development cycle. So, it seems far better to step forward with a universal dialogue to properly marry content strategy and user experience to content management and integration so that POV can be articulated to the content development process laymen.

    This calls for unprecedented collaboration, which can be difficult in a large agency setting. It means content managers and strategists need to work very hard to understand the other’s practice.

    Good content strategists should have a strong understanding of how asset management and integration works (EVEN IN NON-DIGITAL!!), and asset managers and integration specialists have to understand the insights and data sets that influence how a content strategist develops personas, works with experience planning, determines gap analysis and creates a point of view for a given project.

    If content management doens’t understand the results of strategic persona process, assets can’t be tagged properly in databases and potential efficiencies for content integration across communication and campaign platforms will most likely be missed. Missed opportunity creates a creep on scope when new projects and the potential for new content creation comes around. Likewise, if content strategy doesn’t better understand the management and integration process influence on management and evenutal governance of an asset will never be achieved.

    So I’m putitng out an APB for the creation of a content development dictionary of sorts. Content producers need a common set of terms and ways to explain the roles of their colleagues and everyone needs to understand these terms, roles, processes and routines through a common nomenclature.

    Big agencies traditionally suck at this, so I’m especially interested in what you strategists working within this environment are doing to combat it.

    I know it seems silly to have to tell people to integrate into a process for content development, but as agencies and organizations attempt to become less siloed by re-aligning as content production houses, they’re effectively creating new silos by coming up with their own language, process and routine for handling individual steps to creating the work.

    So strategists … have you hugged your content manager or your integration team today? If you haven’t head upstairs and do so. They’re smart and insightful people who will make you think more deeply and cause you to write better strategies.

    Drop thoughts below in the comments and check back for periodic updates to my attempts to write the Content Development dictionary.

    Disclaimer: The opinions reflected in this post ARE NOT necessarily those of my employer. These opinoins are strictly my take on the content development process.


    Tags: , , ,
    Posted in Content Strategy | 5 Comments »