Content Strategy Gut Checks: The Focus Group

Content Strategy Gut Checks: The Focus Group is the second in a series of six posts discussing the testing of content and content strategy models in usability and user testing. Did you miss the first post? Read Part One: Content Strategy Gut Checks: The Café Test.

“The wise man doesn’t give the right answers, he poses the right questions,” — Claude Levi-Strauss

The way I see it, Claude Levi-Strauss’ statement sums up how I view testing, and (in a way) content strategy. The questions we craft and ask of users are crucial for informing the type of data we ultimately use and produce for our digital experiences. And when it comes to posing those questions to gut check our content and content strategy, one of the best tools at our disposal is the focus group.

For starters, I think it’s important to point out that focus groups are NOT usability tests. Focus groups are what I refer to as user tests or user dialogues, which have very different goals from usability testing. A good usability test should focus more on observation and provide us answers with how well a user was able to use and experience both the interface and the data itself. Conversely, if we want to assess users thoughts, feelings or attitudes about a product or our Web site, we’d leverage a focus group.

It’s the difference of what users say vs. what they do — and in the best of all possible worlds, we’ll be able to leverage both insights when planning for content strategy.

When To Use Focus Groups

I personally believe focus groups should be performed early in any web project to both help discover insights into your target audience and prove out any assumptions you might have made regarding their Personal-Behavioral Context or Situational Context (did you really think you’d get through a post on my site without me making a plug for context?).

Also, consider using a focus group if:

How To Get Started

Invite 6 to 12 people to participate in for each focus group session. Depending on your budget or the scope of your project you may need several sessions to get a representative sampling of your targets. Pre-screen to be sure participants are from your target via questionnaire. It is absolutely crucial that the people you invite be from your target demographic. If they aren’t, you’re wasting time and client dollars.

In your invitation to participants, do your best to provide a high-level agenda and include any issues that you’ll be tackling during your session. The focus group itself should last 60 to 90 minutes (any longer and you’d best buy them a meal and plan to bring them back the next day).

Prepare up front. State the purpose of the focus group and provide an outline to the day’s activities. If it’s required of your client (or if you just want to cover all your bases) have them sign a consent form after you’ve given your explanation (e.g.: [PDF]) and bring it to the group. Collect your consent prior to giving them access to other members of the group or the focus group room.

Set up the focus group in an room or location that offers little to no distraction. You want the participants’ full attention since the end results will be analytical reports. Try to set your test group around a table to encourage conversation. You want the group to have the ability to make chatter and if it’s a circular or ovular table, you’ll have a better vantage point to document facial reactions or pose immediate follow up questions.

Before you begin the questioning, ask the participants to introduce themselves and/or wear nametags. Focus groups are a tag team effort (you need a strong moderator and someone to document findings, discretely if possible). It’s the moderator’s job to be aware of the energy in the room. They also need to step in if one person is dominating a conversation and allow for cognitive breaks when it appears they’re needed. The moderator has to keep discussions flowing and keep the group focused on the issues you want to document.

The recorder should only focus on documenting the findings, and he should capture facial expressions, audio, notes on findings etc. I’ve always found it helpful to color code notes per participant or attach a headshot to individual notes when I’m acting as a recorder in a focus group.

Questions, Questions, Questions

A focus group will only be successful if the questions asked are open and neutral. The wording is crucial, so channel your inner Claude Levi-Strauss and be the wise man. Pay special attention to the inflection and tone taken posing a question to the group. The wrong wording or inflection might taint the responses.

Ask the target audience about how they use the web, what their expectations are of what types of content would be on your Web site. If it’s a new section, get their thoughts on how successful their efforts have been in seeking the proposed content. If they have used that type of content, document their experiences. What worked? What didn’t? What would they have preferred to see? It’s here that we start to find the bits that we can apply to situational context and scratch at those oh so elusive behaviors.

Other questions a moderator could pose to a focus group that help influence content strategy include:

There are tons of questions and paths that can be followed, but that would be a MUCH longer post.

Deliverables

Focus groups are for gathering thoughts, feelings or attitudes. That means you need qualitative analysis reports. These should be written for each session. The reports should contain the relevant background of participants who attended individual groups. Dedicate a single report to each session and be sure to have any questions you may have gathered in the screening questionnaire included in your reporting as well.

If you videotape the session, use clips and captures in your reporting to emphasize thoughts and support any hypothesis you had about content, data and design needs.

The more data you have to work with, the easier it will be to make relevant Behavioral/Situational personas to apply to your content strategy project. Client deliverables might include an executive summary, or quotes and images from the session, but the full report should be more useful to your design team and content/digital strategists.

Summing It Up

Focus groups are meant to help predict consumer responses to a site or feature. It’s crucial to know how consumers feel about a project prior to really getting down to the heavier design phase and focus groups are a great forum to getting those feelings out into the open. They require patience and a really solid moderator who can manage conflict and keep the group on the task at hand.

I fully believe great focus groups can be done independent of agencies that specialize in it. An independent content strategist need only be sure he or she has specific plan and goals in mind prior to doing the focus group. If you don’t think you can handle the moderation, find someone internally who can manage conflict or multiple personalities.

Even if you do select an agency to perform your focus group testing, make sure you have influence over the questions asked. A good content strategist should walk away knowing the situations that will call for content and have a better idea of the mix that will be needed to address those situations.

What are your experiences with focus groups? Do you find them useful in planning for content strategies? Drop your thoughts into the comments below.

Content Strategy Gut Checks: The Café Test

Content Strategy Gut Checks: The Café Test is the first in a series of six blog posts accounting for the testing of content and content strategy models in usability testing.

Content strategy can be a long and drawn out process. There is an incredible amount of work that goes into developing and implementing content strategy, so it always frustrates me to learn that many clients are being provided with usability tests that only focus on the user interface and the navigation of a Web site.

This isn’t entirely surprising, as many sites are not created using a “content out approach (see also: @Malarkey), and often times content testing isn’t suggested or performed until post launch — a disastrous folly in its own right.

I’m not advocating that EVERY site should be designed in a content out way, but I do believe that content strategy and its eventual product are deserving of some very specific questions and equal time in usability testing. And one of the best methods for getting a gut check for a site’s content (and content strategy), especially in the early days of your project, is the Café Test.

How To Get Started

If you’re not familiar with café testing, it’s exactly what you would assume it might be. It requires the tester to plop himself in a high traffic area, like a café (preferably one where your target user might be) to approach potential users. The person giving the test should use a sign to attract potential participants or come up with a clever hustle to draw them in. Coffee, tea, beer or $5 usually does the trick.

Once you have a properly imbibed and willing participant, ask for a few demographic questions. How old are they? What is their education level? Have they participated in a study like this before? What do they do for a living? Have them sign a consent form if your company/client requires it (I’ll provide an example consent form a in my next post).

After you’ve got the basic info and signed consent, load up your site via laptop, iPad or your mobile and let them spend some time interacting with your content, not just the site itself.

Should your study require it, direct them to perform a few tasks using the site’s intended UI and PAY CLOSE ATTENTION. This is where café testing gets a little anthropological. Take photos or video if they’ll let you. Study how they interact with the site, paying close attention to how much they read, what their initial impressions are and how they move through the content. Allow them a good five to 10 minutes to interact with the content and then follow their play time with a few questions.

Questions, Questions, Questions

Café tests should always ask specific questions about the content, but they should be kept simple. Remember, this isn’t a focus group, it’s a gut check to see if we’re on the right track and we’ve only bought them a cup of coffee. If we bought them dinner or paid for them to miss work, we could take greater liberties with our requests.

Some sample questions could be:
1. How easy was the [content] for you to understand?
2. Did you believe the [content] was interesting? Explain why or why not.
3. What was the most helpful piece of [content] on the page?
4. Could the [content] have been organized, written or presented in a different way to be more useful?
5. What do you think the [content's] intended purpose was?

(NOTE: [content] could be anything in this case. It could be a specific video, form, photo gallery, text, etc.)

These questions should be written up or recorded and then organized into a report.

Deliverables

The goal of the café test is to obtain a gut check and document the informal glimpses into the perceptions of the public when it comes to your site and its content. They are most useful when tasks are simple enough that formal usability testing may be overkill, or when time is of the essence (like, we launch next week OMG!). The downside is that the target users are not always available.

Deliverables for site stakeholders and (maybe) clients include: spreadsheets, photos, videos, quotes and qualitative comments.

Summing It Up

During the initial stages of your project, café testing can be especially helpful because it’s relatively cheap, and the results will help you get priorities in order or disprove a hypothesis about user behaviors or their potential for comprehension early on.

That said, café testing requires three things to be really successful — well-defined questions, a reasonable testing window (no more than 15 minutes of your subject’s time should be taken) and a decent sampling of users that meet your demographic. If you don’t know where to find them, ask a user. People like to herd and flock, so chances are your users can point you in the right direction.

While the café test isn’t the most scientific of models we could use to analyze our users and content, it starts to give us some early impressions and further hone our strategic goals as we build our site. They’re also a lot of fun to administer and who knows, you just might make a new friend.

Next up, we’ll take a look at First Impressions testing. Stay tuned.

Photo By: Johannes Kleske. Used via: Creative Commons: Attribution-NonCommercial-ShareAlike 2.0 Generic

Evaluating Content: Reading Level

Does your average target user understand the writing found on your Web site? Did you, or your content author, think to evaluate the reading level of your copy prior to publishing? Yeah, I didn’t think so.

If you’re a large organization farming out your Web writing or if you’re a small business with a DIY approach, I’m willing to bet that the author of your content is fairly educated. And while said author(s) can certainly write beyond the average high school student’s reading level it doesn’t mean they should.

One of the biggest problems I discover when I’m tasked with evaluating Web content lies in the reading comprehension level of copy. More often than not, it can be simplified, edited for brevity and structured into smaller paragraphs. Multisyllabic words (how’s that for confusing/technical word choice!) are often presenting reading problems and most of the copy is not optimized for scanning.

Too many sites are just too damn hard to understand for someone trying to find information, make an informed decision or complete a transaction (you know, the things that ultimately lead to that elusive ROI?).

Pharmaceutical manufacturers Web sites are notorious for having readability problems in copy. We’ll use drug maker Glaxo Smith Kline (Advair, Boniva, Paxil, Valtrex, etc.) for our evaluation purposes in discussing the importance of readability.

As you can see by the quick report above, GSK.com has an average reading level of grade 20. So, in theory, to understand the majority of the material on the site, one would have to have significant postgraduate education.

I may be going out on a limb here in assuming that a good chunk of the consumers who use GSK drugs and any potential investors may not have achieved masters or doctoral degrees. So, it should follow that when the less educated are reading through potential side effects of drugs, or are scanning for quick information about the company; this copy isn’t going to make them feel more at ease in using a GSK manufactured medicine or convince them the company is worth their investment dollar.

The problem with GSK mostly lies with a high instance of confusing words and complex sentences. More than 27 percent of the words found on the GSK site are “Complex” (Complex words are polysyllabic – having 3 or more syllables). Even worse are the more than 175 instances of long sentences (10-20 words or more).

How do you improve it? Simplify, my friends. Simplify.

Start aiming for writing copy that younger audiences will understand. I don’t believe that there’s necessarily a magic number, but I generally shoot for 6th to 8th grade. Most journalism schools teach new copywriters to scribe for that comprehension level. It should be no different for most Web sites, unless you have a highly specialized audience (think advanced Web programming, Law Libraries, etc.).

Granted, most drug manufactures create microsites to be more consumer facing and in line with the marketing campaign (Valtrex.com), but even these sites are writing to a high school reading level and using high percentages of confusing words. (Valtrex.com is appropriate for 11th graders FYI)

I’m also of the belief that if corporate sites are in your linking strategy on consumer facing sites (as is the case for GSK.com), you need to have both readable by the same audience, which is clearly not the case here.

Is pharmaceutical stuff sometimes difficult to understand? Absolutely. Do drug names often contain multiple syllables? Absolutely. Does that mean the rest of the content (including descriptions of symptoms the drug treats and potential side effects) needs to be difficult to understand? Absolutely not.

Having trouble getting there, or don’t know how to even get started in evaluating copy readability? It’s not really an exact science (that’s what great editors and content strategists are for), but fortunately, there are lots of nifty tools out there to evaluate this stuff. If you’re looking for a quick evaluation of a page on a site, Read-Able.com is a strong, visually appealing site to generate reports on existing reading levels that will help get your pointed in the right direction.

And, if you’re going to get started in evaluating reading level in web copy, here’s the three things you absolutely need to pay attention to, whether you hire a content strategist or not.

1. Flesch Kincaid reading level. This test is sort of the standard. It’s built into many word processing programs and will give you a general idea of how difficult the content is to digest. If the reading level is too high, cut back on confusing words, simplify sentence structures and utilize elements that make the page easier to scan. Too low? Beef it up a little. Too many syllables in you words? Try breaking them apart or separating thoughts.

2. Make your content brief! More often than not, site owners are trying to reach mass capacity with web content to be better optimized for search. It’s a major disservice to your users and to your governance plan. If you’re using 10 words, do it in five. Don’t use a paragraph when bullets or a diagram can say it better. Utilize checklists, how tos and walkthroughs instead of detailed descriptions or confusing language.

3. Use the common sense test. I know, this is asking a lot, but just like you know pornography when you see it, you’ll know confusing copy when you see it. Don’t be smart for smart’s sake. If you know an eighth grader, ask them if they get the gist of your copy. If they don’t, chances are a good chunk of your visitors don’t get it either.

What tools is your organization using to evaluate reading level? Is it something your strategists are building into their Web toolkits? If not, maybe it’s time you ask them to start doing so, because you may be missing out on conversions or customers.

Image courtesy Brian Talbot

Avoiding Missteps: Accounting For Testing and Refinement in Content Strategy

Sometimes, first dates can leave us with mixed feelings. We take special care to make sure no hair is out of place, that we smell nice and do our best to make sure everything goes according to plan. First dates can be an expensive, exhausting and – depending on your skills – either be a very rewarding or entirely dissapointing experience.

Practicing content strategy for the first time is very much like a first date. It requires careful planning, a lot of get-to-know-you-type conversations and, ultimately, will probably cost you a little more money than you expected it to. If all goes well, that strategy will pay off and bring many years of happy returns, but like any new relationship, content strategy takes time, examination and refinement.

It’s here that many strategies, and relationships for that matter, fall down. If we don’t account for reflection and refinement, we can’t determine how successful we could really be.

Good content strategists start with an audit and inventory of all the content you currently have. This process can be done specific to your online properties, if you’re looking only at the Web/Mobile/Location mediums or acrosss your entire organization at the enterprise level. From that initial audit and inventory, gaps should have been identified and opportunities to refine existing content to fit existing or new audiences (personas) would have been properly communicated to the you, the client.

Executing the production of new content according to our strategy would be the next step. If they’ve done their job, your content strategist will have synched up with your analytics team and determined some success metrics for our content. And with metrics, content that’s on strategy and a usable site in place, everything should go swimmingly right? In theory, absolutely. Here’s how you go about testing to find out if that’s really the case.

So, here’s how you start setting up for the review of those success metrics to determine how to further test and refine your process and existing content.

1. Test to find out if your content is easily consumed

You spend time testing your Web site’s usability, shouldn’t your content get equal treatment? Some kind of test needs to be put into place to test the viability of your content. It’s not as simple as increasing clickthroughs or user time spent on your site, though that will provide an initial baseline as if it’s even being found.

Start with basic questions. Is the content readable? Is it too long (this applies to video or text) Do your users understand it? Could re-wording things be the key to creating influence? Is your message solving your user needs or potential problems? These are all questions we can start to answer with A/B Testing, or simple focus groups. You can be as scientific (think A/B testing, Eye Tracking or utilizing fMRI) or as basic as you want (Usability Testing, User Interviews) to be when it comes to testing for whether or not your content can be easily consumed. The name of the game here is not to launch and leave it, assuming that our strategy is the right one.

2. Take Personal and Situational Behaviors Into Consideration While Testing

Are your personas working hard enough for your content strategy? Did the content strategist account for personal behaviors when developing a content plan? What situations were generated as potential scenarios that require content? If context were acconted for in the up front planning, we’d have already accounted for factors beyond basic socio-economic and media consumption habits and looked at our content as a task or fuciton that helps address a need to a specific user situation. If we haven’t, that should be accounted for in testing for refinement as context will always make our content more useful, meaningful and relevant to users.

Test, refine and test again
This process is never over. You have to keep at it. As content strategists, we owe users increased levels of context and usability. Our job is to not only get them to our Web sites, but to make their lives easier, answer their questions before they have them and leave them feeling satisfied with the overall experiences on our sites. Content is the vehicle to that satisfaction, so we need to keep testing it. Just like that relationship… we’ve gotta keep on keepin’ at it.

Photo used under creative commons license. Photographer: Stuart Bell