Content Strategy Gut Checks: First Impressions Testing

Content Strategy Gut Checks: First Impressions Testing is the third in a series of six posts discussing the testing of content and content strategy models in usability and user testing. Did you miss the first two posts?

Read Part One: The Café Test
Read Part Two: The Focus Group


You’ve got butterflies in your stomach. It’s a nervous, happy, scared out of your mind (but deliriously excited all at the same time) rush. You’ve spared no expense in sprucing yourself up and have taken care to be sure everything is enticing to the eye.

No doubt about it, you’re looking hot. But when users start knocking at your virtual door for their first date, will your content be the horrible garlic breath that turns them off or will they find the spark that keeps them coming back for more of what only you can truly offer?

Just as in dating or a job interview, a first impression can be the most lasting, which is why taking the time to test for them is crucial — both for the visuals and the content.

When To Use First Impressions Testing

As far as I know, “First Impressions Testing” isn’t exactly a formal “usability” test. I’ve always used it as a field test that can be combined with, or performed separately from, the Café Test.

They’re best used early in the web design process or when you need to capture first impressions on a new addition to a site. I also find them valuable for form and e-mail testing. The first impressions gathered are analyzed to determine whether initial reactions have colored a user’s feeling about the remainder of the site/email/etc. First impressions testing that is specific to content should be focused on subjective measures, which could include:

• A user’s satisfaction or dissatisfaction with page content
• A user’s comfort and understanding of content concepts
• A user’s thoughts and impressions about the tone and understanding of the context of the content within the design
• A user’s self-reported thoughts about the purpose of the site and content

How To Get Started

First impressions testing can be performed in a variety of environments and in a variety of ways. There are a few remote services that provide this type of usability testing (e.g. Optimal Workshop’s ChalkMark). You could also contract a testing lab if you don’t have a lot of strong experts in house, but I’m of the opinion that more often than not you don’t need a formal lab to perform first impressions testing.

Testing by Trinity

Setup for a first impressions test is similar to the café test. You can stage in a high traffic area, like a café (preferably one where your target user might be) to approach potential users or invite a section of existing users to a conference room in your office, etc. You can also do this test remotely through a conferencing application. Just be sure to test users one at a time.

If they’ll allow you to do so, take video or photos. If you’re using a laptop, use the onboard camera to record facial expressions. You don’t need a separate moderator, but it helps to have someone take notes when you reach the question portion of the test.

Your willing participant should be seated facing your device of choice with nothing on the screen and then shown the homepage/page/application/etc. for five to 90 seconds. If I were only testing the design, I’d do five to ten seconds maximum, but since we’re talking content here, give them a bit longer to see what they focus on first.

Once the time is up, hide the site and ask the user to begin relating everything they can recall from the page.

Questions, Questions, Questions

When asking the participant to relate their first impressions, focus your questions on subjective measures. Be sure not to be too leading or to use any language that might influence their answers. You want a true first impression, not something you’ve potentially influenced. Ask them to recall everything they can from their short experience with the testing material. Questions can include but aren’t limited to:

• What was the purpose of the [content] on the site?
• What were the key takeaways of what you read/saw/heard?
• Did you understand the content on the page?
• What were the first things you noticed when the page appeared?
• Can you recall or describe the mood of the site?
• How does your overall impression of this [content] influence your perception of the site/product/etc?

Deliverables

Key deliverables from a first impressions test will be qualitative reports. It’s fine to detail a day’s worth of testing into a single report, but sessions can be broken out by individual if you wish.

If you videotape the session, use clips and captures in your reporting to bring back to designers and content stakeholders. Just make sure you capture all of the thoughts, feelings and end with how those impressions color a user’s opinion of what the experience is as a whole.

Summing It Up

Testing first impressions for the content of the site is tricky because a user may naturally be drawn to site visuals prior to diving into the content. That being said, any qualitative data you gather during first impressions testing should be taken for what it is — a field test.

Use those impressions to be sure you have the right calls to action, the right amount of space allocated for content and the right mix of visuals to put content in the right context based on user expectations. No one wants to be the one with the garlic breath and you don’t want your user’s first impressions to cloud his or her perception of what you have to offer down the road. So test to be sure you can make a first good impression before you toss yourself to the world.

“Disgust” (photo) by Jeremy Brooks. Used via CC BY-NC 2.0 License.

“Testing” (photo) by Rebecca Partington. Used via CC BY-SA 2.0 License

Content Strategy Gut Checks: The Café Test

Content Strategy Gut Checks: The Café Test is the first in a series of six blog posts accounting for the testing of content and content strategy models in usability testing.


Content strategy can be a long and drawn out process. There is an incredible amount of work that goes into developing and implementing content strategy, so it always frustrates me to learn that many clients are being provided with usability tests that only focus on the user interface and the navigation of a Web site.

Cup of Coffee and iPad in a German Cafe

This isn’t entirely surprising, as many sites are not created using a “content out approach (see also: @Malarkey), and often times content testing isn’t suggested or performed until post launch — a disastrous folly in its own right.

I’m not advocating that EVERY site should be designed in a content out way, but I do believe that content strategy and its eventual product are deserving of some very specific questions and equal time in usability testing. And one of the best methods for getting a gut check for a site’s content (and content strategy), especially in the early days of your project, is the Café Test.

How To Get Started

If you’re not familiar with café testing, it’s exactly what you would assume it might be. It requires the tester to plop himself in a high traffic area, like a café (preferably one where your target user might be) to approach potential users. The person giving the test should use a sign to attract potential participants or come up with a clever hustle to draw them in. Coffee, tea, beer or $5 usually does the trick.

Once you have a properly imbibed and willing participant, ask for a few demographic questions. How old are they? What is their education level? Have they participated in a study like this before? What do they do for a living? Have them sign a consent form if your company/client requires it (I’ll provide an example consent form a in my next post).

After you’ve got the basic info and signed consent, load up your site via laptop, iPad or your mobile and let them spend some time interacting with your content, not just the site itself.

Should your study require it, direct them to perform a few tasks using the site’s intended UI and PAY CLOSE ATTENTION. This is where café testing gets a little anthropological. Take photos or video if they’ll let you. Study how they interact with the site, paying close attention to how much they read, what their initial impressions are and how they move through the content. Allow them a good five to 10 minutes to interact with the content and then follow their play time with a few questions.

Questions, Questions, Questions

Café tests should always ask specific questions about the content, but they should be kept simple. Remember, this isn’t a focus group, it’s a gut check to see if we’re on the right track and we’ve only bought them a cup of coffee. If we bought them dinner or paid for them to miss work, we could take greater liberties with our requests.

Some sample questions could be:
1. How easy was the [content] for you to understand?
2. Did you believe the [content] was interesting? Explain why or why not.
3. What was the most helpful piece of [content] on the page?
4. Could the [content] have been organized, written or presented in a different way to be more useful?
5. What do you think the [content’s] intended purpose was?

(NOTE: [content] could be anything in this case. It could be a specific video, form, photo gallery, text, etc.)

These questions should be written up or recorded and then organized into a report.

Deliverables

The goal of the café test is to obtain a gut check and document the informal glimpses into the perceptions of the public when it comes to your site and its content. They are most useful when tasks are simple enough that formal usability testing may be overkill, or when time is of the essence (like, we launch next week OMG!). The downside is that the target users are not always available.

Deliverables for site stakeholders and (maybe) clients include: spreadsheets, photos, videos, quotes and qualitative comments.

Summing It Up

During the initial stages of your project, café testing can be especially helpful because it’s relatively cheap, and the results will help you get priorities in order or disprove a hypothesis about user behaviors or their potential for comprehension early on.

That said, café testing requires three things to be really successful — well-defined questions, a reasonable testing window (no more than 15 minutes of your subject’s time should be taken) and a decent sampling of users that meet your demographic. If you don’t know where to find them, ask a user. People like to herd and flock, so chances are your users can point you in the right direction.

While the café test isn’t the most scientific of models we could use to analyze our users and content, it starts to give us some early impressions and further hone our strategic goals as we build our site. They’re also a lot of fun to administer and who knows, you just might make a new friend.

Next up, we’ll take a look at First Impressions testing. Stay tuned.

Photo By: Johannes Kleske. Used via: Creative Commons: Attribution-NonCommercial-ShareAlike 2.0 Generic