Content Strategy Gut Checks: The Café Test

Content Strategy Gut Checks: The Café Test is the first in a series of six blog posts accounting for the testing of content and content strategy models in usability testing.


Content strategy can be a long and drawn out process. There is an incredible amount of work that goes into developing and implementing content strategy, so it always frustrates me to learn that many clients are being provided with usability tests that only focus on the user interface and the navigation of a Web site.

Cup of Coffee and iPad in a German Cafe

This isn’t entirely surprising, as many sites are not created using a “content out approach (see also: @Malarkey), and often times content testing isn’t suggested or performed until post launch — a disastrous folly in its own right.

I’m not advocating that EVERY site should be designed in a content out way, but I do believe that content strategy and its eventual product are deserving of some very specific questions and equal time in usability testing. And one of the best methods for getting a gut check for a site’s content (and content strategy), especially in the early days of your project, is the Café Test.

How To Get Started

If you’re not familiar with café testing, it’s exactly what you would assume it might be. It requires the tester to plop himself in a high traffic area, like a café (preferably one where your target user might be) to approach potential users. The person giving the test should use a sign to attract potential participants or come up with a clever hustle to draw them in. Coffee, tea, beer or $5 usually does the trick.

Once you have a properly imbibed and willing participant, ask for a few demographic questions. How old are they? What is their education level? Have they participated in a study like this before? What do they do for a living? Have them sign a consent form if your company/client requires it (I’ll provide an example consent form a in my next post).

After you’ve got the basic info and signed consent, load up your site via laptop, iPad or your mobile and let them spend some time interacting with your content, not just the site itself.

Should your study require it, direct them to perform a few tasks using the site’s intended UI and PAY CLOSE ATTENTION. This is where café testing gets a little anthropological. Take photos or video if they’ll let you. Study how they interact with the site, paying close attention to how much they read, what their initial impressions are and how they move through the content. Allow them a good five to 10 minutes to interact with the content and then follow their play time with a few questions.

Questions, Questions, Questions

Café tests should always ask specific questions about the content, but they should be kept simple. Remember, this isn’t a focus group, it’s a gut check to see if we’re on the right track and we’ve only bought them a cup of coffee. If we bought them dinner or paid for them to miss work, we could take greater liberties with our requests.

Some sample questions could be:
1. How easy was the [content] for you to understand?
2. Did you believe the [content] was interesting? Explain why or why not.
3. What was the most helpful piece of [content] on the page?
4. Could the [content] have been organized, written or presented in a different way to be more useful?
5. What do you think the [content’s] intended purpose was?

(NOTE: [content] could be anything in this case. It could be a specific video, form, photo gallery, text, etc.)

These questions should be written up or recorded and then organized into a report.

Deliverables

The goal of the café test is to obtain a gut check and document the informal glimpses into the perceptions of the public when it comes to your site and its content. They are most useful when tasks are simple enough that formal usability testing may be overkill, or when time is of the essence (like, we launch next week OMG!). The downside is that the target users are not always available.

Deliverables for site stakeholders and (maybe) clients include: spreadsheets, photos, videos, quotes and qualitative comments.

Summing It Up

During the initial stages of your project, café testing can be especially helpful because it’s relatively cheap, and the results will help you get priorities in order or disprove a hypothesis about user behaviors or their potential for comprehension early on.

That said, café testing requires three things to be really successful — well-defined questions, a reasonable testing window (no more than 15 minutes of your subject’s time should be taken) and a decent sampling of users that meet your demographic. If you don’t know where to find them, ask a user. People like to herd and flock, so chances are your users can point you in the right direction.

While the café test isn’t the most scientific of models we could use to analyze our users and content, it starts to give us some early impressions and further hone our strategic goals as we build our site. They’re also a lot of fun to administer and who knows, you just might make a new friend.

Next up, we’ll take a look at First Impressions testing. Stay tuned.

Photo By: Johannes Kleske. Used via: Creative Commons: Attribution-NonCommercial-ShareAlike 2.0 Generic

5 replies
  1. Constance Semler
    Constance Semler says:

    Daniel, it’s true. Sometimes you need “triage” testing.
    Usability testing focused on tasks is relatively easy to measure without directly asking subjects questions about usability. You can see during the test itself the length of time it takes to complete a task, number of mistakes, and emotional response apparent through facial expression and comments. When it comes to testing content, though, how can one know if that obvious sort of data corresponds to the IA or to the content? It seems the only way to get at data about content is to ask people to talk about the content. Just mulling this over in the days following a class on IA.

  2. Daniel Eizans
    Daniel Eizans says:

    Constance, I agree with you in part. My struggle with usability testing (or any testing for that matter) is that there are so many potential distractions for a user. They might be too hot or too cold. They may not have had much sleep the night before they’re tested. We have no idea if they got in a huge fight with their partner minutes before they arrive.

    That said, I try to offer up two forms of testing when I café test. If I’m testing only the content, I dump the content (text or otherwise) into a wireframe environment to get a litmus test for that initial emotional response. I’ll provide half of my sampling with content in a wireframe environment and half with content married to the creative. I find that having both gives me a general reaction to how easily the content is understood and how it is taken in context with the creative elements that surround it.

    Also, usability tests are only as good as the questions you use. I tend to report more on the specific questions I have that ONLY address the content and try to make mental notes or use video to capture emotional response in this area as well.

    Obviously, emotional responses are much tougher to measure (or interpret) so making quick assumptions based on a single round of café testing isn’t advised. I like to marry the data I get from a café test with Card Sort data and Focus Groups if I have time.

  3. Rahel Bailie
    Rahel Bailie says:

    Testers, in or out of cafes, can be tired, cold, hungry, etc so it reflects real-world tensions during use. I like your approach, and have done similar guerilla testing. My favourite: testing inverters, so scoured my subdivision for owners of boats and RVs. Great results!

  4. Michael O'Neill
    Michael O'Neill says:

    Really like the idea of this. Reminds me of how I used to use Mechanical Turk to test the clarity of marketing messages, copy in drip campaigns, etc…. Was a toxic environment, so internal feedback was highly politcal and often the blatant expression of ulterior motives and hippo posturing. MTurk testing helped me get a perspective that wasn’t biased by petty political agendas. Of course, it wasn’t perfect either. But, like cafe testing, it is a useful tool to have in your belt.

    -Michael

  5. Daniel eizans
    Daniel eizans says:

    @Rahel – Agree with you. I just wish we could read minds and figure out what might influence a user during a test! The inverter testing sounds interesting. We will have to chat about that sometime.

    @Michael – I find that depending on a sites purpose (some stuff I’ve done for the EPA comes to mind) you’ll find bias in the testing, no matter what you use and I think that’s ok. The real trick, as you’ve eluded to, is to keep in mind that each test is one part of a bigger data need.

    Thanks to both of you for the feedback.

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *