• Issue #34

When Focus Groups Go Wrong

Group of folks talking during a focus group session.

What you need to know to make focus groups work within your digital strategy research

We borrow heavily from other disciplines when we select research methods for marketing and digital strategy. But, unlike many other research methods, the focus group was specifically designed for testing messaging and marketing strategy. Did you know that researchers started using the focus group method to draw out opinions on military propaganda in the 1940s? The U.S. Government wanted to test which messages were most effective in increasing support for our participation in World War II.

Focus groups are a popular research method for their efficiency. It’s usually pretty easy to gather a small group of people and ask them a series of questions. It’s quicker than conducting 6-10 one-on-one interviews. Plus, one-on-one interviews can feel intimidating to the interviewee and interviewer alike. Group conversations don’t rely exclusively on the chemistry between the interviewer and a single participant to produce meaningful data.

But, it’s not convenience that makes focus groups a useful research method. In qualitative research, participants often want to be as helpful as possible. They want to provide the data they think the researcher wants to gather. They also can’t help themselves. They want to look thoughtful and well-reasoned in their opinions. They’re sharing their identity with you - a little about what makes them tick.

However, the push and pull of group dynamics can often cause participants to let go of their expectations for their own performance; they get caught up in the conversation and reacting to one another. As a researcher, you can glean more nuanced and contextualized data from these social responses. The qualitative data surfaced in group discussions can provide rich and novel insights that one-on-one interviews fail to capture.

However, focus groups are harder to facilitate than you might think.

Facilitation is about more than just asking questions and letting the magic happen. As a facilitator, you have to balance your own formality with the rapport you are trying to generate between you and the participants. You have to understand how to start a conversation, and then be able to effectively deploy strategies to nurture them between others. Then you have to gracefully move group conversations along across a number of different topics.

We’ve all seen examples of focus group research that don’t produce particularly helpful data. So, what goes wrong, and how can we fix it?

1. You are relying on the facilitator to simultaneously collect data.

Like many things, facilitation is part art and part science. There are interesting conversation patterns and techniques for managing group dynamics (for example: emerging, deepening, and closing; or, move, follow, oppose, bystand). But, there is also a lot of demonstrative listening, modeling empathy and interest, and figuring out how to encourage some people to talk more while managing the dominant opinionators in the room.

When I am done leading focus groups, I often feel thoroughly exhausted. Drawing out that social data requires steady concentration and perceptive, improvisational moves. That’s why I recommend recording focus groups and inviting notetakers. Take some of the pressure off the facilitator (please, God.).

2. You’re not spending time creating a rapport among participants.

I hate icebreakers. It’s as awkward to ask those kinds of questions as it is answering them. I am terrible at coming up with something interesting about me on the spot, and I hate subjecting others to it. Icebreakers are meant to establish group trust and camaraderie among participants. We can do better than saying, “What’s something you’re looking forward to this month?”

First, budget significant time to break the ice so you don’t feel rushed. That way, if the first question or two don’t strike a chord with folks, you have the time to ask others. Second, vary the types of information you seek. Not everyone will feel the same way about saying one interesting thing about themselves. Others might have an easier time recommending a favorite television show or movie. Still, others might prefer to ask their own questions or play a short round of Pictionary.

Also, consider your transitions between icebreakers and your data-gathering questions. Give your participants a way to ease into the main content. Ask them questions that give them a chance to shine and brag a little.

3. You’re leading the witnesses.

One of the most complex parts of facilitating focus groups is providing acknowledgment and evidence of deep listening without agreeing with opinions. Part of the way we build rapport with one another is by finding things in common. So, it’s easy to validate someone’s feelings and experiences with expressions of solidarity. That’s part of the reason that icebreakers are so essential. They allow you to build that rapport before the meaty questions, so your data collection can be a little impersonal.

I address my eagerness to agree when facilitating in a few different ways. For one, I ask others individually how their experiences are similar or different. This relieves the pressure I feel to confirm the impression. Another thing I do is continuously thank my participants for sharing their perspectives. This communicates the listening and interest without having agreed to their position.

4. You’re using projective techniques wrong.

Projective techniques are used when facilitators have an intuition that participants are not stating their true feelings on an issue. This could be for various reasons; they might not feel entirely safe with other focus group members, they might feel uncomfortable going against the opinion of other members of the group, or they might even worry they’ll hurt feelings. In any research project, it can be challenging to get some folks to give you negative feedback that helps you uncover issues.

Projectives, developed in the field of psychology, aim to get below the surface. The most famous is probably the Rorschach test, but others will sound much more familiar to you. Below are some popular ones:

  • Finish this sentence: “I use Brand X because…”
  • Word association: “I say a word, and you say the first thing that comes to mind.”
  • Asking users to describe how a neighbor, friend, or family member might feel about a given product or service.

Often, inexperienced facilitators build in projectives as sequenced questions in their scripts. But, particularly in focus groups, projectives can disrupt the conversation between participants and change participants’ perceptions about what’s expected of them. It can be hard to recover a conversation once you switch to asking each participant individually to answer a projective question. The spirit of conversation will wither, and participants will treat the activity like a group interview.

In order to use projectives properly, it goes without saying that you need a rapport with participants. Additionally, you need to draw out their conscious opinions and preferences before you can effectively draw out unconscious opinions and preferences. Projectives can produce fun and exciting data, but they should be used only when a facilitator detects hesitancy to be forthcoming with honest answers.

5. You are collecting both usability data and branding data at the same time.

Focus group data can answer different kinds of questions. And, sometimes, when we’re planning our focus groups, it’s easy to brainstorm the questions before we target the data we want to collect.

For usability research, a focus group often aims to uncover thoughts and feelings related to a user journey. The questions may revolve around:

  • the context of using a product or service,
  • what our participants remember from using a product or service (e.g., points of success or delight and points of frustration or confusion), and
  • how the product or service fits into the whole experience (e.g., how a website process supports the overall customer experience).

For market research, a focus group is used to surface ideas that can test our assumptions and create a broad swath of new data to develop further quantitative research projects. Data collected in a focus group can support persona development, brand associations and equity testing, and competitor analyses.

When we muddle these purposes together, though, we wind up with partial data that doesn’t wholly answer either usability or marketing questions. And our data can be challenging to parse. User experience questions draw out parts of the journey that are confusing or frustrating, along with parts that are easy to understand and simple to complete. Those aren’t usually brand attributes, though. Asking participants to switch gears and talk about the brand’s identity might produce information based on that confusing/frustrating to easy/straightforward continuum.

6. Your focus groups are your single source of data.

When done well, focus groups produce rich and exciting data. They can also produce some wildly unexpected results. I’ve heard participants say near nonsensical things in a group setting or get flustered and say things that are irrelevant or unlikely to be true.

It’s why it’s so essential to triangulate the data to get in focus groups. Relying on your focus groups as your single source of data opens up tremendous vulnerabilities in your decision-making and strategy.

Focus groups provide insights that can be tested with further quantitative or qualitative analysis. But, they can never provide a complete story of one participant’s journey, they cannot reveal actual behavioral data, and they cannot statistically validate experiences. If your question is “what other questions should we be asking,” focus groups are where it’s at! But otherwise, focus groups rarely provide enough information for true data-driven decision-making. They are just as likely to lead you astray as they are to give you juicy, informative truths.

Keep on focus grouping on.

I am not here to dissuade you from running focus groups. Focus groups are often the workhorses of our research plans, and for good reason. To do them well, they take significant work and an understanding of how they could fit into a broader research portfolio. Just be mindful of their limitations and use them to collect the data that answers your research questions.

And should you ever need assistance with research planning or facilitation, don’t hesitate to reach out.

- Kristin Van Dorn