Market Research: Why Detecting Talkative Respondents Doesn’t Work

We recently published a special report titled 25 Common Field Mistakes to Avoid When Conducting Your Qualitative Market Research. In #12, we suggest that researchers remove open-ended screening questions that purport to identify respondents who are outgoing and self-explanatory.

Here is the full text of point #12:

“If your tester contains open-ended questions aimed at getting expressive types of people, delete those questions. Questions like that don’t work and needlessly lengthen your evaluation process. Yes, you want to exclude respondents who can’t or won’t express themselves, but you don’t need to.” an additional question to identify these people. Well-trained recruiters will weed them out in the first few minutes of screening. If you want more reassurance that your respondents will be outgoing and communicative, go overboard with recruiting and include a pre-discussion phone interview. This would be done by the moderator, who would then select the appropriate respondents.”

Our suggestions elicited many comments. Some readers agreed with us, others did not. One of the most interesting feedback we received came from a qualitative fieldwork manager at a large, full-service research firm. It read like this: “I disagree with one item on your list about excluding open-ended questions from raters. Good recruiters can easily spot articulate respondents, but tired or distracted recruiters can occasionally go on autopilot.” … I think a bit of quality in a filter is worth the time.”

Do you agree with this reader’s comment? You should?

What is an articulation question?

Articulation questions measure the respondent’s ability to communicate. Articulation questions also judge respondents’ answers. expected Communicativeness in a focus group or interview.

Some synonyms for “communicative” include: outgoing, open, forthcoming, talkative, unrestrained, talkative. So who decides what is communicative? The recruiter? The recruiting supervisor? The customer who reads word for word in your daily reports? And how much communicative is enough? How much is too much?

Even the most seasoned recruiters cannot determine how outgoing, open, approachable, talkative, wild, or talkative a respondent will be at some point in the future. That is a decision that recruiters are not qualified to make. But they can be counted on to spot respondents who have…

• language barriers
• casual attitudes toward the recruiter, the recruiter’s questions, or research
• reservations about their ability to attend the investigation
• any communication problems during the selection process

What you should watch…

Respondents get tired or go on autopilot when recruitment interviews take too long (10 minutes or more).

Articulation questions do not belong at the end of your assessment. For whatever reason, the joint assessment is almost always done at the end of the assessment interview. But why is a question that is supposed to be so important placed at the end of the assessment, when the chances of respondents being tired or distracted are the highest? What are recruiters learning about respondents at this point in the process that they don’t already know?

Articulation questions don’t belong in front of your evaluator either. Well-trained recruiters immediately engage respondents in a conversation about the details of the research. It is during this prelude to screening questions that recruiters address respondents’ questions and concerns and make an assessment of respondents’ communication skills.

Joint problems are not magic bullets that ensure good participants in the focus groups. These questions simply ask recruiters to use their own biased judgment to decide if a respondent can communicate clearly.

Articulation questions lengthen your assessment. Remember this. The longer your filter, the higher your costs.

Respondents become anxious when asked questions outside of left field that are not related to the assessment questions. When asked, “What is a gazinkle?” or “How many different things can you do with a paper clip?” or “If you were a tree…?” could stump even the most eloquent respondent. Crazy questions from confused recruiters and frustrated respondents. This line of questioning is the territory of the moderator.

Of course, the dynamics of the group and the personalities of the respondents affect how open and receptive the respondents will be. For example, a person may communicate on the phone with the recruiter, but feel intimidated if an aggressive personality dominates the group. Or, a respondent may not be as comfortable with the research topic as she thought and feel out of place, especially if the topic offered during recruitment was vague. How can recruiters know how respondents will perform under a variety of conditions? Handling reserved respondents is the moderator’s area of ​​expertise.

In fact, moderators are better qualified to know what can and should be expected from respondents in terms of communicativeness and articulation. So it makes sense that, as we suggested in point 12 of our special report, moderators should pre-interview respondents and select the right personalities for research.

So what about articulation questions that are useful for snapping distracted or unaware recruiters out of their daze (as our reader suggested)? Assuming that a tired and distracted recruiter missed all the red flags during selection, will the articulation question suddenly remind the recruiter that the respondent is not talkative? What should you do with recruiters on “autopilot”? Simple.

The researcher’s job is not to craft questions that keep recruiters alert and focused. Tired or distracted recruiters are not an asset to your research. They don’t help you get great respondents. And neither are articulation problems. Don’t use either one.