16 Aug Why limit research to just one moderator?
I hear this a lot from clients: “It’s important that we use the same moderator throughout the research for consistency’s sake.”
But is it really? Of course, there is a lot to be said for consistency across interviews. After all, where would we be if we asked different questions of every respondent? If, say, in a creative concept test, we asked some participants to rank concepts according to their appeal and asked others to rank them according to credibility? Obviously, no good would come of this.
But where so much of qualitative research is about inspiring ideas and generating insights, I would argue that using just one moderator can do more to inhibit your research than help it. At Shapiro+Raj / CarbonSix, we have been using multiple moderators with great success and to the delight of our most forward-thinking clients. Here’s why:
- Moderator bias. Many of us think that the consistency achieved by using a single moderator on a research project means that there won’t be any bias. But using one moderator doesn’t eliminate bias; it simply masks it. No person is without bias, and that includes your market research moderator. The best we can do is minimize the impact of any one moderator’s bias by using more than one for every project. The key is to do it right, by spreading interviewers across respondent types and segments to avoid confounding the results.
- Consistency isn’t everything. Why not capitalize on the different styles moderators bring to capture more insights? Yes, two different moderators might generate different insights and different ideas. But that doesn’t mean they will lead to different conclusions. For a strategic research team, more is better – more insights means richer, more nuanced recommendations.
- Moderator fatigue stifles insights. Some of our qualitative research projects can include more than 50 or 60 individual interviews. Moderators get fatigued after asking the same questions and hearing similar responses over and over again. Even the most experienced and talented moderator will eventually stop or minimize probing when they think they have heard it all. They assume they understand what a respondent means and so they stop asking. They can’t help it. It’s human nature and I am guilty of this myself. (Incidentally, this is also why interviews sometime start out taking at least an hour at the outset of fieldwork, but easily wrap up in 45 minutes by the end of fieldwork.) When we stop probing, we stop learning. A fresh moderator is far less likely than a fatigued moderator to miss an important nugget of insight.
If you’re still nervous, simply ask moderators to watch a few of each other’s interviews. Or ask both (all) moderators to attend the first day of interviewing, so that everyone can participate in the important knowledge sharing that happens on day one of fieldwork.
Done properly, using multiple moderators can improve the quality of your research and inform better business decisions. Don’t fear moderator differences. Embrace them.
by Kim Mizrahi, President, Shapiro+Raj/Carbon Six New York