We may not like to admit it but making mistakes is an expected part of the scientific process. As market and social researchers, we regularly discover errors, learn from them, and improve our processes because of them. You might even say that errors are essential for innovating and generating better insights. From the initial stages of data collection to later stages of results presentation, embracing mistakes is a pathway to better research.

‍

Champion Methodological Mistakes

Plenty of errors are likely to occur during the research design and data collection phase. They reveal unexpected gaps that can be anticipated and prevented in future projects with better training, tools, and processes. How can we embrace these errors?

• Expect flawed data collection tools. Whether you’re working with discussion guides, questionnaires, diaries, or bulletin board outlines, errors such as leading questions, unclear scales, and missing response options are bound to appear. Plan for these problems by including time in the schedule for colleagues to review your tools, perhaps adding incentives for creative solutions. Further, be sure to pilot test the final tool with a few participants prior to the full launch.
• Train for neutrality. As hard as we try to avoid it, questionnaire authors, interviewers, and moderators have feelings and opinions that show up as biased questions, micro-expressions/body language, and tone of voice. This unintentional lack of neutrality can decrease participation rates and distort results. To address this problem, attend refresher training courses on a regular basis. You’ll not only be reminded of common mistakes you may have forgotten about but you’ll also learn new techniques that have gained prominence since your last training.
• Plan for sampling gaps. Every researcher knows how easy it is to recruit a large group of research participants. At the same time, every researcher also knows how difficult it is to recruit participants who are representative of the target population. When budget and field dates are pressuring you, avoid the temptation to overlook representativeness. Build extra time into the schedule and actively oversample difficult target audiences from the beginning to avoid the biases that will result from settling on non-representative samples.

‍

Embrace Analytical Errors

Once data has been collected, the potential for errors doesn’t stop. By embracing errors during the analytical phase, we can achieve more thorough and nuanced data analysis.

• Seek outliers. Sometimes, outliers are discovered to be mistakes like miscodes or shifted columns. These are easily corrected. However, other outliers are analytical puzzles that need to be deciphered. It’s easy to automatically dismiss statistical anomalies but first consider whether they signal an unexpected insight. Mark the anomaly so that once you have a better understanding of the entire research problem, you can go back to it and determine if it was more meaningful than first realized.
• Contradict yourself. As you’re reviewing the analysis and building theories, actively contradict your own ideas. Try to prove the alternative hypothesis. Ask yourself what is missing. Assume errors have been made. Consider if you’ve overgeneralized beyond what the data is telling you. This will help you to avoid the confirmation bias that can arise out of early discoveries.
• Encourage dissent. Throughout your analysis, invite other team members to independently interpret your data. Incentivize them to contradict your interpretations and conclusions with plausible alternatives. And be prepared to put your ideas aside when other ideas are more likely.
• Leverage technology. Rushing through every potentially important theoretical model or statistical analysis is a fast-track to making errors. Use AI tools to dramatically improve your productivity and accuracy. Read our case studies to learn how C+R Research and Frost & Sullivan use Ascribe and Voxco Online to build complex questionnaires and code qualitative data faster, more accurately, and at a lower cost.

‍

Capitalize on Communication Glitches

In many cases, stakeholders review research reports without guidance from the researcher. Consequently, it’s essential that reports are clear, engaging, and powerful.

• Test reports with stakeholders. After spending days and weeks writing a report, your headlines and summaries will seem clear and concise – to you. Schedule sufficient time into your plan so that non-researchers can review it for clarity. Invite them to interpret and mis-interpret charts and tables, and point out conclusions that don’t make sense. Incentives are a great idea here too as many people don’t feel comfortable sharing criticisms of their colleagues’ work.
• Use plain language. As much as you love talking about various statistical tests and study designs with your colleagues, research jargon is not plain language and will not help readers who are further down the chain. Yes, share the jargon as learning opportunities but also incorporate sufficient descriptions so that people who don’t know your jargon will still understand what you’re talking about.
• Highlight limitations. Limitations are not flaws or mistakes. They are acknowledgements that research can never uncover every possible insight regarding every possible scenario. Since no one knows the limitations of the research better than you do, share details about any study constraints during verbal presentations and in the report. Preventing misunderstandings is a key part of your role and your clients will appreciate the guardrails.

‍

Convert Errors into Enhancements

Even the best researchers can’t avoid mistakes. What they do, however, is implement processes to anticipate, detect, and learn from those mistakes. Leverage AI and automation to reduce errors arising out of tedious activities. Collaborate with colleagues and other teams to invite dissenting opinions and alternative insights. Formalize debriefing sessions to identify problems and implement processes to prevent them in future projects.

Remember, mistakes aren’t failures. They’re feedback. By normalizing the presence of errors and fostering a culture of feedback, researchers can improve research validity and methodologies, build trust with stakeholders, and produce more impactful results. Embrace imperfection and the continuous improvement that goes along with it!

If you’d like to work with an experienced team of researchers who also value continual improvement, please get in touch with one of our research experts.

‍