Moving On From DIY Surveys

Published on Mar 19, 2024 by Daniel Chiat

How To Know When To Elevate Your Marketing Research And Enlist Help

Throughout my career, whether working with a small non-profit or one of the largest financial institutions in the world, I’ve seen many examples of effective organizational leaders prioritizing risk management strategies to ensure stability and growth. And I’ve also seen what happens when leaders do not implement these strategies – it can get ugly, fast.

For this article, I’d like to explore the difference between self-service (or more colloquially, DIY) surveys and surveys that are managed by a third party with expertise – and more specifically, how risk management sits at the nexus of these two research approaches.

First, it should be mentioned that the convenience and reasonable cost of self-service survey platforms such as Survey Monkey, Alchemer, and Qualtrics make the DIY survey approach somewhat attractive. Plus, you don’t need to draw up a contract with a research supply company before diving into the project! In all seriousness, there may be instances in your day-to-day responsibilities when using one of these platforms is totally warranted and appropriate.

But for our purposes, assume you’re faced with a key business decision that deserves thorough research from your customer base to inform action. Let’s run through the key stages of such a research project and note where risk can be mitigated by ditching the DIY path:

  1. Designing Research Approach: Before questions are written, responses collected, and data analyzed, establishing the right approach is paramount to project success. What is the proper balance of qualitative and quantitative data necessary? What quantitative methodologies, such as Max Diff or Discrete Choice, would best fit the business objectives at hand? Without leveraging research partners in the research design, the greater the risk of these key business objectives remaining unanswered by project’s end.   
  2. Survey Development: Good research is built on good, actionable questions that address key business questions and hypotheses. Writing questions in house, without a more objective and experienced third-party to help craft and review, increases the chance for survey data that is garbage in, garbage out.
  3. Survey Testing: Once a survey is written, it must be programmed into an online format – and tested rigorously so that all end users can easily complete the survey (as-intended). The more complex a survey becomes with skip logic and different question types, the more likely a coding mistake happens. And without multiple rounds of testing all survey permutations, the more likely the original programming mistake lives on through the data collection phase and disrupts the eventual response dataset.
  4. Data Collection: When sending out a survey to real-life customers, what could go wrong while collecting data? Unfortunately, plenty. A big risk is obtaining a biased sample, which might mean for instance that survey respondents in the aggregate are too heavily comprised of older individuals when the goal of the research was to primarily gather feedback from millennials. An experienced third-party closely monitors the sample coming in and has guardrails in place to ensure the final dataset is demographically and/or behaviorally representative compared to the research objectives.
  5. Reporting: This is the stage where all the hard work begins to translate into decision-making and actionability. But even if a comprehensive presentation with exquisite charts is prepared, the data must be interpreted accurately before making any decisions. For example, if a new product is found to be appealing among 75% of survey respondents, business leaders must have the proper context to best understand how good that score is compared to similar products in the marketplace, among various demographic groups, etc.

Beyond these individual stages, the umbrella factor of project timing and resources is highly relevant to this discussion. Because even if a DIY survey approach manages to avoid all these common risks, it must do so within timing and resource constraints – something that an effective third party research supplier will execute within the project scope.

So the next time you are thinking about tackling a key research project in-house, please keep these risks in mind – and if any resonate with you, I’d love to speak with you about how Socratic might be able to help you mitigate them.

Daniel Chiat is a Senior Research Director at Socratic Technologies. He has nearly 15 years of market research experience with a focus in leading highly customized quantitative studies, supporting large global financial services firms, small non-profit organizations, and many other types of clients in between. Daniel is passionate about obtaining actionable data for his clients that lead to informed decision-making.

1 2 3 11