Artificial Intelligence (AI) is now a critical part of many business functions. While this takes different forms such as machine learning and powerful algorithms for analysis, the use of AI to optimize survey design provides the ability to increase the value of a respondent's feedback. Recently, Socratic and Reveal AI partnered to model a survey where a chatbot was used to optimize the impact respondent feedback. Here is how the AI chatbots enhanced the open-ended survey responses and why it is quickly becoming indispensable in the modern research toolkit.
The process of obtaining and analyzing open-ended feedback has always been a laborious and time-consuming endeavor. This is partly due to the random and varying level of information given by the respondent. Additionally, the ability to capture information as a text box did not provide a means to qualify or gauge respondent feedback.
Recently the use of quality meters that provide respondents with a scale to help gauge their response slightly improved the quality of their feedback. However, such quality meters do not encourage or prompt the respondent to expand or clarify in great detail the information provided.
In contrast, AI chatbot tools utilizing NLP (Natural language Processing) are dramatically increasing the power of open-end feedback by providing real-time prompts that naturally direct respondents to deliver rich, high-quality input leading to unprecedented levels of insights. This information then feeds the analytics processes to create more business centric information that business leaders can act upon.
Socratic utilized an Omnibus survey which segmented 600 respondents into 4 groups. Each group was asked four open ended questions and the feedback mechanism varied by the following:
Across the four options, those using the AI chatbots produced more, and higher quality feedback than traditional open-end questions. The chatbot with two probes produced 2.5x more feedback, and the one with four probes produced 5x more feedback. While the amount of information provided is important, the other factor is the quality of the information that the recipients provided. Without thoughtful responses the feedback could limit the usefulness to the research.
Socratic used the Flesch Kincaid Grade Level formula to review the feedback to measure the reading grade level of the feedback of each segment. Those that were exposed to the AI chatbot provided responses that were a full grade level higher compared to those who used the traditional text box feature. This indicates that given the opportunity to expand or add to their feedback via AI probing questions, respondents not only gave more feedback, but also supportive details and information related to their responses thus improving the quality of the input to the analysis.
An obvious question that arises is how does a chatbot impact the survey and the respondent's engagement with the research. Can the use of the chatbot deter the participants from responding to the questions? Does the extra probing on a question create time issues and drop off of the participants engagement?
As with any research and survey design, the need to understand when and how to use the chatbot within a research study so that it improves the participants experience while not requiring a significant amount of time. When Socratic interviewed the participants regarding their use of the chatbot in the survey, 61% said it was easier to share their opinions this way and 62% felt they were able to share everything they wanted with the chatbot. These results indicate that with proper design, the AI chatbots as a function of marketing research can create better insights for the client while not impacting the overall research effort.
The integration of AI chatbots in market research represents an important shift. By improving engagement and enhancing the richness of open-ended responses, Socratic’s adoption of AI chatbots along with advanced data sciences to better understand the drivers of choice helps the client receive business centric insights for strategic decision-making.
While many of the uses for AI are for efficiency; it is also a tool for discovery. And as uses like AI chatbots become a viable option in research, the potential to revolutionize market research is only beginning.
If you would like a demonstration of how AI chatbots can improve your insights, contact Socratic to set up a time where we can provide more information about the design, implementation, use cases and benefits of having an AI chatbot as part of your research.
Lee Streu is an Executive Vice President at Socratic Technologies. Lee has 30 years of experience in supply-side and corporate insights roles. Lee has supported decision-making and research efforts across several industries including financial services, consumer products, technology, healthcare, restaurant & hospitality, and utilities/communications. At Socratic, Lee is involved in helping to scope & design all research engagements and in providing senior oversight with each of Socratic’s client partners.
You can contact Lee at [email protected]
Andrea Baird is a Text Analytics Director at Socratic Technologies. Andrea has over 20 years experience doing custom market research. Andrea has supported business leaders across several industries among fortune 500 companies, non-profits and other niche companies. Andrea is skilled at jumping into projects at any point along the timeline from scope to final presentation, and leads text analysis projects for Socratic Technologies.
You can contact Andrea at [email protected]
