A Perspective on AI-Based Text Analytics

Published on Feb 05, 2024 by Lee Streu

Technology has created a complete paradigm-shift in how we process and analyze open-end responses today.

My first job in marketing research started in the last century. So, I have perspective on the dramatic evolution of open-end processing and analysis within the marketing research industry.

The old way of treating open-end feedback:

For background, my first job was with a marketing research supplier in New York City, and the training regimen for first-time research analysts was to learn from the ground up. I spent the first three weeks working in the open-end coding department, a team of approximately a dozen part-time students and retirees.

The open-end data that we used was sourced from mail surveys, mall intercept surveys, and telephone surveys. Online based surveys were not yet mainstream, and smartphones did not exist yet. As one can imagine, the open-end quality was not great. With mail and mall intercept surveys, there was little motivation to handwrite much of what the consumer shared, and it was difficult to read and interpret snippets of partially cohesive thoughts. With telephone surveys, the telephone interviewers wrote as little as possible because they were incented to get to the end of the survey and not on the quality of open-end feedback.

The coding process was highly subjective and prone to incomplete code frames and reading comprehension miscues. The open-end coding process started with taking a random sample of open-end verbatims (e.g., 50 of the 200 completed surveys) and creating a code frame. This code frame then became the answer key by which we read and coded all open-end responses. As we read an open-end, we simultaneously interpreted the underlying intent of the words and matched that up with a code in the code frame. It was a highly subjective process that was made worse by poor quality open-end inputs.
The codes were merged into the dataset, but there was limited utility with what we can do with the coded open-end responses. A data processor would tabulate (i.e., cross-tab) the coded open-end data against segments of interest (e.g., total, by age, by gender). That was the bulk of what we did with open-end data, unless we wanted to include open-end verbatims in the analysis for more context.

The modern approach to open-end feedback:

Technology has created a complete paradigm-shift in how we process and analyze open-end responses today.

Nearly all of Socratic’s surveys today are collected using online and mobile surveys. There are several tools that we regularly use to encourage better quality with open-end feedback. We have a ‘quality meter’ underneath the open-end input box that provides real-time feedback to the survey respondent on the quality of their feedback. We also have an option for survey respondents to record their feedback on video, and this helps with more conversational flow and top-of-mind thoughts when providing open-end feedback. Lastly, we are developing capabilities to use AI chatbots to probe survey respondents in gaining richer feedback in open-end responses. Individually and in combination, these technological advances produce better quality feedback and a greater volume of open-end feedback (versus the paper and telephone open-end feedback from years past).

The very recent advances in AI are rapidly improving and expanding the level of insights we can generate from open-end responses. AI benefits include:

  • Efficiency and Automation: AI automates the process of extracting valuable insights from large volumes of text data. This significantly reduces the time and effort that is required compared to manual coding and analysis.
  • Less open to interpretation: Natural Language Processing (NLP) algorithms, a subset of AI, enable machines to understand and interpret human language with an exceptional understanding of context and nuance. The machines consistently apply this understanding across large datasets without the influence of human interpretation.
  • Data Summarization: AI models can generate concise summaries of large volumes of text, making it easier for users to quickly grasp the key points without having to read through extensive open-end responses.
  • Topic Modeling / Finding the white space: AI algorithms can identify topics and themes within a body of text that deviate from the central topics. Socratic overlays visual mapping and correlation matrices on top of open-end datasets to see how white space topics interact with other open-end responses (e.g., whether the white space topics are completely unique or related to other themes in the open-end responses). This process makes it easier to extract meaningful insights.
  • Sentiment Analysis: AI-driven sentiment analysis can accurately determine the sentiment expressed in text, helping businesses understand customer opinions, preferences, and reactions. This information is highly valuable for brand management and customer relations.
  • Advanced Modeling with Closed-End Data: Socratic combines open-end and closed-end responses for sophisticated driver analysis and segmentation that is informed by machine learning algorithms within the modeling.

These tools can be applied across several research methodologies, including:

  • Concept Testing to get levels deeper on understanding why an idea is accepted or rejected. Open-end analysis can also help to further develop early-stage ideas that do not have well-defined use cases and value propositions.
  • Front line Insights to learn from internal employees/associates who are the closest to the customer.
  • Market Strategy as a broad-based way to cast a net over all the topics that are relevant to the target market.
  • Satisfaction Tracking to understand the issues beyond what is being tracked and monitored.
  • Brand Tracking as an early monitor system on new positioning opportunities and competitor activities.

AI will continue having an increasingly disruptive impact (in a good way).

For the reasons described above, AI dramatically increases the utility of open-end feedback and helps companies make informed decisions with better insights depth and quality.

Lee Streu is an Executive Vice President at Socratic Technologies. Lee has 30 years of experience in supply-side and corporate insights roles. Lee has supported decision-making and research efforts across several industries including financial services, consumer products, technology, healthcare, restaurant & hospitality, and utilities/communications. At Socratic, Lee is involved in helping to scope & design all research engagements and in providing senior oversight with each of Socratic’s client partners.
You can contact Lee at [email protected]

1 2 3 4