‍

‍

There is a relatively simple formula which describes “weak” or “narrow” artificial intelligence: AI = ML+TD+HITL. To be more specific, this is the definition of supervised machine learning, which is the most common method to produce artificial intelligence. The acronyms in the formula stand for:

• AI = artificial intelligence
• ML = machine learning
• TD = training data
• HITL = human in the loop

‍Strong artificial intelligence - as defined by the Turing test - is when a human has a conversation with a machine and cannot tell it was not a human, based on the way it responds to questions. The optimists believe that strong AI is 10-15 years away whilst the realists/pessimists say not before the end of this century.    

How can “weak” AI be applied in research and insights:

Over 90% of all human knowledge accumulated since the beginning of time, is unstructured data. That is text, images, audio, or video. The other 10% are numbers in tables which is what quantitative market researchers usually use. The qualies, they are the ones using unstructured data, but the volume is limited to a few pages or a few video clips that a person can read/watch in a couple of days.  

Other than reading, listening to, or viewing unstructured data, 15 years ago there was no other way to discover their content and understand their meaning. Thankfully (especially if we are dealing with big data) today there is a way to discover and understand the information hidden in mega-, giga-, tera- or n-ta-bytes of data; you guessed it, it is AI. Machine learning allows us to create models that can process large files of text or images in seconds, and annotate sentences, paragraphs, sections, objects, or even whole documents with topics, sentiment and specific emotions. Sentiment and semantic analysis are the two most popular ways to analyse and understand unstructured data with the use of machine learning or a rules based approach. When the unstructured data to be analysed is in text format, the discipline falls under Computer Science (not linguistics funnily enough) and is called Natural Language Processing (NLP) or Text Analytics. 

Semi-supervised-, unsupervised- and deep-learning are other forms of machine learning, used to a smaller extent in a market research context, even though deep learning implementation is picking up speed - especially for image analytics. 

Use cases for unstructured data analytics:

There is a multitude of users, data sources and use cases within an organisation. Let’s take a look at relevant data sources first:

1. Social Media; this is the most popular source and the related discipline is social intelligence
2. Other public websites
3. Answers to open ended questions
4. Transcripts of in-depth interviews and focus group discussions
5. Call centre conversations with customers
6. Organic conversations on private online communities

ESOMAR mainly caters to the market researchers in organisations globally, but there are many more users of text and image analytics solutions sitting in different departments, that can benefit from using AI to understand unstructured data. Here is a combined list of users and use case examples for each one, which is not exhaustive by any means:

1. Market research - for insights from social and other unstructured data sources
2. Public relations - to manage brand and corporate reputation
3. Customer service - to respond to questions, complaints and requests
4. Advertising - to leverage positive testimonials
5. Marketing - to find and leverage influencers
6. Product Development - to learn about missing product features or ones that are not appreciated by consumers
7. Innovation (beyond new product development) - to learn about emerging trends and new product use cases
8. Competitive Intelligence - to gauge how competitors are doing in an industry or product category
9. Operations - to learn about issues that need fixing
10. Finance (together with marketing) - to find out about sentiment towards pricing
11. Board - to benchmark and track sentiment on governance
12. Sales - to find sales leads who express purchase intent

Questions that social intelligence can answer perfectly:

If we agree that social intelligence is currently the most popular application of AI in research and insights then it does make sense to review possible questions that can be answered using it. 

1. How successful was my advertising campaign on social media? What do people say about it?
2. How can I improve my advertising campaign on social media while it is live?
3. How does my brand performance on social media benchmark against competitors in terms of engagement and sentiment?
   
   • Likes, shares, video views, followers
   • Positive/negative sentiment
   • Net sentiment score
   • Engagement ratio
   • Top 3 and bottom 3 posts in terms of engagement
4. Which types of marketing campaigns work on social and which do not?
5. What are my customers happy and unhappy about?
6. What are my competitors’ customers happy and unhappy about?
7. Which are the conversation drivers online?
8. What digital content should I be sharing to help engage and help my brand’s performance
9. What product or service features are customers not happy with?
10. What product or service features would they like to have that are not available?
11. What operational issues do people complain about when it comes to my brand versus competitors?
12. What is my brand’s share of voice overall and for individual sentiments?
13. What specific emotions do consumers express for my brand and its competitors?
14. What is my share of posted images?
15. What are the stories around the consumer posts that include images?
16. What are the sentiment trends by brand?
17. What are the topic conversation trends by brand? What are the popular sub-topics of these topics?
18. When do conversations peak? What are the subjects, sources, times that usually online posts about a topic peak?
19. Which online sources are the most important for my brand and for the category?
20. Is there any white space when it comes to post volume on topics and sentiment by brand?
21. In which languages is my brand most mentioned compared to competitors?

Social Intelligence and traditional market research methods:

If you are amenable to a bold statement such as “social intelligence may replace some traditional market research methods used to solicit consumer opinions” then here is a list to consider:

1. Usage & Attitude studies (U&A)
2. Exploratory FGDs to discover how consumers talk about a subject
3. FGDs to find out which brand image attributes to measure
4. Advertising Campaign Tracking
5. Brand Equity Tracking
6. Volumetric forecasting
7. New product development research
8. Customer Satisfaction Surveys, customer experience measurement
9. Qualitative - Landscape Framework

Of course whether social intelligence can replace them altogether or enhance them depends on the country, language and product category.  If you have not embraced the use of AI yet, to tap into the wealth of unstructured data available to us everywhere, then at least keep an open mind and keep asking questions that will help you make an informed decision when the right time comes.