Unpacking Unknown Unknowns

Lucky you. The fact that you are reading this article right now suggests that you’re interested or to some extent involved in social insights at an incredibly exciting time. The recent evolution of machine intelligence in the Natural Language Understanding domain brought us to new heights in our ability to decode the true meaning behind human language at a gigantic data scale.

This is big - it enabled the Social Intelligence practice to become a mature research discipline with a truly unique insight value that is hard to live without; especially for insight professionals operating in the consumer trend, front-end innovation, brand, and market understanding domains. But what makes social data so precious for these research use cases? This is usually the moment when we hear the two magic words “Unknown Unknowns”. Let’s get to the bottom of what’s behind the mysterious catchphrase and what it takes to actually discover them.

What exactly is an Unknown Unknown...

The term Unknown Unknowns is attributed to Donald Rumsfeld who used it in a Defence Department briefing in 2002, in the context of risk management and national security threats beyond our awareness and understanding. A more universal example of Unknown Unknowns is a rainforest’s complex ecosystem. We’ve certainly accumulated an expansive knowledge about the Amazon rainforest, yet we still don’t know enough to formulate the right questions or identify our current knowledge gaps. In social intelligence terms, the expression usually refers to the notion of the human truth behind big data – the phenomena ‘we don’t know we don’t know’. There could be untapped, unmet consumer needs or pain points we haven’t yet discovered or an emerging opportunity space in its infancy that might be out there in the wild.

Why it matters

Discovering the unknown unknown blind spots from 'people data' is really the pinnacle of social intelligence work. It’s what makes social intelligence so advanced as, unlike many other research methods, it gives us a boundless experimental landscape to find the unexpected and honour the complexity of openly shared consumer experiences within the subject we’re delving into.

Let’s look at an example...

In some recent work for a multinational food corporation, we explored the role of foods within the context of wholesome wellbeing. We developed a comprehensive view on what wholesome wellbeing means for consumers and how they define it.

We analysed over 6 million consumer signals to explore which behaviours and attitudes they adopt within their holistic physical and mental wellbeing regime and which diet’s, foods and specific ingredients are an integrated part of it. Then we applied an econometric prediction model leveraging social and search data to forecast the future impact for over 30 ingredients within the wholesome wellbeing landscape.

This future scenario horizon was critical to prioritise trends and take the guesswork out of their current innovation lanes. But beyond, our immersing into emotional and functional needs as well as unmet needs and rising demand spaces within brought the unknown unknowns to the surface that eventually initiated the development of novel innovation initiatives based on truly consumer-centric and uncharted need-solution pairs beyond the beaten paths.        

How do we set up for Unknown Unknowns discovery?

So what can we take away from this case? Being able to identify Unknown Unknowns is a widely accepted benefit of social insights compared to other research methods. This is because we operate in the sweet spot of big data and thick data analytics. We have the rare opportunity to mix the insight value of a large-scale dataset with the value from the granularity and depth of authentic human stories.

The latest advancements in deep learning have led us to this sweet spot, but the idea of invincible AI algorithms that miraculously extract Unknown Unknowns from half a million social posts about paper towels or frozen vegetables remains a naïve utopia. Exploiting the true value of these emerging Unknown Unknowns from unmet needs, blooming trends and niche audiences requires a graceful combination of machine (artificial) and human (organic) intelligence.

Blending Machine (artificial) and Human (organic) Intelligence

Semantic AI in the form of unsupervised, computational machine-learning models are an incredibly powerful way to model and decode the meaning behind emerging expressions. Algorithms, now readily available to us, not only cluster and size the topic landscape within a social data corpus, but also surface their contextual surrounding.

To go back to the wholesome wellbeing case: Here is a good example of how the surrounding contextual dimension helps empower the human analyst to further explore the “semantic neighbourhood” within a topic cluster. In the Immunity Boost topic, we see that Turmeric is strongly associated with its anti-inflammatory properties by consumers. A potential innovation platform, perhaps but not necessarily.

Machine intelligence does a great job in laying the foundations, helping us navigate between complex contextual layers – not easily available to the human eye.  Bottom-up topic modelling techniques are therefore vital in early stages to establish a comprehensive understanding of the social landscape, without the inevitable natural human biases that creep in.

However, it’s rather unlikely that any AI model will enable us to surface true Unknown Unknowns in a systematic, automated fashion. Once identified, we must now make sense of the phenomena we discover. This is where human intelligence from experienced cultural analysts, equipped with analytical, behavioural frameworks becomes essential. Immersion into underlying consumer motivations, usage occasions, emotional and functional needs are all crucial.  With this further excavation, we begin to see that the ancient spice Turmeric, in fact became a super ingredient because of various consumer benefits and need spaces.  Not only an anti-inflammatory but helpful in anti-aging, weight loss, and natural teeth whitening!

What’s the conclusion?

Unknown unknowns are more than a catchphrase to create appetite for social intelligence. They are the true revelations and the often-quoted sticky insights that eventually turn our work into actions that create business impact. They are the ultimate pointers that lead to new growth opportunities for innovation and brand growth. But it takes more than AI models to decode them. It requires Humanized AI with a meaningful blend of machine and organic intelligence from experienced cultural analysts with applied research rigor.

Sandro Kaulartz is Chief Research Officer, Social Intelligence Analytics at Ipsos check out Ipsos's profile in our social intelligence directory.

‍