How can we use digital conversations to predict the future?

As digital conversations help us understand the mindsets, opinions, and interests of consumers, they can prove to be incredibly valuable for informing different aspects of the business. One of the more advanced uses of conversational data is in making predictions about the future.

At this year’s Observe Summit, Amy A Laine, IBM Distinguished Market Intelligence Professional, Ben Klein, Associate Director of Consumer Business Intelligence at Johnson & Johnson, and Jane Quigley, Chief Client Officer at Converseon, discussed the use of digital conversations to predict the future.

Getting buy-in for social insights

Before you can start using conversational insights to make predictions, you first have to get buy-in from the top. Both Ben and Amy agreed that it’s important to keep track of everything you’re doing with social data and show how it’s making an impact.

To get buy-in, you need to demonstrate the power of social insights by showing its impact on business KPIs. “Continue to illustrate impact with deepening social intelligence and show how you can apply social methodology to answer business questions,” explained Amy.

The challenges of getting quality data

Now comes the challenging part: getting quality data out of digital conversations. Although social media serves as a valuable source of data, it also comes with a lot of noise. This makes relevancy a challenge as you have to go as broad as possible to include all the data you need, but eliminate the noise to narrow down the most relevant insights.

Additionally, accurate sentiment analysis is another big challenge. Out-of-the-box sentiment models usually take a one-size-fits-all approach, which prevents you from getting too specific with your analysis.

Eliminating the noise from digital conversations

In order to eliminate the noise, you need to take a closer look at your Boolean logic. In particular, focus on your exclusions.

For example, collecting data on all terms around “cloud” might be unable to return meaningful results for cloud computing stakeholders. The data will be muddled with conversations about “rain clouds”, “cloud nine”, and “clouded thinking” etc. Unless you specifically exclude these terms in your analysis, they will appear in your results.

So, while social media serves as the world’s largest focus group, your Boolean logic serves as the “focus group screener,” which allows you to recruit specific participants that are ideal for your study.

Similarly, with sentiment analysis, customisation is extremely important. You need a specific sentiment model that is able to discern the context of the conversation in order to accurately read into the sentiment behind a statement.

‍

Understanding the “why”

For any research project, it’s important to approach it with a clear idea of what you’re seeking. Why are you collecting the data?

This is why having a hypothesis or a use case in mind is effective. You can then organise the data in a way that helps answer business questions specific to you. In this case, it would be about future predictions.

“There’s a level of customisation that’s important to make sure you’re getting the best use of the data. But you also don’t want to do too much in the sense that you’re leading the data to tell you something that they think you want to hear,” Ben pointed out.

Tools vs. manual analysis

Social listening tools help make the job efficient, but you can’t rely on tools alone to get really granular and accurate with your research. Social data is based on language, which is inherently imperfect and biased. As such, there’s a need for deliberate attention that only manual analysis can offer.

“The quality of your data determines the quality of your decisions, and in turn, the quality of the action taken by those in your business,” according to Amy.

Making your analysis predictive

Data is more powerful in the presence of other data. So, social data is best viewed together with other data sources, such as customer service call logs, sales data etc. This allows you to establish a link to sales or to accurately judge whether you have a brand reputation crisis on your hand.

“So where we stand, triangulation is key and when data converges, it converges on the truth,” Amy said.

When moving into modeling for predictive analytics, you want a lot of data that’s clean and relevant so you can test if there’s a relationship between things. What movement do I see in my methods when something changes in either or both?

“The best way to become predictive from social is to identify those leading and lagging indicators and their relationships with outside data,” agreed Ben. When we talk about building those relationships, you have to do that through historical data.

“Go back in time and run those correlations and see what the relationships are that are building that you can then use to become predictive.”

‍