Human vs. machine: Can tools provide accurate sentiment analysis?

Sentiment analysis is an important part of social listening. It adds context to social media conversations, so brands can understand the emotions behind them. This helps them get a better idea of how consumers are really talking about them, the pain points they need to address, and how they stack up against the competition.

Many listening platforms already come with built-in sentiment analysis. But how accurate is it? Can brands rely on it as a true measure of sentiment? This is one of the topics that came up during SI Tech Demo Day 2021.

How do tools generally calculate sentiment?

At the most basic level, tools will calculate the number of positive and negative words in a conversation. So one negative word equals -1 and one positive word equals +1. They will then determine the overall sentiment score surrounding that conversation.

But this process has its limitations, since the tool may not be able to detect subtleties of sarcasm or irony. That means they often bring up false positives and false negatives, which will affect the overall sentiment score.

Most modern social listening tools use natural language processing (NLP) for more accurate sentiment analysis. They analyse conversations to determine the tone behind certain words and phrases. The goal is to understand the attitudes behind opinions and statements even when sarcasm and slang are involved.

Some tools even go further than the surface-level sentiment expressed in conversations. NetBase Quid, for instance, also measures brand passion. It looks at whether people have positive or negative emotions toward a brand and gives context about how strong those emotions are.

On the same note, D-TAG can also measure the depth of a feeling to help you dig deeper into your audience’s attitudes and emotions. So you get a more nuanced and reliable understanding of how people feel about your brand.

NicheFire’s AI also uses a nuanced spectrum to get a more complete understanding of consumer sentiment. It looks at the nuance of language to determine things like buying tendency, brand loyalty, and passion.

Factors to consider in sentiment analysis

Sarcasm, irony, and slang may affect the precision of sentiment analysis. But many listening tools have been programmed to detect these elements that are often present in natural human language.

One other major concern is in analysing non-English languages. With AI technology being originally designed for English language, sentiment analysis for other languages tends to be less accurate.

Even without considering sentiment, working with non-English data is a challenge on its own. These tools often struggle to accurately consider cultural differences. There may be local sayings and words in your target language that mean something completely different. For example, the common euphemism for “dying” in the US and the UK is “kicking the bucket.” But in Germany, people will use the phrase “giving your spoon away.” Without a local analyst to give that cultural context, it’s hard to gather accurate insights.

And literal translations no longer work to accurately read the information and analyse sentiment from it. Even within the same language and country, there may be many regional words and sayings that don’t make sense to someone from a different part of the country. The same way “puppy chow” in the Midwest refers to a popular homemade snack and not dog food.

So there are many nuances, cultural differences, and local slang that could affect the accuracy of sentiment analysis.

While social listening tech has evolved to give an “accurate enough” analysis of sentiment, there are scenarios that will still call for a human perspective. Especially when it comes to sentiment analysis in non-English conversations, having a human analyst to add more cultural context is important.

‍