AI Driven Sentiment Analysis
What is Sentiment analysis?
Sentiment Analysis is a method of analyzing human emotions expressed in words through AI.Sentiment analysis helps companies to understand negative attitudes towards their products, empowering them to make a change and address those issues in real time.Sentiment analysis can be achieved by following predetermined metric to understand how positive, neutral, or negative a piece of text sounds.
- “I really like the new design of your Car!” → Positive
- “I’m not sure if I like the new Car” → Neutral
- “The new Car is awful!” → Negative
Types of Sentiment Analysis
Emotion Detection Sentiment Analysis
Emotion detection sentiment analysis is more algorithm based sentiment analysis by which identifying emotion in piece of text.lexicon and machine learning are used in emotion detection sentiment analysis.Lexicons works based on valence directory. Valence directory marks words in text either positive,negative or neutral.Take phrase
“Good people sometimes have bad days”
Valence directory considers the word “good” as positive ,“bad” as negative and the rest of words as neutral.Overall sentiment score(StSc) of phrase can be calculated using below formula
StSc= number of positive words-number of negative words / Total number of words
If sentiment score is negative,phrase consider as negative,else positive sentiment score classified as positive phrase and score of zero means phrase is neutral.
Aspect based sentiment analysis(ABSA)
Aspect based sentiment analysis derives text data and defines its sentiment based on its aspects.Usually service based companies use this technique to analyze consumer feedback data by correlating emotions to different aspects of a product or service.Aspects refers attribute of product or services such as “user reviews on a product”
ABSA retrieves sentiments and aspects of product.
Sentiment:Positive or negative opinions of product
Aspects:feature,category of product
ASBA is more important, since customers’ feedback can be sorted automatically through ASBA. Nowadays customers spend more time expressing their thoughts and experience of products they used or purchased.ASBA is widely used because,it can handle massive amounts of data. It is impossible for teams to analyze huge data manually.
How to do Aspect based sentiment analysis
Step 1:Gather information
Before getting into ABSA,we need to collect information.Internal data gathered through surveys,Customer Relationship Management(CRM) software and external data via Web scraping tools
Surveys
Online Survey tools like surveymonkey and Typeform creating and sharing surveys make easy
NPS(Net promoter score)
Companies uses NPS software to collect information from their customers.NPS is measurement of how much customers are satisfied with a company’s product.The score is calculated by asking customers one question “On a scale from 0 to 10, how likely are you to recommend this product/company to a friend or colleague?”.Based on aggregated NPS scores ,company can service, customer support, delivery, etc. for increased customer loyalty.
Most commonly used NPS softwares are satismeter,promoter.io,Delighted.
NPS calculation
NPS is calculated by subtracting the percentage of detractors from promoters.customers who answer with a 9 or 10 (known as ‘promoters’) and customers who answer the NPS question with a 6 or lower (known as ‘detractors’).
Customer Relationship Management(CRM) software
CRM software is a technology for managing all your company’s relationships and interactions with customers.It is important for companies to retain their customers for the long term by giving the best impression on the first sale.companies can identify vulnerabilities in their product using ABSA models and improve and predict customer demands to beat competitive edge.
Web Scraping Tools
Web scraper is a tool for extracting publicly available data from websites. Dexi.io,portia and ParseHub are web scraping tools.
Step 2:Execute Aspect Based Sentiment Analysis
NPS Templates are available in MonkeyLearn tool to create ABSA.Choose appropriate template and upload data collected through surveys,NPS scoring.
After uploading data file,match data to right fields in each column
Created at:Date when feedback was sent
Text:Feedback test
Score:NPS score provided by customer
After mapping these columns,Enter name for workflow and click on create button
Then MonkeyLearn tool will process data
Once MonkeyLearn data processing completed,report will be generated
Fine Grained Sentiment Analysis
A sentence is broken into phrases or clauses and each part evaluated in connection with others.Fine grained sentiment are used for processive comparative expressions. For example
Oneplus is way better than iPhone
Fine grained sentiment analysis results are more precise.It allows users to understand both how customers analyze your products and which feature or aspect they discuss.
eg:The keyboard on my laptop stopped working after 2 months.
So that companies can identify features that need to be improved.
Intent based Sentiment Analysis
Intent based sentiment analysis used to classify intent of text whether it’s negative,positive or neutral.
If customers complain about dissatisfaction about a product they purchased on online sites,it can motivate customer service centers to contact customers and solve the problem.
Benefits of conducting sentiment analysis for businesses.
1.Derive Meaning from Formless data
It is a common challenge that most companies face to analyze feedback manually for large groups of customers. With help of Sentiment analysis ,data gathered from customers are organized by their emotional tone positive,negative and neutral
2.Take Rapid action against awful experience
companies have to provide better customer experience to beat competitors.Companies should act upon quickly,if their customers are facing abhorrent experiences.Companies can take necessary actions before they lost their customers using sentiment analysis.Sentiment analysis examine and categorize customer data using emotional content.Negative feedback will be highlighted.
3.Develop Chatbots to increase customer engagement
By Analyzing data collected through sentiment analysis, companies can develop chatbots and answer customers’ queries.
How AI-based sentiment analysis works?
Earlier companies used surveys to understand customers feelings about products and services.Recent days companies depends on sentiment analysis which is subfield of natural language processing(NLP)
Sentiment analysis follows 2 techniques
- Rule based sentiment analysis
- Machine learning based sentiment analysis
Rule based sentiment analysis
Rule based sentiment analysis defined by a set of algorithms. Algorithms applied on sentiments taken from customer data.Algorithms can use NLP techniques such as tokenization and stemming.This methods follows dictionary that contain words with sentiment score
| Word | Sentiment |
| Good | 0.5 |
| Excellent | 0.9 |
| Terrible | -0.9 |
Rule based sentiment analysis does not consider how words are arranged in a sentence.New rules can be added. But it will update existing rules. Dictionary must be properly maintained with the latest rules.
Machine Learned Based Sentiment Analysis
Machine Learned Based Sentiment Analysis introduced to detect sarcasms,synonyms and other complex sentiments in a sentence.This uses ML model to extract data using labeled datasets.Algorithms developed after training ,deduce sentiments from new text.This method doesn’t follows predefined rules.
For better results,both rule based sentiment analysis and ML model can be combined into a system.
Sentiment Analysis use cases in real life experience
- Brand building and monitoring management
- Social media posts
- Customer support request
- Customer reviews from surveys
- Text from customer reviews and emails
Brand building and monitoring management
Use Case:How KFC use sentiment analysis in brand building
KFC struggled in past for giving customers feel good experience and healthy food in competitive world.KFC used sentiment analysis for brand building and monitoring.They tried memes and pop culture iconography
meme:piece of text,an image in a humorous way,rapidly spread over the internet by internet users.
Pop culture iconography:by using RoboCop to promote the newest product.As results,customers can engage with brands and then it leads to engaging products.
Use Case:How Google is doing it?
Google uses sentiment analysis sentiment analysis application to improve their products.
example:Google chrome browser development team uses sentiment analysis technique to continuously monitoring customer reviews
Development team looking for not only text written by customer feedback ,also sentiment (positive or negative),Specific aspects of product scalability,UI or extensions
Sentiment analysis integrated Chrome extensions are introduced to recognize facial expressions to determine if you are sad or happy.
ML5js libraries integrate facial recognition to chrome extension applications.
Social Media Sentiment Analysis
Use Case:How Nike use social media sentiment analysis
Colin Rand Kaepernick was better known for kneeling during the national anthem played in NFL 1981 “I am not going to stand up to show pride in a flag for a country that oppresses Black people and people of color. To me, this is bigger than football,” Kaepernick commented.He was criticized by most of public.Meanwhile he got a lot of support, Nike sponsored him to use #justdoit hashtag in his tweets. It was a risky move because Nike needs to keep its reputation among the public. Sentieo tool used to evaluate public feedback by analyzing tweets before and after sponsored to Kaepernick to use #justdoit.
Initially public opinions were negative.But gradually positive sentiments received from the public.
How Watchshop used text sentiment Analysis
Watchshop is an online retail company.Customer data is collected through surveys and emails and using AI tool Lumoa to get a final sentiment score.Based on Score they improved product listing screens.They executed A/B testing to see which app version performs better.
How Customer support Sentiment Analysis helpful for Mobile providers:Mobile Carrier
Mobile provider Companies use customer support sentiment analysis to find out customers at risk of tossing.
Speech to text software was installed to set out call center interactions and then user sentiments and aspects were evaluated using Repustate.After calculating final sentiment score for each customer,if the score is below a certain threshold for too long ,it will automatically send an apology message to customer.
Solutions to Defeat Challenges in sentiment Analysis
1.Emojis
Social media content most probably text basis ,sometimes includes emojis to express sentiment.NLP methods are trained to be language specific. Even Though NLP can extract data from images,While extracting emojis through emotion analysis,it will be considered special characters and removed during the process. So that actual customer feedback insight will not be understood
Solution
Emotion analyzer tools were introduced to analyze emojis. These tools decode language in emojis itself and will separate them from special characters.
3.Sarcasm detection
In Sarcastic text people express their negative emotions in positive words.Mostly sentiment score will be calculated as positive even though feedback added by customer is negative
Solution
Top tier sentiment analysis API will be helpful to detect actual sentiment of person by analyzing actual context of feedback
4.Polarity
Problem
Sometimes it’s not easy to classify sentences as positive,negative or neutral ,since feedbacks are robust and apparent. eg:”That was terrible experience”
Solution
Such polarity issues can be resolved by,giving polarity scoring to words in training dataset and classify between “very Good” and “Slightly good”
5.Multilingual data
Problem
As companies expand,they have to deal with customers from multiple languages. Most probably sentiment analysis will be developed for one language,some sentiments may get lost in translation.This becomes a problem,while handling sentiment analysis for languages other than english feedbacks.
Solution
Only solution for this issue ,sentiment analysis, is to design in such a way a uniquely trained platform and for each language to have a named entity recognition model to analyze feedback in their own langugae.