Getting Started with Sentiment Analysis
The most direct definition of the task is: “Does a text express a positive or negative sentiment?”. Usually, we assign a polarity value to a text. This value is usually in the [-1, 1] interval, 1 being very positive, -1 very negative.
`
Why is sentiment analysis useful
Sentiment analysis can have a multitude of uses, some of the most prominent being:
- Discover a brand’s / product’s presence online
- Check the reviews for a product
- Customer support
Why sentiment analysis is hard
There are a few problems that make sentiment analysis specifically hard:
1. Negations
A classic argument for why using a bag of words model doesn’t work properly for sentiment analysis. “I like the product” and “I do not like the product” should be opposites. A classic machine learning approach would probably score these sentences identically.
2. Metaphors, Irony, Jokes
Computers always have trouble understanding figurative language. “The best I can say about this product is that it was definitely interesting …”. Here, the word “interesting” plays a different role than the classic, positive meaning.
3. Multiple sentiments in the same text
A complex text can be segmented into different sections. Some sections can be positive, others negative. How do we aggregate the polarities?
“The phone’s design is the best I’ve seen so far, but the battery can definitely use some improvements”
Here we can see the presence of two sentiments. Is the review a positive one or a negative one? Is having a not-so-great battery a deal breaker?
These seem indeed to be complex problems. The solutions aren’t simple at all. In fact, all these issues are open problems in the field of Natural Language Processing.
For now, the best approach is to tune your algorithms to your problem as best as possible. If you are analyzing tweets, you should take emoticons very seriously into account. If you are studying political reviews, you should correlate the polarity with present events. In the case of the phone review, you should weigh the different properties of the phone according to a set of rules, maybe combine the approach with some domain-specific knowledge.
Available Corpora
There are a few resources that can come in handy when doing sentiment analysis.
- Movie reviews: IMDB reviews dataset on Kaggle
- Sentiwordnet – mapping wordnet senses to a polarity model: SentiWordnet Site
- Twitter airline sentiment on Kaggle
- First GOP Debate Twitter Sentiment
- Amazon fine foods reviews
In this tutorial, we’ll use the IMDB movie reviews corpus. It has enough samples to do some interesting analysis on it. Download it from here: IMDB movie reviews on kaggle. The corpus has many files, containing unlabeled data and test data. We’re only interested in the labeledTrainData.tsv.zip file. Unzip the file somewhere at your convenience and let’s start.
Reading the data
1 2 3 4 5 6 7 8 | import pandas as pd data = pd.read_csv("labeledTrainData.tsv", header=0, delimiter="\t", quoting=3) # 25000 movie reviews print data.shape # (25000, 3) print data["review"][0] # Check out the review print data["sentiment"][0] # Check out the sentiment (0/1) |
The sentiment in this corpus is 0 for negative and 1 for positive. As you can see, it also contains some HTML tags, so remember to clean those up later. Let’s shuffle the data and split it for training and testing.
1 2 3 4 5 6 7 8 9 10 11 | import random sentiment_data = zip(data["review"], data["sentiment"]) random.shuffle(sentiment_data) # 80% for training train_X, train_y = zip(*sentiment_data[:20000]) # Keep 20% for testing test_X, test_y = zip(*sentiment_data[20000:]) |
Using SentiWordnet
One of the most straightforward approaches is to use SentiWordnet to compute the polarity of the words and average that value. The plan is to use this model as a baseline for future approaches. It’s also a good idea to know about SentiWordnet and how to use it.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 | from nltk.stem import WordNetLemmatizer from nltk.corpus import wordnet as wn from nltk.corpus import sentiwordnet as swn from nltk import sent_tokenize, word_tokenize, pos_tag lemmatizer = WordNetLemmatizer() def penn_to_wn(tag): """ Convert between the PennTreebank tags to simple Wordnet tags """ if tag.startswith('J'): return wn.ADJ elif tag.startswith('N'): return wn.NOUN elif tag.startswith('R'): return wn.ADV elif tag.startswith('V'): return wn.VERB return None def clean_text(text): text = text.replace("<br />", " ") text = text.decode("utf-8") return text def swn_polarity(text): """ Return a sentiment polarity: 0 = negative, 1 = positive """ sentiment = 0.0 tokens_count = 0 text = clean_text(text) raw_sentences = sent_tokenize(text) for raw_sentence in raw_sentences: tagged_sentence = pos_tag(word_tokenize(raw_sentence)) for word, tag in tagged_sentence: wn_tag = penn_to_wn(tag) if wn_tag not in (wn.NOUN, wn.ADJ, wn.ADV): continue lemma = lemmatizer.lemmatize(word, pos=wn_tag) if not lemma: continue synsets = wn.synsets(lemma, pos=wn_tag) if not synsets: continue # Take the first sense, the most common synset = synsets[0] swn_synset = swn.senti_synset(synset.name()) sentiment += swn_synset.pos_score() - swn_synset.neg_score() tokens_count += 1 # judgment call ? Default to positive or negative if not tokens_count: return 0 # sum greater than 0 => positive sentiment if sentiment >= 0: return 1 # negative sentiment return 0 # Since we're shuffling, you'll get diffrent results print swn_polarity(train_X[0]), train_y[0] # 1 1 print swn_polarity(train_X[1]), train_y[1] # 0 0 print swn_polarity(train_X[2]), train_y[2] # 0 1 print swn_polarity(train_X[3]), train_y[3] # 1 1 print swn_polarity(train_X[4]), train_y[4] # 1 1` |
Let’s compute the accuracy of the SWN method
1 2 3 4 5 | from sklearn.metrics import accuracy_score pred_y = [swn_polarity(text) for text in test_X] print accuracy_score(test_y, pred_y) # 0.6518 |
The SentiWordnet approach produced only a 0.6518 accuracy. In case this figure looks good, keep in mind that in the case of binary classification, 0.5 accuracy is the chance accuracy. If the test examples are equally distributed between classes, flipping a coin would yield a 0.5 accuracy.
NLTK SentimentAnalyzer
NLTK has some neat built in utilities for doing sentiment analysis. I wouldn’t name them “industry ready”, but they are definitely useful and good for didactical purposes. Let’s check out SentimentAnalyzer.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | from unidecode import unidecode from nltk import word_tokenize from nltk.classify import NaiveBayesClassifier from nltk.sentiment import SentimentAnalyzer from nltk.sentiment.util import extract_unigram_feats, mark_negation # mark_negation appends a "_NEG" to words after a negation untill a punctuation mark. # this means that the same after a negation will be handled differently # than the word that's not after a negation by the classifier print mark_negation("I like the movie .".split()) # ['I', 'like', 'the', 'movie.'] print mark_negation("I don't like the movie .".split()) # ['I', "don't", 'like_NEG', 'the_NEG', 'movie._NEG'] # The nltk classifier won't be able to handle the whole training set TRAINING_COUNT = 5000 analyzer = SentimentAnalyzer() vocabulary = analyzer.all_words([mark_negation(word_tokenize(unidecode(clean_text(instance)))) for instance in train_X[:TRAINING_COUNT]]) print "Vocabulary: ", len(vocabulary) # 1356908 print "Computing Unigran Features ..." unigram_features = analyzer.unigram_word_feats(vocabulary, min_freq=10) print "Unigram Features: ", len(unigram_features) # 8237 analyzer.add_feat_extractor(extract_unigram_feats, unigrams=unigram_features) # Build the training set _train_X = analyzer.apply_features([mark_negation(word_tokenize(unidecode(clean_text(instance)))) for instance in train_X[:TRAINING_COUNT]], labeled=False) # Build the test set _test_X = analyzer.apply_features([mark_negation(word_tokenize(unidecode(clean_text(instance)))) for instance in test_X], labeled=False) trainer = NaiveBayesClassifier.train classifier = analyzer.train(trainer, zip(_train_X, train_y[:TRAINING_COUNT])) score = analyzer.evaluate(zip(_test_X, test_y)) print "Accuracy: ", score['Accuracy'] # 0.8064 for TRAINING_COUNT=5000 |
I’ve obtained a 0.8064 accuracy using this method (using only the first 5000 training samples; training a NLTK NaiveBayesClassifier takes a while). Not quite happy yet.
NLTK VADER Sentiment Intensity Analyzer
NLTK also contains the VADER (Valence Aware Dictionary and sEntiment Reasoner) Sentiment Analyzer.
It is a lexicon and rule-based sentiment analysis tool specifically created for working with messy social media texts. Let’s see how well it works for our movie reviews.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | from nltk.sentiment.vader import SentimentIntensityAnalyzer from sklearn.metrics import accuracy_score vader = SentimentIntensityAnalyzer() def vader_polarity(text): """ Transform the output to a binary 0/1 result """ score = vader.polarity_scores(text) return 1 if score['pos'] > score['neg'] else 0 print vader_polarity(train_X[0]), train_y[0] # 0 1 print vader_polarity(train_X[1]), train_y[1] # 0 0 print vader_polarity(train_X[2]), train_y[2] # 1 1 print vader_polarity(train_X[3]), train_y[3] # 0 1 print vader_polarity(train_X[4]), train_y[4] # 0 0 pred_y = [vader_polarity(text) for text in test_X] print accuracy_score(test_y, pred_y) # 0.6892 |
Pretty disappointing: 0.6892. I know for a fact that VADER works well for other types of text. It’s just not the case for our problem. Keep this tool in mind for your projects. Let’s try to tie things up, and build a proper classifier with Scikit-Learn.
Building a binary classifier with scikit-learn
For our last experiment, we’re going to play with a SVM model from Scikit-Learn. We’ve played already with text classification in the Text Classification Recipe. Make sure you brush up on the text classification task.
One new important addition is using bigrams. Bigrams are pairs of consecutive words. In general N-grams are tuples of N consecutive words. Here’s what I mean:
1 2 3 4 5 | from nltk import bigrams print list(bigrams("We are playing with bigrams".split())) # [('We', 'are'), ('are', 'playing'), ('playing', 'with'), ('with', 'bigrams')] |
Using bigrams instead of unigrams (aka words) is a trick for improving performance in text classification.
By using bigrams, we preserve “more context” for the words in the text.
We’re going to build:
- Unigram classifier (with
mark_negationand without) - Bigram classifier (with
mark_negationand without) - Unigram and bigram classifier (with
mark_negationand without)
1. Unigram classifier
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | from nltk import word_tokenize from nltk.sentiment.util import mark_negation from sklearn.feature_extraction.text import CountVectorizer from sklearn.pipeline import Pipeline from sklearn.svm import LinearSVC from sklearn.base import TransformerMixin clf = Pipeline([ ('vectorizer', CountVectorizer(analyzer="word", tokenizer=word_tokenize, # ! Comment line to include mark_negation and uncomment next line #tokenizer=lambda text: mark_negation(word_tokenize(text)), preprocessor=lambda text: text.replace("<br />", " "), max_features=10000) ), ('classifier', LinearSVC()) ]) clf.fit(train_X, train_y) clf.score(test_X, test_y) # with mark_negation 0.84760000000000002 # without mark_negation 0.84440000000000004 |
2. Bigram classifier
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | bigram_clf = Pipeline([ ('vectorizer', CountVectorizer(analyzer="word", ngram_range=(2, 2), tokenizer=word_tokenize, # tokenizer=lambda text: mark_negation(word_tokenize(text)), preprocessor=lambda text: text.replace("<br />", " "),)), ('classifier', LinearSVC()) ]) bigram_clf.fit(train_X, train_y) bigram_clf.score(test_X, test_y) # with mark_negation 0.86760000000000004 # without mark_negation 0.87119999999999997 |
3. Unigram and Bigram classifier
Just by changing the ngram_range to (1, 2) we obtain the unigram and bigram model
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | unigram_bigram_clf = Pipeline([ ('vectorizer', CountVectorizer(analyzer="word", ngram_range=(1, 2), tokenizer=word_tokenize, # tokenizer=lambda text: mark_negation(word_tokenize(text)), preprocessor=lambda text: text.replace("<br />", " "),)), ('classifier', LinearSVC()) ]) unigram_bigram_clf.fit(train_X, train_y) unigram_bigram_clf.score(test_X, test_y) # with mark_negation 0.88219999999999998 # without mark_negation 0.88300000000000001 |
Strangely enough, using mark negation actually lowered the accuracy of our classifiers in all cases. The classifier using unigrams and bigrams won the contest with a 0.883 accuracy.
If you are curious what features are extracted you can use this code:
1 2 3 | # Check the feature names print unigram_bigram_clf.named_steps['vectorizer'].get_feature_names() |
That’s it. This was an extensive introduction to sentiment analysis. Hopefully, you got an understanding of what the task of doing Sentiment Analysis implies, what are the most important problems we face and how to overcome them.
Good job George
Thanks 😀 Glad to help
NLTK Sentiment Analyzer program returns zero accuracy always. I didn’t modify your code, same used but could n’t find the error:
Here’s the code:
from unidecode import unidecode
import random
import pandas as pd
from nltk import word_tokenize
from nltk.classify import NaiveBayesClassifier
from nltk.sentiment import SentimentAnalyzer
from nltk.sentiment.util import extract_unigram_feats, mark_negation
data = pd.read_csv(“labeledTrainData.tsv”, header=0, delimiter=”\t”, quoting=3)
# 25000 movie reviews
print(data.shape) # (25000, 3)
print(data[“review”][0]) # Check out the review
print(data[“sentiment”][0]) #Check out the sentiment(0 / 1)
sentiment_data = list(zip(data[“review”], data[“sentiment”]))
random.shuffle(sentiment_data)
# 80% for training
train_X, train_y = list(zip(*sentiment_data[:10]))
#print(train_y)
# Keep 20% for testing
test_X, test_y = list(zip(*sentiment_data[10:12]))
def clean_text(text):
text = text.replace(“”, ” “)
#text = text.decode(“utf-8”)
return text
# mark_negation appends a “_NEG” to words after a negation untill a punctuation mark.
# this means that the same after a negation will be handled differently
# than the word that’s not after a negation by the classifier
print(mark_negation(“I like the movie .”.split()))# [‘I’, ‘like’, ‘the’, ‘movie.’])
print(mark_negation(“I don’t like the movie .”.split()) ) # [‘I’, “don’t”, ‘like_NEG’, ‘the_NEG’, ‘movie._NEG’]
# The nltk classifier won’t be able to handle the whole training set
TRAINING_COUNT = 10
analyzer = SentimentAnalyzer()
vocabulary = analyzer.all_words([mark_negation(word_tokenize(unidecode(clean_text(instance))))for instance in train_X[:TRAINING_COUNT]])
print(“Vocabulary: “, len(vocabulary)) # 1356908
print(“Computing Unigran Features …”)
unigram_features = analyzer.unigram_word_feats(vocabulary, min_freq=10)
print(“Unigram Features: “, len(unigram_features)) # 8237
analyzer.add_feat_extractor(extract_unigram_feats, unigrams=unigram_features)
# Build the training set
_train_X = analyzer.apply_features([mark_negation(word_tokenize(unidecode(clean_text(instance))))
for instance in train_X[:TRAINING_COUNT]], labeled=False)
# Build the test set
_test_X = analyzer.apply_features([mark_negation(word_tokenize(unidecode(clean_text(instance))))
for instance in test_X], labeled=False)
trainer = NaiveBayesClassifier.train
classifier = analyzer.train(trainer, zip(_train_X, train_y[:TRAINING_COUNT]))
print(_train_X)
print(_test_X)
score = analyzer.evaluate(zip(_test_X, test_y))
print(“Accuracy: “, score[‘Accuracy’] ) # 0.8064 for TRAINING_COUNT=5000
Hey Ratha,
Why do you have in your clean_text function:
text = text.replace("", " ")instead oftext = text.replace("?", " ")
You are putting one space between each letter, making them words, so instead of words as features, you have letters.
Sorry sir, again i got the same as 0. I’m using python3.5. So utf-8 not supported by this version3.5. So i removed that statement from cleaning function. I got the following output.
Training classifier
[{‘contains(.)’: True, ‘contains(,)’: True, ‘contains(the)’: True, ‘contains(a)’: True, ‘contains(and)’: True, ‘contains(is)’: True, ‘contains(to)’: True, ‘contains(that)’: True, ‘contains(of)’: True, …]
[{‘contains(.)’: True, ‘contains(,)’: True, ‘contains(the)’: True, ‘contains(a)’: True, ‘contains(and)’: True, ‘contains(is)’: True, ‘contains(to)’: False, ‘contains(that)’: True, ‘contains(of)’: False, ‘contains(I)’: True, “contains(”)”: True, ‘contains(it)’: True, ‘contains(but)’: False}]
Evaluating NaiveBayesClassifier results…
Accuracy: 0
Hi Ratha,
Please create a Gist on Github and send it over.
Bogdan.
https://gist.github.com/RathaRR/f2c6438e87066a62ae994efde374d3ed
The Gist you posted doesn’t even implement the Sentiment Analyzer …
I used the code under “Using SentiWordnet” heading in your post.
https://gist.github.com/RathaRR/3d8eb4c65f7a6fe4cda1bcd040ae41de
You are training on 5 samples … Please take a good look at your code before. Set TRAINING_COUNT to 5000 and also notice how you set the traning set to only: sentiment_data[:10]
I tried for 5000, Even though i got 0.
https://gist.github.com/RathaRR/3d8eb4c65f7a6fe4cda1bcd040ae41de
Kind of late, but I ran into this same issue and figured out the problem.
The users running into the 0 accuracy bug are trying to port the code to Python 3. In Python 3 the zip() built-in returns a generator instead of a list. The other code posted here addresses this in the sections that will throw an exception (due to indexing) by wrapping the zip() in a list(), however, they do not similarly wrap those passed to the SentimentAnalyzer.
The SentimentAnalyzer will not complain when passed a generator in place of a list in this way, but the evaluate method does not work properly and appears to simply return an accuracy of 0 without doing anything substantial. Strangely, the train method seems to work fine either way, and the inconsistency makes me suspect this is actually a bug in NLTK. In any case, wrapping the zip() passed to evaluate in a list() should make the code work as expected.
Forget to post code. Just in the solution is clear:
score = analyzer.evaluate(list(zip(_test_X, test_y)))Thank you!
Dear George.,
Excuse me how can I get steps of build dictionary using Vader dictionary on facebook comments and can you help me by sending me figure shows a list of output ( positive and negative words ).
Thank you.
Regards.
Hi Raed,
I’d love to help you out with info, unfortunately, I don’t fully understand the question.
The Vader method should work on FB comments out of the box.
Bogdan.
Thank you bro
for the method that I work on it as the below
Methodology
The Implementation of this study start with extract message column from Facebook comments using face pager tool then the second step is the pre-processed for extracted data which started with Tokenization then Stemming finely POS Tagging in this step identified which words are adjectives then we identified which words related with sentiments (adjectives) by getting an example of happy words an sad words by using vader dictionary via nltk tool after that selected the words (positive , negative ) we have to classifiy these words as positive and negative words from these point we built a lexicon and we produced like a database in side lexicon there is a list of words( positive , negative) . Once we got this lexicon we used it to tag our post again to built training data . the process of building training data include it comments and there class so after that to we have to do machine learning on this data to train the midel by using one of the clasifires like ( Naivebyse ,SVM ….) Once the model get high accuracy parameters we will test it in our work. Figure 1 shows an architectural methodology of the proposed combined Lexicon and Machine Learning techniques. Our report progress consist of three steps namely data extraction, pre-processing and build dictionary.
I have progress report but I do not know how can explain the part of build dictionery
thank you.
Thanks a lot for this post, this was very helpful. I am trying now with some samples to see how the predictor is doing, but I am struggling on how the input-data has to look like. When I tokenize it, I get an array of predictions per word. When I enter a sentence I get an error message, that it was expecting an interable, not a string. When I enter a list, python wants me to send a string, etc… How would I have to structure a simple sentence or paragraph to get the prediction?
Hi Peter,
Are you referring to the sklearn model? Sklearn models require a “list” of inputs for the transform/predict methods.
So, model.predict([“this is a text”, “this is another text”]) should do the trick
Model is a pipeline containing the vectorizer and the predictor. if model is only the predictor then
model.predict(vectorizer.transform([“this is a text”, “this is another text”])) will do
Thanks. I used the sklearn model indeed. model.predict([“bla bla bla”]) works. Thanks!!
[…] a way, you created a Bag-Of-Words model when you tried text classification or sentiment analysis. It basically means you take the available words in a text and keep count of how many times they […]
Hi George,
Really a great article for beginners like me.
Can you please suggest me something to detect exaggeration in any article?
Thanks,
Pritam
Hi Pritam,
Sorry, haven’t done anything in this area. Seems like an interesting one. Let me know if you find something interesting 🙂
Regards,
Bogdan.
[…] often have to deal with the simple task of Binary Classification. Some examples are: Sentiment Analysis (positive/negative), Spam Detection (spam/not-spam), Fraud Detection […]
Hi bogdani,
Ur article is good for the beginners.thanks for your article.
also i wld like to know that is it possbile to write the result of each review and its corresponding sentiwordnet score in csv file.i am trying but not able to write.
if so kindly provide me the code.
Hi SSMadhu,
Here’s how to add a column to dataframe: https://stackoverflow.com/questions/40045632/adding-a-column-in-pandas-df-using-a-function
Something like this should work: data[‘polarity’] = data[‘text’].apply(swn_polarity)
share ur mail id. i have send my code . i have got some error.kindly do the needful
May I know why I got 0 accuracy for NLTK SentimentAnalyzer?
Hey Zozo,
Not sure, you might need to provide more details 🙂 Are you sure the classes from train/test set are identical?
Bogdan.
Hi Bogdan,
Would you help me to check this code ?
import pandas as pd#read data
# 25000 movie reviews
data = pd.read_csv("labeledTrainData.tsv", header=0, delimiter="\t", quoting=3)
print (data.shape) # (25000, 3)
#print (data["id"][0])
print (data["review"][0]) # Check out the review
print (data["sentiment"][0]) # Check out the sentiment (0/1)
#split for training and testing
import random
sentiment_data = list(zip(data["review"], data["sentiment"]))
random.shuffle(sentiment_data)
# 80% for training
train_X, train_y = list(zip(*sentiment_data[:20000]))
# Keep 20% for testing
test_X, test_y = list(zip(*sentiment_data[20000:]))
def clean_text(text):
text = text.replace("", " ")
#text = text.decode("utf-8")
return text
from unidecode import unidecode
from nltk import word_tokenize
from nltk.classify import NaiveBayesClassifier
from nltk.sentiment import SentimentAnalyzer
from nltk.sentiment.util import extract_unigram_feats, mark_negation
# mark_negation appends a "_NEG" to words after a negation untill a punctuation mark.
# this means that the same after a negation will be handled differently
# than the word that's not after a negation by the classifier
print (mark_negation("I like the movie .".split())) # ['I', 'like', 'the', 'movie.']
print (mark_negation("I don't like the movie .".split())) # ['I', "don't", 'like_NEG', 'the_NEG', 'movie._NEG']
# The nltk classifier won't be able to handle the whole training set
TRAINING_COUNT = 5000
analyzer = SentimentAnalyzer()
vocabulary = analyzer.all_words([mark_negation(word_tokenize(unidecode(clean_text(instance))))
for instance in train_X[:TRAINING_COUNT]])
print ("Vocabulary: ", len(vocabulary)) # 1356908
print ("Computing Unigram Features ...")
unigram_features = analyzer.unigram_word_feats(vocabulary, min_freq=10)
print (*"Unigram Features: ", len(unigram_features)) # 8237
analyzer.add_feat_extractor(extract_unigram_feats, unigrams=unigram_features)
# Build the training set
_train_X = analyzer.apply_features([mark_negation(word_tokenize(unidecode(clean_text(instance))))
for instance in train_X[:TRAINING_COUNT]], labeled=False)
# Build the test set
_test_X = analyzer.apply_features([mark_negation(word_tokenize(unidecode(clean_text(instance))))
for instance in test_X], labeled=False)
trainer = NaiveBayesClassifier.train
classifier = analyzer.train(trainer, zip(_train_X, train_y[:TRAINING_COUNT]))
score = analyzer.evaluate(zip(_test_X, test_y))
print ("Accuracy: ", score['Accuracy']) # 0.8064 for TRAINING_COUNT=5000
Hello George,
nice post. It helped me a lot!
Awesome to hear that 🙂
Hi ,
I am trying build generic sentiment analyser bu considering negations as well. can youplease guide me, how can i handle double negations?
Thank you
If you carefully read the http://nlpforhackers.io/sentiment-analysis-intro/ article, you’ll see that at one point we make use if the
mark_negationfunction. Check that outHi Bogdan,
I have a question regarding the accuracy of the method of SentiWordNet. As I know that SentiWordNet is an unsupervised learning method, how can it be possible to obtain an accuracy for unsupervised learning?
*I am sorry if this question does not make sense. I am new to this field. Thank you!
Hi Priscilla,
Thank you for your question. Doing sentiment analysis with SentiWordNet is not exactly unsupervised learning. In fact, it is not a machine learning model at all. We call this a “Corpus-based method”. We have polarities annotated by humans for each word. Using these polarities we apply a heuristic method for deriving the polarity of the entire text. We hope that averaging the polarities of the individual words will be a good predictor of the overall sentiment of the entire text. We already know what the polarities of the texts are because the texts come from an annotated corpus. This is how we can compute the accuracy of the method. If the texts were not annotated with a sentiment score, indeed, there’s no way we can compute the accuracy.
Hope this makes sense,
Bogdan.
Hi, can you share me code for aspect based sentiment analysis [email protected]
Hi,
This actually helped me a lot.
I am wondering what could be done in the case of emails or a whole document. Like, I tried using the Vader Sentiment Analyzer to find the sentiment of one the Email:
“”All done! I don’t think she was too impressed but she’ll have to get over it! I’ve to do John tomorrow I’m allergic. Delighted I didnt have him on the same day as Mary though… Nightmare!!! Here’s the notes from today.””
and the results produced were:
[{‘neg’: 0.059, ‘neu’: 0.779, ‘pos’: 0.162, ‘compound’: 0.7067}]
Even though it is clear that the email has a negative sentiment, the analyzer returned it as a text with positive sentiment.
Could you suggest me what I should do in this case or is there a better way to build a sentiment analyzer for the Documents.
Cheers.
Hi Shivansh,
The Vader Analyzer is geared towards short, social-media type text. It won’t be effective for what you need. Remember, a model works best on the data it was trained on, hence you’ll need to build a model trained on that type of emails.
Best,
Great article for Beginners . Definitely would like to see more of such articles.
Thank you, Sir 🙂