Sentiment Analysis in Python Using Flair

Author(s): Mahesh Tiwari, PhD

Originally published on Towards AI.

Welcome to the next blog post in the series on sentiment analysis! Today, we will be exploring Flair, one of the methods used in the Python library for sentiment analysis.

The sentiment analysis was done for the movie “Extraction 2” using the collected Twitter data. You can find the data on kaggle.com download it from this link. You can find the complete list of Sentiment analysis series in this link which includes extracting data from Twitter, preprocessing, sentiment analysis using TextBlob and sentiment analysis using Vader.

Flair for Sentiment Analysis: A Powerful NLP Library

Flair is an NLP library developed by Zalando Research [1], focusing on sentiment analysis tasks. It uses sequence labeling, a deep learning approach, to train models to predict sentiment labels for each word or token in a text [2]. This enables the model to capture the sentiment expressed by individual words or phrases within a sentence or document. Flair provides pre-trained models in a variety of languages [3] that may be utilized without requiring considerable initial training. Additionally, it enables the pre-trained models to be fine-tuned using unique datasets relevant to a particular application or area.

The main benefit of Flair is its capacity to extract contextual information [2], taking into account nearby words and sentence construction, to more fully comprehend the mood communicated in complicated and confusing texts. Text data must first be preprocessed before the pre-trained model can be loaded and used to predict the emotion of inputs using Flair. The user-friendly and straightforward API makes the integration of sentiment analysis capabilities into NLP applications easier. Flair is an effective tool for academics and developers to create precise sentiment analysis models for a variety of applications, including social media monitoring, customer feedback analysis, and opinion mining [2].

Note: To install Flair we can use pip install flair

classifier = TextClassifier.load('en-sentiment')

for index, row in df.iterrows():
 text = row['tokens'] 
 sentence = Sentence(text)
 classifier.predict(sentence)
 sentiment = sentence.labels[0].value
 score = sentence.labels[0].score
 df.loc[index, 'sentiment'] = sentiment
 df.loc[index, 'score'] = score


print(df.head())

Below is the main code snippet that is used for sentiment analysis using Flair.

# Load the pre-trained sentiment analysis model
classifier = TextClassifier.load('en-sentiment')

# Iterate over each row in the DataFrame # :( its really time consuming :(
for index, row in df.iterrows():
 text = row['tokens'] 
 sentence = Sentence(text)
 
 # Predict the sentiment for the current sentence
 classifier.predict(sentence)
 
 # Get the predicted sentiment and score
 sentiment = sentence.labels[0].value
 score = sentence.labels[0].score
 
 # Update the DataFrame with the sentiment and score
 df.loc[index, 'sentiment'] = sentiment
 df.loc[index, 'score'] = score

# Print the 'sentiment' column of the DataFrame
print(df['sentiment'])

The code sample categorizes feelings in a DataFrame using the Flair package. Each entry in the DataFrame is processed using a loaded sentiment analysis model that has already been trained. The classifier is used to determine the sentiment and score predictions for each text. The projected sentiment and score are then added to the DataFrame.

The above code added two new columns, which are

Visualization

Next, the matplotlib library is imported as plt. The code counts the occurrences of each sentiment category in the ‘sentiment’ column of the DataFrame and stores the counts in the sentiment_counts variable. Then, a bar plot is created.

import matplotlib.pyplot as plt

# Count the occurrences of each sentiment label
sentiment_counts = df['sentiment'].value_counts()

# Plot a pie chart
plt.pie(sentiment_counts.values, labels=sentiment_counts.index, autopct='%1.1f%%')
plt.title('Sentiment Analysis Results')

plt.show()

# Print the number of counts for each sentiment
for sentiment, count in sentiment_counts.items():
 print(f"{sentiment}: {count}")

Conclusion

Based on the sentiment analysis results obtained using Flair for the Extraction 2 movie data scrapped from Twitter, we have the following sentiment counts

NEGATIVE: 6073

POSITIVE: 3926

As we can see that it only contains negative and positive sentiments. Interestingly it neglects neutral sentiments. One obvious reason for this is the sentiment labels used in Flair’s pre-trained sentiment analysis models typically include positive and negative sentiments, but not neutral.

We can extend Flair's sentiment analysis capabilities by training a custom model to include a neutral sentiment label. However, please note that this would require collecting or annotating a suitable dataset and training the model specifically for your desired sentiment classification task.

Based above sentiment counts, it seems that the movie received negative feedback, which is due to the lack of neutral sentiments in the pre-trained model. So it may not reflect the clear analysis we are looking for. It’s wise to build a custom model on our data and do further analysis.

References

[1] Zalando Research. (n.d.). Flair — a powerful NLP library. Retrieved from https://github.com/zalandoresearch/flair

[2] Akbik, A., Blythe, D., & Vollgraf, R. (2018). Contextual String Embeddings for Sequence Labeling. Retrieved from https://www.aclweb.org/anthology/C18-1139/

[3] Akbik, A., Bergmann, T., Blythe, D., Rasul, K., Schweter, S., Vollgraf, R., & Zalando SE. (2019). FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP. Retrieved from https://www.aclweb.org/anthology/N19-4010/

FOLLOW ME to be part of my Data Analyst Journey on Medium.

Let’s get connected on Twitter or you can email me at [email protected] for project collaboration, knowledge sharing or guidance.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI