Combating Media Bias with AWS Amazon Comprehend

Author(s): Bence L. Tóth

Originally published on Towards AI.

Combating Media Bias with NLP U+007C Towards AI

In the world of fake news and ideology-driven subjective media coverage, it is questionable which sources of journalism can be considered “reliable”. It happens many times that two different news outlets share two completely different takes on the same story. “Experts” point out different consequences of events from the whole spectrum, mostly the ones that suit their own agenda.

In these times, it is important to identify the biases of different news organizations towards certain topics.

Can machine learning help to discover these biases?

AWS

Amazon Comprehend is a natural language processing (NLP) service that uses machine learning to find insights and relationships in text. (Source: aws.amazon.com).

Combining AWS Comprehend with the web scraping capabilities of the R language (rvest package), an analysis can be carried out, with two articles that are written on the same topic, from two news sites with supposedly different background ideologies.

Trump’s coverage

The topic is the evaluation of Donald Trump’s first 100 days as President of the United States. In American politics, actions that took place within the first hundred days after the election are often a good approximation of the overall route that the current president is willing to take in his/her term, therefore a highly covered topic from both the left and the right.

The first article on Trump’s first 100 days is from Breitbart News, what is considered to be an alt-right news agency with columnists such as Milo Yiannopoulous and former chairman of Breitbart and White House Chief Strategist, Steve Bannon.

The second article is from Buzzfeed, a considerably left-wing internet media company, funded by Jonah Peretti, the co-founder of The Huffington Post.

Parsing the HTML content of these articles with web scraping and a CSS selector, the raw text is available. After minor data cleaning, it is possible to run Amazon’s sentiment analysis on each paragraph of the articles. Each of these is assigned an overall sentiment of “Positive”, “Neutral”, or “Negative”, with extra data on the percentage of assuredness. Eg.: a 0.78 positive measure of a paragraph means that Comprehend is 78% sure that that paragraph has a positive sentiment.

The summary of the two articles can be seen below:

We can see that Breitbart has more neutral statements than Buzzfeed, and has a higher mean of positive sentiments. On the other hand, it almost has the same negative sentiments, while Buzzfeed’s negative value is more than 3 times bigger than that of Breitbart.

The boxplot below illustrates the difference between negative sentiments:

The median of negative sentiments is a bit higher in Buzzfeed paragraphs, but non-substantially. It seems that there are some parts of the text which have a way higher certainty of negative tone which shifts the mean to be bigger in their article. This could mean that Breitbart’s takes on the worse parts of Trump’s first 100 days are not stated that harshly, while Buzzfeed uses strong words to emphasize their negative assessment of certain legislation.

Afterword

This analysis does not necessarily prove leniency towards the topic from either side as the sentiment analysis of AWS might be off for more sophisticated sentences, cannot understand sarcasm and furthermore, comparing two random articles of the same topic does not infer bias in an entire news agency. However, further analysis can be done with various topics and more similar articles to potentially infer something about certain sites’ stand on broader ideologies.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI