Fake Reviews: Maybe You Should Be Worried About AI’s Writing (and Reading) Skills

Author(s): Dora Cee

Originally published on Towards AI.

In a recent, rather troubling, study humans could detect fake reviews with a measly 55.36% success rate. As for AI? It boasted a 96.64% accuracy.

Fake reviews have become a steady crutch for many companies relying on misleading information to hoard in sales. To make things worse, we are a creative lot when it comes to hacking “the system”. Hence, many have already taken false promotion further by reaching for AI.

Needless to say, the abuse of machine-generated recommendations results in an unfair playing field between retailers and disillusioning shopping experiences for consumers. On the flip side, at least we are getting better at taming AI. The irony is that humans are just not good at telling apart writings penned by humans and those by AI if the latter is given enough learning material. It’s almost like we’ve never encountered the lore of Pandora’s Box before.

But I digress — let’s get back to e-commerce first. What should be a useful source of word-of-mouth insight for assessing a product’s quality, relevance and suitability have become notably fractured. All these artificial outputs seem to be gradually eroding trust in online reviews, at a time when social trust is already in decline.

The problem is not only the staggering number of deceptive reviews and spam, which come in many shades of fiction. The social media influencer promoting the latest snake oil is one thing, but machine learning has unleashed a storm of fabrications on a whole other level. Current text generation methods can now create fine-tuned, realistic scripts — and their detection is challenging for humans.

Topics Covered:

At a Quick Glance: Introductory Pointers
Data Mining Truthful Reviews
The Pen is Mightier Than The Cue
Detection Strategies for Fake Reviews
Combating AI with AI
Practical Everyday Tips for Spotting a Fake
Key Takeaways

At a Quick Glance: Introductory Pointers

Unable to tell fact from fiction, consumers are already rightfully full of doubts. Below are some of the problems this downward spiral is slowly cementing.

U+27A4 Though we may still not have hit rock bottom, eventually the damage to the source credibility of reviews might downgrade them to nothing but the online marketplace iteration of junk mail.

U+27A4 Algorithms also refer back to reviews to determine a product’s ranking in a category. This means that computer-generated reviews can be weaponized to either inflate or deflate marketplace ranking artificially, resulting in unfair competition.

U+27A4 Companies can target rivals by harnessing the destructive impact of negative reviews to lower the other’s visibility on online platforms and search engines. Note that your competitors could also poach your customers by using your brand. In this digital battle of flooding the market with fake appraisals, ultimately only the swindlers can gain ground unless fake reviews become easier to spot and firmly disincentivized.

U+27A4 The damage to a seller’s reputation can also spiral out of control in the age of social media and swift information exchange. Negative reviews can be easily circulated on multiple platforms, potentially making the company in question go viral for all the wrong reasons. And guess what? Unethical practices or even just rumors can get you canceled overnight.

U+27A4 Competitive advantage can also be obtained by manipulating users’ perceptions about products and companies alike. Confirming this fact, a 2016 study by Luca estimated that a one-star addition to a Yelp rating leads to a 5–9% increase in revenue.

U+27A4 Still not convinced? Allow me to frame it differently. In a 2015 report, the CMA estimated that “£23 billion a year of UK consumer spending is potentially influenced by online reviews”. Don’t underestimate the power of social proof.

Data Mining Truthful Reviews

Refraining from cheating at the online marketplace game comes with some pretty powerful upsides.

U+278A First of all, authentic reviews describe the quality of the product or service in a more believable way. This, in turn, paves the way for a more constructive type of communication.

U+278B Second, consumers are able to better tell whether it is really what they need and what they should expect. In the end, a disclosure of honest experiences and impressions is in the interest of both the buyer and the seller.

U+278C It identifies aspects that are relevant to the product and its usage. Some of these may have entirely escaped the company’s attention, for example, due to insufficient testing or misaligned mental models.

U+278D On the other side of the equation, companies can use them as valuable feedback for shortcomings and improvements. Many firms pay good money for surveys and product testing.

Why, then, would anyone want to ignore original comments from the consumer directly, especially when they come free of charge? Analyzing data from reviews can be put to good use if considered and harnessed properly.

The Pen is Mightier Than The Cue

A fake review can obviously be composed by a human as part of an exploitive marketing campaign or other incentivized pursuit. However, technology is reaching advanced levels of generated copy-writing and skill at deception. Already, people are finding it challenging to tell the difference between an AI-produced text and a genuine consumer’s opinions.

Exploiting the easy distribution channel of the digital world is the most recent, appealing shortcut to dishonest sellers. Sadly, the influx of fake users and content could become unmanageable unless measures are implemented to tackle misinformation.

This practice is already forbidden by The Consumer Protection from Unfair Trading Regulations 2008, as per the below:

(11) Using editorial content in the media to promote a product where a trader has paid for the promotion without making that clear in the content or by images or sounds clearly identifiable by the consumer (advertorial).

The problem remains the lack of detection strategies being available and used (for now). Once the crackdown on fake reviews begins, chances are individual companies relying on false claims will be crushed under substantial fines.

One challenge in telling truth from lie is that spammers evolve as tactics and advice for sharper perception become common knowledge. Case in point: a 2018 study by Plotkina, et al. found that only 57% of consumers could detect a fake review, even after learning of information cues to keep in mind. This being only slightly above the level of chance does not paint an optimistic picture of our ability to flag lies in written context.

Detection Strategies for Fake Reviews

The other problem is the sheer number of reviews online and their promptly scaling volume. Combing through them manually is an impossible task, so researchers are reaching for an automatic method. Bringing algorithms to the rescue means they scour the data samples to identify patterns.

According to a study by Cardoso, et al. (2018), these detection strategies can be:

• content-based, where the emphasis is on textual content,
• behavior-based, checking for atypical and suspicious behaviors,
• information-based, focusing on product characteristics, or
• known as spammer group detection, gauging connections between reviewers.

Automatic fake review detection has only had partial success, but combining different types of approaches perform better overall. So far it has been rare to see automatic and manual methods utilized together. Nevertheless, it’s worth a mention that it still comes with marginal gains, according to research by Harris (2019).

Third-party tools can also be used to weed out fake reviews. For example, both Fakespot and Reviewmeta specialize in spotting inauthentic Amazon reviews.

Combating AI with AI

A study published in 2022 by Salminen, et al. has found that our machine counterparts do better at reading between the lines than us.

They took previous research further by generating 20,000 AI-faked reviews and taking another 20,000 reviews written by humans. Then, they enlisted machine classifiers that were trained to distinguish between computer-generated reviews and original reviews. Finally, the Team Machine results were compared with Team Human to see who was better at calling out fabrications.

(Needless to say, the study is far more comprehensive than this short, simplistic summary. The scope of the research extended to multiple models involving different algorithms. Plotting them against each other, the researchers picked the most efficient winner in terms of AI success and human efficiency at identifying scams.)

The result? Machines with their conditioned classifiers outperformed humans in both recognizing AI-written reviews and “original” (i.e. human) reviews. Looking at the highest-performing model, AI could detect spam reviews with a 96.64% accuracy, whereas the human crowd performed with a meager 55.36% success rate. Once again, our guesses are only slightly above chance level.

Practical Everyday Tips for Spotting a Fake

If the case above has made you feel mentally overwhelmed, you are not alone. Here are some tips to keep in mind when trying to uncover sham reviews and flaky endorsements.

U+27A4 Use “fake checkers” like the ones mentioned above. In due course more of these types of tools should become available to suss out truth from fiction. Especially as combating scam reviews turns increasingly urgent, it’s a fair guess to say such checkers will be more in demand than ever.

U+27A4 Most online marketplaces signal if the reviewer has actually bought the product. Look out for verified purchase badges and similar terms. Note that this isn’t a fool-proof method, either. Some sellers offer “refunds” and financial incentives like gift cards and vouchers in return for 5-star-reviews.

U+27A4 Read the review: observe its tone and context. Does it stand out from others in an almost cringe-worthy degree? Based on the review, does the purchase sound almost too good to be true? Or is it detailing a rival product’s better features instead of focusing on the one at hand?

Even if the phrasing doesn’t sound entirely unnatural, you can pick up on cues that betray the author’s reliability. If the picture being painted is already looking skewed and distorted, you may well be in trickster company.

U+27A4 Know your consumer rights. Which.co.uk has kindly rounded up a useful list covering the most vital points. These include returns policies, delivery rights, as well as getting repairs, replacements, and refunds, amongst else.

Key Takeaways

U+278A The practice of fake reviews can result in a decline of overall faith in even authentic reviews. Further problems include damage to sellers’ reputations, and gaining competitive advantage by means of manipulation.

U+278B Reviews in general can be an extremely useful source of information and honest feedback, identifying shortcomings and necessary improvements.

U+278C Technology is excellent at fabricating scam appraisals, and we are not very good at detecting these.

U+278D Fake review detection models and tools to address these issues come in many flavors of strategies.

U+278E Researchers have come up with an AI model that can spot fellow computer-crafted text with a 96.64% accuracy. At the same task, humans had a poor 55.36% success rate, which is only slightly above chance-level.

U+278F Increase your odds of spotting a scam review by using “fake checkers”, keeping an eye out for badges confirming verified purchases, paying attention to the review’s tone and context, and finally, by being familiar with your consumer rights.

References & Credits

• Cardoso, E. F., Silva, R. M., & Almeida, T. A. (2018). Towards automatic filtering of fake reviews. Neurocomputing, 309, 106–116. https://doi.org/10.1016/j.neucom.2018.04.074
• Harris, C. G. (2019). Comparing Human Computation, Machine, and Hybrid Methods for Detecting Hotel Review Spam. Paper presented at the Digital Transformation for a Sustainable Society in the 2, 75–86. https://doi.org/10.1007/978-3-030-29374-1_7
• Luca, M. (2011). Reviews, Reputation, and Revenue: The Case of Yelp.Com. SSRN Electronic Journal. https://dx.doi.org/10.2139/ssrn.1928601
• Plotkina, D., Munzel, A., & Pallud, J. (2020). Illusions of truth — Experimental insights into human and algorithmic detections of fake online reviews. Journal of Business Research, 109, 511–523. https://doi.org/10.1016/j.jbusres.2018.12.009
• Salminen, J., Kandpal, C., Kamel, A. M., Jung, S., & Jansen, B. J. (2022). Creating and detecting fake reviews of online products. Journal of Retailing and Consumer Services, 64, 102771. https://doi.org/10.1016/j.jretconser.2021.102771
• Six Ways to Repair Declining Social Trust by Kristin M. Lord
• Cancel culture: Trouble for brands or just noise? by Nina Lentini
• Online reviews and endorsements: Report on the CMA’s call for information by CMA
• Guidance on the Consumer Protection from Unfair Trading Regulations 2008 by BERR
• Images by Mohamed Hassan, Pixabay

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI