Neural Networks Can Auto-Generate Reviews That Fool Humans (arxiv.org)

Fake reviews used to be crowdsourced. Now they can be auto-generated by AI, according to a new research paper shared by AmiMoJo: In this paper, we identify a new class of attacks that leverage deep learning language models (Recurrent Neural Networks or RNNs) to automate the generation of fake online reviews for products and services. Not only are these attacks cheap and therefore more scalable, but they can control rate of content output to eliminate the signature burstiness that makes crowdsourced campaigns easy to detect. Using Yelp reviews as an example platform, we show how a two phased review generation and customization attack can produce reviews that are indistinguishable by state-of-the-art statistical detectors.
Humans marked these AI-generated reviews as useful at approximately the same rate as they did for real (human-authored) Yelp reviews.

Comments

Or, rather,

[tietokone-olmi]

The expected quality of product reviews is so bad that a human doing mediocrely is indistinguishable from a neural net doing very well.

So?

[null etc.]

This doesn't really matter.

Go to amazon, search for "fidget spinner". Sort by "Avg. customer review", and click on the first result, "SamHity Cube in Style With Infinity Cube Pressure Reduction Toy - Infinity Turn Spin Cube Edc Fidgeting - Killing Time Toys Infinite Cube For ADD, ADHD, Anxiety, and Autism Adult and Children". You can tell right away that this is going to be a high-quality product, driven by a focused and effective product branding strategy.

133 5-star reviews, must be good, right? Let's check out what some of the reviews have to say:

"Said it before, as these are stocking stuffer for my sons, one the best charger/data cords out there." Huh, a fidget cube is also a charger/data cord?

"We love our camera! Works great, the night vision & picture and surprisingly clear." Wow! I had no idea the $8.89 fidget cube was also a night-vision camera.

"This product is great and worked exactly as described. I would highly recommend others to get this and see what I'm talking about. Especially for the price this item is well worth the buy!" I love highly specific reviews!

OK, let's tamp down some of the noise by only viewing verified purchases. "No results found." What?

So anyways, I discovered a huge number of these types of products with fake reviews over the past few months. Two months ago, I alerted amazon to the problem via multiple customer support channels. According to my last chat with an amazon product person, "my ticket is still open". When I asked him what's so challenging about spending 10 seconds to confirm that a top-ranking product has nothing but fraudulent reviews, he disconnected from chat.

So yeah, who cares if fake reviews can be written convincingly. Amazon certainly has a low bar when it comes to tolerating fraudulent reviews.

Re:So?

[Northdot]

One problem Amazon has is that they can't simply take down a product with fake reviews, because that further weaponizes the creation of fake reviews - bad actors would start generating obviously fake positive reviews for their competitors to suppress the product listing.

Google has done a similar thing by recently weaponizing "bad links" (counting them against a site instead of ignoring them), and this has resulted in a sh*t storm of bad links as people try to downgrade their competitors in the search results. It's been a disaster for honest websites.

Re:So?

[null etc.]

I love all the glowing reviews on Amazon which end with some variation of "I received this product at a discount or free in exchange for my unbiased review".

This year, Amazon disallowed vendors to offer promotional discounts in exchange for reviews. However, 98% of highly-trafficked reviews were written prior to this change in policy, and will likely remain prominent for the foreseeable future.

OTOH

[nospam007]

It doesn't take much to fool humans, as we have lately noticed.