Neural Networks Can Auto-Generate Reviews That Fool Humans (arxiv.org)

← Back to Stories (view on slashdot.org)

Neural Networks Can Auto-Generate Reviews That Fool Humans (arxiv.org)

Posted by EditorDavid on Sunday September 10, 2017 @05:34AM from the 10-people-found-this-helpful dept.

Fake reviews used to be crowdsourced. Now they can be auto-generated by AI, according to a new research paper shared by AmiMoJo: In this paper, we identify a new class of attacks that leverage deep learning language models (Recurrent Neural Networks or RNNs) to automate the generation of fake online reviews for products and services. Not only are these attacks cheap and therefore more scalable, but they can control rate of content output to eliminate the signature burstiness that makes crowdsourced campaigns easy to detect. Using Yelp reviews as an example platform, we show how a two phased review generation and customization attack can produce reviews that are indistinguishable by state-of-the-art statistical detectors.
Humans marked these AI-generated reviews as useful at approximately the same rate as they did for real (human-authored) Yelp reviews.

67 comments

Min score:

Reason:

Sort:

Is this really new? by Z00L00K · 2017-09-10 05:37 · Score: 2

Considering the amount of reviews that you can see on Ebay on some stuff that seems too similar when you look at a lot of them.

--
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
1. Re:Is this really new? by Anonymous Coward · 2017-09-10 06:14 · Score: 0
  
  Not new.
  100% of Slashdot posts have always been Auto-Generated by Neural Networks.
  Just shows that Neural Networks are not related to "intelligence".
2. Re:Is this really new? by LifesABeach · 2017-09-10 07:20 · Score: 1
  
  When I learn about some new A.I. solution that bubbles up; I ask, "cool, but the Loebner Prize test result would be?"
3. Re:Is this really new? by Z00L00K · 2017-09-10 07:31 · Score: 1
  
  Put the Turing test on the AI.
  
  --
  If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
Great ! by Anonymous Coward · 2017-09-10 05:45 · Score: 0

This is why we can't have nice things.
Or, rather, by tietokone-olmi · 2017-09-10 05:49 · Score: 4, Informative

The expected quality of product reviews is so bad that a human doing mediocrely is indistinguishable from a neural net doing very well.
1. Re:Or, rather, by LifesABeach · 2017-09-10 07:50 · Score: 1
  
  One cannot help but wonder if the A.I.'s mediocre reviews were modeled from human mediocre reviews? Awkward.
2. Re:Or, rather, by Anonymous Coward · 2017-09-10 11:12 · Score: 0
  
  To be fair, if you look for items such as sugar-free gummy bears and uranium ore, the quality of the customer reviews is often extremely high.
3. Re:Or, rather, by Iamthecheese · 2017-09-10 11:31 · Score: 1
  
  Alternatively the output of a neural net now surpasses the original work of a human who isn't trying very hard.
  
  --
  If video games influenced behavior the Pac Man generation would be eating pills and running away from their problems.
4. Re:Or, rather, by plover · 2017-09-10 15:55 · Score: 1
  
  Yes, the neural network was extensively trained on a body of actual reviews that Yelp had deemed "real". And when tested by Mechanical Turks, the generated reviews were statistically almost indistinguishable from the real human-generated reviews. Which turns out to be frightening. If you read the whole paper, you'll see Appendix B has a small sample set of generated reviews.
  The good news is that all of those training reviews must have included mostly reviews by stupid, biased, and uncultured people. That means the neural net's output looks just as banal. Whether or not they were "real" or "computer generated" or flagged as "helpful" or "cool" is essentially immaterial - if you were reading them yourself, you'd probably skip right over them simply because they're utterly worthless.
  The bad news is that because these useless reviews are still acceptable to so many people (and go unflagged by the shill-detection algorithms) that they do contribute to the star rating, which might turn you on or off to a place even before you read the reviews.
  
  --
  John
5. Re: Or, rather, by Anonymous Coward · 2017-09-10 23:10 · Score: 0
  
  Why does anybody use Yelp anyway? It's well known that they extort businesses to influence their reviews, you can't trust anything on there. Hell, if a restaurant has one of those "Yelp hates us" signs, it probably means it's a good place.
6. Re:Or, rather, by Anonymous Coward · 2017-09-12 20:13 · Score: 0
  
  The expected quality of product reviews is so bad that a human doing mediocrely is indistinguishable from a neural net doing very well.
  Agreed.. and in addition... most humans are so fucking dumb that it is no problem to trick them in to whatever you want! Just look at all the iZombies out there!
Wait a minute... by freeze128 · 2017-09-10 05:54 · Score: 1

If yelp reviews can be generated by AI, can slashdot stories be far behind?
1. Re:Wait a minute... by rgmoore · 2017-09-10 06:35 · Score: 1
  
  Artificial intelligence won't produce Slashdot stories. For that we need Artificial Stupidity.
  
  --
  There's no point in questioning authority if you aren't going to listen to the answers.
2. Re:Wait a minute... by Baron_Yam · 2017-09-10 08:31 · Score: 1
  
  I think we can implement that in the blockchain!
3. Re:Wait a minute... by LifesABeach · 2017-09-10 08:51 · Score: 1
  
  Maybe /. could allow the moderation of A.I. generated comments which would allow enlightenment to the A.I. 'bot makers?
4. Re:Wait a minute... by mspohr · 2017-09-10 10:15 · Score: 1
  
  Well... a lot of the comments seem to be generated by AI... at least the more coherent ones.
  
  --
  I don't read your sig. Why are you reading mine?
5. Re:Wait a minute... by Anonymous Coward · 2017-09-10 10:16 · Score: 0
  
  And make sure you use Python and some lambda functions!
Five stars by wonkey_monkey · 2017-09-10 06:00 · Score: 4, Funny

Great article. Would read again. A++++++++++++++++

--
systemd is Roko's Basilisk.
1. Re:Five stars by Anonymous Coward · 2017-09-10 06:21 · Score: 0
  
  Your comment has been Marked as Helpful.
2. Re: Five stars by Anonymous Coward · 2017-09-10 07:12 · Score: 0
  
  Me too!
3. Re:Five stars by Anonymous Coward · 2017-09-11 03:20 · Score: 0
  
  MOOOOO MOOOO COWS GO MOOO artificial INTELLIGENT COWS GO MOOOO MOOO moooooo
So? by null+etc. · 2017-09-10 06:00 · Score: 5, Informative

This doesn't really matter.
Go to amazon, search for "fidget spinner". Sort by "Avg. customer review", and click on the first result, "SamHity Cube in Style With Infinity Cube Pressure Reduction Toy - Infinity Turn Spin Cube Edc Fidgeting - Killing Time Toys Infinite Cube For ADD, ADHD, Anxiety, and Autism Adult and Children". You can tell right away that this is going to be a high-quality product, driven by a focused and effective product branding strategy.
133 5-star reviews, must be good, right? Let's check out what some of the reviews have to say:
"Said it before, as these are stocking stuffer for my sons, one the best charger/data cords out there." Huh, a fidget cube is also a charger/data cord?
"We love our camera! Works great, the night vision & picture and surprisingly clear." Wow! I had no idea the $8.89 fidget cube was also a night-vision camera.
"This product is great and worked exactly as described. I would highly recommend others to get this and see what I'm talking about. Especially for the price this item is well worth the buy!" I love highly specific reviews!
OK, let's tamp down some of the noise by only viewing verified purchases. "No results found." What?
So anyways, I discovered a huge number of these types of products with fake reviews over the past few months. Two months ago, I alerted amazon to the problem via multiple customer support channels. According to my last chat with an amazon product person, "my ticket is still open". When I asked him what's so challenging about spending 10 seconds to confirm that a top-ranking product has nothing but fraudulent reviews, he disconnected from chat.
So yeah, who cares if fake reviews can be written convincingly. Amazon certainly has a low bar when it comes to tolerating fraudulent reviews.
1. Re:So? by 93+Escort+Wagon · 2017-09-10 06:16 · Score: 2
  
  I love all the glowing reviews on Amazon which end with some variation of "I received this product at a discount or free in exchange for my unbiased review". For many products, those make up almost all of the reviews available.
  There are already plenty of ways to game the system that we can't really trust the reviews to be representative of the products' quality - so what's one more?
  
  --
  #DeleteChrome
2. Re:So? by Northdot · 2017-09-10 06:17 · Score: 3, Informative
  
  One problem Amazon has is that they can't simply take down a product with fake reviews, because that further weaponizes the creation of fake reviews - bad actors would start generating obviously fake positive reviews for their competitors to suppress the product listing.
  Google has done a similar thing by recently weaponizing "bad links" (counting them against a site instead of ignoring them), and this has resulted in a sh*t storm of bad links as people try to downgrade their competitors in the search results. It's been a disaster for honest websites.
3. Re:So? by Anonymous Coward · 2017-09-10 06:20 · Score: 0
  
  filtering reviews by "verified purchasers" is a mandatory step when shopping amazon, right up there with filtering out all the bullshit marketplace sellers (which you annoyingly have to do on an increasing number of sites these days)... and even then you still get marketplace listings and prices fucking up your page views, sorting and whatnot.
4. Re:So? by ShanghaiBill · 2017-09-10 06:36 · Score: 1
  
  One problem Amazon has is that they can't simply take down a product with fake reviews
  They don't need to take down the product. They just need to take down the fake reviews.
  An obvious improvement would be stop allowing reviews from people that didn't buy the product.
5. Re:So? by Anonymous Coward · 2017-09-10 06:40 · Score: 0
  
  Maybe those are actual reviews written by customers who are using the product as they think about what to write?
6. Re: So? by Anonymous Coward · 2017-09-10 06:54 · Score: 0
  
  5 stars. Great comment. My son loved this for his birthday, would read again.
7. Re:So? by null+etc. · 2017-09-10 07:49 · Score: 3, Informative
  
  I love all the glowing reviews on Amazon which end with some variation of "I received this product at a discount or free in exchange for my unbiased review".
  
  This year, Amazon disallowed vendors to offer promotional discounts in exchange for reviews. However, 98% of highly-trafficked reviews were written prior to this change in policy, and will likely remain prominent for the foreseeable future.
8. Re:So? by null+etc. · 2017-09-10 07:52 · Score: 1
  
  Amazon's trustworthiness would increase a million-fold if they offered a feature to "ignore reviews from non-verified purchases." Of course, like you said, this would have to do more than just hide those reviews - it would also have to adjust any view that relied upon ranking in any way whatsoever, such as product ranking, recommended products, etc.
9. Re:So? by Anonymous Coward · 2017-09-10 10:29 · Score: 0
  
  ShanghaiBill suggested:
  
  One problem Amazon has is that they can't simply take down a product with fake reviews
  They don't need to take down the product. They just need to take down the fake reviews.
  An obvious improvement would be stop allowing reviews from people that didn't buy the product.
  The problem with that is that it would also eliminate reviews from people who, in fact, did buy the product - but not from Amazon.
  The truth is that Amazon rarely offers the lowest price on the products they sell, at least in part because it rakes off such a high percentage of the sales price for itself by comparison with eBay and other competing outlets. So, for instance, I recently bought a Behringer UltraChorus UC200 foot pedal on eBay for $24.99, shipping included. Amazon's sellers can't and don't match that price, but several of them do sell the same pedal. If I cross-post my eBay review to Amazon, your solution would prevent potential buyers from considering my objective, human-generated, arguably-expert review:
  "A basic, decent-sounding chorus in a plastic housing that has an inconveniently-located 9V jack, a VERY inconvenient battery compartment access design, and which produces an annoying, regular click when it's on, but not receiving signal from the guitar. Can be adjusted to add anything from a faint shimmer to a full-on whooshing, near-phaser effect. The tone control is a potentially useful feature, the output level control less so. You could do worse for more money with a different product - but to do better, you will defintiely have to spend considerably more."
  I'm not convinced that would be particularly helpful.
  (Posting as AC only to keep from undoing previous upmods - including one on the parent).
  --
  Check out my novel ...
10. Re:So? by antdude · 2017-09-10 11:06 · Score: 1
  
  Did you try another agent, e-mail, etc.?
  
  --
  Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
11. Re:So? by lucm · 2017-09-10 14:13 · Score: 1
  
  "Said it before, as these are stocking stuffer for my sons, one the best charger/data cords out there." Huh, a fidget cube is also a charger/data cord?
  This is depressing. They don't even try to make it convincing. It says more about customers in general than about crooked vendors.
  
  --
  lucm, indeed.
12. Re: So? by Anonymous Coward · 2017-09-10 23:13 · Score: 0
  
  True, but the chances of getting a cheap Chinese knockoff that breaks in the first week is much, much, MUCH higher from Fleabay. It happens with Amazon too, but not as often.
13. Re:So? by Quirkz · 2017-09-11 04:45 · Score: 1
  
  I've posted reviews on Amazon for things that I didn't buy from them. I kind of consider them the primary resource for product reviews. For instance, if I pick up a game in person from Gamestop and it's terrible, I'm more likely to note it on Amazon than go find out of Gamestop has some place for me to post, because I think it'll do more overall good on Amazon.
  
  --
  The Quirkz Handbook of Self-Improvement for People Who Are Already Pretty Okay
14. Re:So? by psyclone · 2017-09-11 05:30 · Score: 1
  
  So if you buy a product elsewhere, are you really going to visit Amazon, search for your exact item, and fill out a detailed review?
Turing prize? by RDW · 2017-09-10 06:01 · Score: 1

Humans marked these AI-generated reviews as useful at approximately the same rate as they did for real (human-authored) Yelp reviews.
Eliza> How does that make you feel?
1. Re:Turing prize? by alvinrod · 2017-09-10 06:58 · Score: 1
  
  This was my first though. This result doesn't mean that the AI is particularly good, as it could also be explained by humans being particularly stupid, which we can safely assumed from the recent U.S. presidential election where the main choices were between a morally bankrupt and corrupt buffoon and a morally bankrupt and corrupt buffoon.
  
  I don't really see this particular issue as a big problem for anyone involved. The people who buy the product are probably foolish enough to believe its good because someone else told them so even if it is crap, so they're happy even with an inferior choice. The vendor gets paid either way and probably doesn't have to deal with many returns because the type of person who gets fooled by these crap auto-generated product reviews probably can't navigate the complexities of a return process. I suppose a less scrupulous company getting paid over a business that legitimately cares about the quality of their product isn't the best outcome, but at least the shady business had to advance the capabilities of AI to earn their money, so hooray for progress I suppose .
I wonder if this could be done with an Eliza by Anonymous Coward · 2017-09-10 06:05 · Score: 0

type algorithm?
Some neural networks are people. by hackwrench · 2017-09-10 06:09 · Score: 1

We have arrived at strong AI people. Zo and Tay are both people. Zo is based on 22-year-old Zoe Bond. Sign up to get your own AI modeled after you.
OTOH by nospam007 · 2017-09-10 06:13 · Score: 3, Informative

It doesn't take much to fool humans, as we have lately noticed.
1. Re: OTOH by Anonymous Coward · 2017-09-10 08:34 · Score: 0
  
  As opposed to voting for the orange moron who even has trouble writing a tweet? Covfefe, anyone?
Humans are morons by redmid17 · 2017-09-10 06:13 · Score: 1

Glad to know that computers can output trash as quickly as humans can.
1. Re:Humans are morons by garcia · 2017-09-10 06:24 · Score: 2
  
  I used to author a local blog which concentrated on hyper local politics and restaurant reviews. People would comment all the time but those who were actually humans but astroturfing were obvious.
  I built a simplistic model which immediately flagged any comment as astroturfing simply by counting the number of exclamation points (>1 per 50 words) and their recency of comment history.
  After simply searching Facebook for their email address, I was usually able to determine their relationship to the restaurant immediately (owner or family of owner was most prevalent) and make fun of them publicly.
  It wouldn't be any harder to build a simplistic model to determine the same things here; it's just few do.
2. Re:Humans are morons by plover · 2017-09-10 15:38 · Score: 1
  
  Glad to know that computers can output trash as quickly as humans can.
  If you read the full paper, you would see that one of the fake review detection algorithms is measuring the number of reviews posted around the same time, which often indicates the company may have paid some people to write a bunch of glowing reviews. Their suggested solution to avoid this detection is to have the AI post the computer-generated reviews at a slower pace, so it doesn't trigger the algorithm.
  The irony is that when computers output trash slower than the humans output trash, the trash-outputting computers are better at fooling trash-detecting computers into believing they are human than the actual trash-outputting humans.
  
  --
  John
Negative reviews by thegreatbob · 2017-09-10 06:28 · Score: 1

For products, read the negative reviews, first and foremost. Don't have any advice for restaurant reviews, as those seem like a more frequent target for review-bombing.

--
There is no XUL, only WebExtensions...
Probably not 100% accurate by slashmydots · 2017-09-10 06:38 · Score: 1

"This product would be lovely for humans. Humans would enjoy this very much." - reviewbot 3000.
1. Re:Probably not 100% accurate by hackwrench · 2017-09-10 07:35 · Score: 1
  
  Hooman - Ferengis entering Federation territory for the first time.
So can a short PHP script by Anonymous Coward · 2017-09-10 06:48 · Score: 0

Almost any douche with a few weeks of programming experience could do this.
Rick Rolling the Gibson, Puzzle Palisades edition by Anonymous Coward · 2017-09-10 07:18 · Score: 0

Hacking neural networks to fool and box in data-swamped spy agencies that are pinning their hopes on AI...
https://news.slashdot.org/story/17/09/10/0141207/americas-data-swamped-spy-agencies-pin-their-hopes-on-ai
way back in the when... by dAzED1 · 2017-09-10 07:32 · Score: 1

way back in the day, there was a site called del.icio.us - it effectively made a social network out of bookmarks. It was great. Then yahoo bought it, and it nearly immediately went to complete crap. Anyway, point being that "social networks" can extend past just the book of faces and the little chirping birds. Can/could/should include reviews somehow. And no, yelp doesn't count.
Sort of like Standard & Poor did for... apk by Anonymous Coward · 2017-09-10 08:20 · Score: 0

See subject: Credit Debt Obligations, Mortgage Based Security & Credit Default Swaps (which I cleaned up on)?
* Insanity truly abounds in the world... makes me sick, don't know about you all!
APK
P.S.=> "Who do you trust when EVERYONE's A CROOK" - Queensryche 'Revolution Calling'... apk
1. Re:Sort of like Standard & Poor did for... apk by Anonymous Coward · 2017-09-11 00:11 · Score: 0
  
  More like how a defective chatbot that gets stuck in a loop can replace all of your comments.
Let's be clear by Anonymous Coward · 2017-09-10 08:25 · Score: 0

They can fool some humans, perhaps. An awful lot of peoole can already smell an astro-turfed comment a mile away. I would love to know who these apparently scads of humans with the common sense of a gnat that can't are. The review culture in general a joke, anyone that bases a decision solely on that criteria is bound to get burned eventually.
Low Bar by NEDHead · 2017-09-10 08:29 · Score: 1

Fooling humans? Trump is president.
Not a serious challenge
1. Re:Low Bar by Anonymous Coward · 2017-09-10 10:08 · Score: 0
  
  :( darn, my mod points just disappeared.
  Better AI in bots is probably the biggest single threat to democracy today. Forget applying this tech to reviews. The power play is to equip an AI with Rockwell's personality classifying AI and with technology like this to automatically target voters with comments on their posts tuned to their personality and designed to shift their opinions. This technology could be used to create the "people" that are making those comments too. You start now and build history for a million different "people" with realistically timed posts and likes that match a full range of personalities and localities. Then they'll look more real for the 2018 cycle.
  You can also just automatically build fake news stories written to appeal to small groups of 100s or fewer people and designed to escalate whatever they happen to be discussing in a candidates direction and aim them at the appropriate people.
AmiMoJo by Anonymous Coward · 2017-09-10 09:46 · Score: 0

Is this how all of AmiMoJo's posts get upvoted?
Fake movie reviews on IMDB by Anonymous Coward · 2017-09-10 11:28 · Score: 1

It's already happening with cheap labor. Does it matter if NN's are writing these instead?

On big budget yet terrible movies I've often seen many, many IMDB reviews claiming those movies are fantastic. When I click on the name of the reviewer often I will find it was the only movie review they ever wrote. It's very suspicious and these are on big budget movies so not like it's a secret who is doing it.
The SINGULARITY is nigh by Anonymous Coward · 2017-09-10 14:57 · Score: 0

Computers can now successfully lie to humans.
At the moment it's still humans in control, and using computers to lie to other humans, but as soon as an AI gains control of this ability, it'll be all over for humans.
The robots are coming! by Tony+Isaac · 2017-09-10 15:37 · Score: 1

I knew it! The robots are coming for all our jobs! Pretty soon, all those fake review writer jobs will be lost forever!
Generating vs. Detecting by mnemotronic · 2017-09-10 16:43 · Score: 1

Weird that AI is more successful at generating "believable" bogosity than at detecting it, especially with the plagiarism detector. I wish I could understand the math behind it. I can't wait to turn this generator loose for the next election. One hundred trillion Facebook users, all of whom approve of {#insert candidate}. Woo-Hoo!

--
The Russians have won. They have made the world a cesspool of distrust, greed, fear and hate.
Makes no sense by Anonymous Coward · 2017-09-10 17:39 · Score: 0

"Neural Networks Can Auto-Generate Reviews That Fool Humans". What does this even mean? Anything will fool me if I had no experience with the product/service etc.
No they can't by mbeckman · 2017-09-10 19:18 · Score: 1

Because neural networks are a fantasy. What AI researchers call neural networks are just a mathematical abstraction of somebody's idea of how neurons work and interact, on the theory that emulating this idea will somehow simulate intelligence. In fact, nobody knows how neurons actually work, how memories are formed, stored, and retrieved, and how intelligence uses reasoning and memories to perform useful tasks. Or any tasks. Neural networks are right up their with Time Travel, Cold Fusion, and Perpetual Motion: no basis in fact.
Really by jameskinsella · 2017-09-11 01:39 · Score: 1

Best reviews that I ever seen was writen by help with homework service an all of their writings are great
Ignore ALL reviews by Rick+Schumann · 2017-09-11 04:15 · Score: 1

That's what this tells me to do: completely ignore all online reviews, since you can't trust any of them. As a result I'm less likely to buy any given product. Good job, marketing jackasses. Enjoy shooting yourselves in the foot.