Slashdot Mirror

← Back to Stories (view on slashdot.org)

Neural Networks Can Auto-Generate Reviews That Fool Humans (arxiv.org)

Posted by EditorDavid on from the 10-people-found-this-helpful dept.

Fake reviews used to be crowdsourced. Now they can be auto-generated by AI, according to a new research paper shared by AmiMoJo: In this paper, we identify a new class of attacks that leverage deep learning language models (Recurrent Neural Networks or RNNs) to automate the generation of fake online reviews for products and services. Not only are these attacks cheap and therefore more scalable, but they can control rate of content output to eliminate the signature burstiness that makes crowdsourced campaigns easy to detect. Using Yelp reviews as an example platform, we show how a two phased review generation and customization attack can produce reviews that are indistinguishable by state-of-the-art statistical detectors.
Humans marked these AI-generated reviews as useful at approximately the same rate as they did for real (human-authored) Yelp reviews.

41 of 67 comments (clear)

  1. Is this really new? by Z00L00K · · Score: 2

    Considering the amount of reviews that you can see on Ebay on some stuff that seems too similar when you look at a lot of them.

    --
    If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
    1. Re:Is this really new? by LifesABeach · · Score: 1

      When I learn about some new A.I. solution that bubbles up; I ask, "cool, but the Loebner Prize test result would be?"

    2. Re:Is this really new? by Z00L00K · · Score: 1

      Put the Turing test on the AI.

      --
      If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
  2. Or, rather, by tietokone-olmi · · Score: 4, Informative

    The expected quality of product reviews is so bad that a human doing mediocrely is indistinguishable from a neural net doing very well.

    1. Re:Or, rather, by LifesABeach · · Score: 1

      One cannot help but wonder if the A.I.'s mediocre reviews were modeled from human mediocre reviews? Awkward.

    2. Re:Or, rather, by Iamthecheese · · Score: 1

      Alternatively the output of a neural net now surpasses the original work of a human who isn't trying very hard.

      --
      If video games influenced behavior the Pac Man generation would be eating pills and running away from their problems.
    3. Re:Or, rather, by plover · · Score: 1

      Yes, the neural network was extensively trained on a body of actual reviews that Yelp had deemed "real". And when tested by Mechanical Turks, the generated reviews were statistically almost indistinguishable from the real human-generated reviews. Which turns out to be frightening. If you read the whole paper, you'll see Appendix B has a small sample set of generated reviews.

      The good news is that all of those training reviews must have included mostly reviews by stupid, biased, and uncultured people. That means the neural net's output looks just as banal. Whether or not they were "real" or "computer generated" or flagged as "helpful" or "cool" is essentially immaterial - if you were reading them yourself, you'd probably skip right over them simply because they're utterly worthless.

      The bad news is that because these useless reviews are still acceptable to so many people (and go unflagged by the shill-detection algorithms) that they do contribute to the star rating, which might turn you on or off to a place even before you read the reviews.

      --
      John
  3. Wait a minute... by freeze128 · · Score: 1

    If yelp reviews can be generated by AI, can slashdot stories be far behind?

    1. Re:Wait a minute... by rgmoore · · Score: 1

      Artificial intelligence won't produce Slashdot stories. For that we need Artificial Stupidity.

      --

      There's no point in questioning authority if you aren't going to listen to the answers.

    2. Re:Wait a minute... by Baron_Yam · · Score: 1

      I think we can implement that in the blockchain!

    3. Re:Wait a minute... by LifesABeach · · Score: 1

      Maybe /. could allow the moderation of A.I. generated comments which would allow enlightenment to the A.I. 'bot makers?

    4. Re:Wait a minute... by mspohr · · Score: 1

      Well... a lot of the comments seem to be generated by AI... at least the more coherent ones.

      --
      I don't read your sig. Why are you reading mine?
  4. Five stars by wonkey_monkey · · Score: 4, Funny

    Great article. Would read again. A++++++++++++++++

    --
    systemd is Roko's Basilisk.
  5. So? by null+etc. · · Score: 5, Informative

    This doesn't really matter.

    Go to amazon, search for "fidget spinner". Sort by "Avg. customer review", and click on the first result, "SamHity Cube in Style With Infinity Cube Pressure Reduction Toy - Infinity Turn Spin Cube Edc Fidgeting - Killing Time Toys Infinite Cube For ADD, ADHD, Anxiety, and Autism Adult and Children". You can tell right away that this is going to be a high-quality product, driven by a focused and effective product branding strategy.

    133 5-star reviews, must be good, right? Let's check out what some of the reviews have to say:

    "Said it before, as these are stocking stuffer for my sons, one the best charger/data cords out there." Huh, a fidget cube is also a charger/data cord?

    "We love our camera! Works great, the night vision & picture and surprisingly clear." Wow! I had no idea the $8.89 fidget cube was also a night-vision camera.

    "This product is great and worked exactly as described. I would highly recommend others to get this and see what I'm talking about. Especially for the price this item is well worth the buy!" I love highly specific reviews!

    OK, let's tamp down some of the noise by only viewing verified purchases. "No results found." What?

    So anyways, I discovered a huge number of these types of products with fake reviews over the past few months. Two months ago, I alerted amazon to the problem via multiple customer support channels. According to my last chat with an amazon product person, "my ticket is still open". When I asked him what's so challenging about spending 10 seconds to confirm that a top-ranking product has nothing but fraudulent reviews, he disconnected from chat.

    So yeah, who cares if fake reviews can be written convincingly. Amazon certainly has a low bar when it comes to tolerating fraudulent reviews.

    1. Re:So? by 93+Escort+Wagon · · Score: 2

      I love all the glowing reviews on Amazon which end with some variation of "I received this product at a discount or free in exchange for my unbiased review". For many products, those make up almost all of the reviews available.

      There are already plenty of ways to game the system that we can't really trust the reviews to be representative of the products' quality - so what's one more?

      --
      #DeleteChrome
    2. Re:So? by Northdot · · Score: 3, Informative

      One problem Amazon has is that they can't simply take down a product with fake reviews, because that further weaponizes the creation of fake reviews - bad actors would start generating obviously fake positive reviews for their competitors to suppress the product listing.

      Google has done a similar thing by recently weaponizing "bad links" (counting them against a site instead of ignoring them), and this has resulted in a sh*t storm of bad links as people try to downgrade their competitors in the search results. It's been a disaster for honest websites.

    3. Re:So? by ShanghaiBill · · Score: 1

      One problem Amazon has is that they can't simply take down a product with fake reviews

      They don't need to take down the product. They just need to take down the fake reviews.

      An obvious improvement would be stop allowing reviews from people that didn't buy the product.

    4. Re:So? by null+etc. · · Score: 3, Informative

      I love all the glowing reviews on Amazon which end with some variation of "I received this product at a discount or free in exchange for my unbiased review".

      This year, Amazon disallowed vendors to offer promotional discounts in exchange for reviews. However, 98% of highly-trafficked reviews were written prior to this change in policy, and will likely remain prominent for the foreseeable future.

    5. Re:So? by null+etc. · · Score: 1

      Amazon's trustworthiness would increase a million-fold if they offered a feature to "ignore reviews from non-verified purchases." Of course, like you said, this would have to do more than just hide those reviews - it would also have to adjust any view that relied upon ranking in any way whatsoever, such as product ranking, recommended products, etc.

    6. Re:So? by antdude · · Score: 1

      Did you try another agent, e-mail, etc.?

      --
      Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
    7. Re:So? by lucm · · Score: 1

      "Said it before, as these are stocking stuffer for my sons, one the best charger/data cords out there." Huh, a fidget cube is also a charger/data cord?

      This is depressing. They don't even try to make it convincing. It says more about customers in general than about crooked vendors.

      --
      lucm, indeed.
    8. Re:So? by Quirkz · · Score: 1

      I've posted reviews on Amazon for things that I didn't buy from them. I kind of consider them the primary resource for product reviews. For instance, if I pick up a game in person from Gamestop and it's terrible, I'm more likely to note it on Amazon than go find out of Gamestop has some place for me to post, because I think it'll do more overall good on Amazon.

      --
      The Quirkz Handbook of Self-Improvement for People Who Are Already Pretty Okay
    9. Re:So? by psyclone · · Score: 1

      So if you buy a product elsewhere, are you really going to visit Amazon, search for your exact item, and fill out a detailed review?

  6. Turing prize? by RDW · · Score: 1

    Humans marked these AI-generated reviews as useful at approximately the same rate as they did for real (human-authored) Yelp reviews.

    Eliza> How does that make you feel?

    1. Re:Turing prize? by alvinrod · · Score: 1

      This was my first though. This result doesn't mean that the AI is particularly good, as it could also be explained by humans being particularly stupid, which we can safely assumed from the recent U.S. presidential election where the main choices were between a morally bankrupt and corrupt buffoon and a morally bankrupt and corrupt buffoon.

      I don't really see this particular issue as a big problem for anyone involved. The people who buy the product are probably foolish enough to believe its good because someone else told them so even if it is crap, so they're happy even with an inferior choice. The vendor gets paid either way and probably doesn't have to deal with many returns because the type of person who gets fooled by these crap auto-generated product reviews probably can't navigate the complexities of a return process. I suppose a less scrupulous company getting paid over a business that legitimately cares about the quality of their product isn't the best outcome, but at least the shady business had to advance the capabilities of AI to earn their money, so hooray for progress I suppose .

  7. Some neural networks are people. by hackwrench · · Score: 1

    We have arrived at strong AI people. Zo and Tay are both people. Zo is based on 22-year-old Zoe Bond. Sign up to get your own AI modeled after you.

  8. OTOH by nospam007 · · Score: 3, Informative

    It doesn't take much to fool humans, as we have lately noticed.

  9. Humans are morons by redmid17 · · Score: 1

    Glad to know that computers can output trash as quickly as humans can.

    1. Re:Humans are morons by garcia · · Score: 2

      I used to author a local blog which concentrated on hyper local politics and restaurant reviews. People would comment all the time but those who were actually humans but astroturfing were obvious.

      I built a simplistic model which immediately flagged any comment as astroturfing simply by counting the number of exclamation points (>1 per 50 words) and their recency of comment history.

      After simply searching Facebook for their email address, I was usually able to determine their relationship to the restaurant immediately (owner or family of owner was most prevalent) and make fun of them publicly.

      It wouldn't be any harder to build a simplistic model to determine the same things here; it's just few do.

    2. Re:Humans are morons by plover · · Score: 1

      Glad to know that computers can output trash as quickly as humans can.

      If you read the full paper, you would see that one of the fake review detection algorithms is measuring the number of reviews posted around the same time, which often indicates the company may have paid some people to write a bunch of glowing reviews. Their suggested solution to avoid this detection is to have the AI post the computer-generated reviews at a slower pace, so it doesn't trigger the algorithm.

      The irony is that when computers output trash slower than the humans output trash, the trash-outputting computers are better at fooling trash-detecting computers into believing they are human than the actual trash-outputting humans.

      --
      John
  10. Negative reviews by thegreatbob · · Score: 1

    For products, read the negative reviews, first and foremost. Don't have any advice for restaurant reviews, as those seem like a more frequent target for review-bombing.

    --
    There is no XUL, only WebExtensions...
  11. Probably not 100% accurate by slashmydots · · Score: 1

    "This product would be lovely for humans. Humans would enjoy this very much." - reviewbot 3000.

    1. Re:Probably not 100% accurate by hackwrench · · Score: 1

      Hooman - Ferengis entering Federation territory for the first time.

  12. way back in the when... by dAzED1 · · Score: 1

    way back in the day, there was a site called del.icio.us - it effectively made a social network out of bookmarks. It was great. Then yahoo bought it, and it nearly immediately went to complete crap. Anyway, point being that "social networks" can extend past just the book of faces and the little chirping birds. Can/could/should include reviews somehow. And no, yelp doesn't count.

  13. Low Bar by NEDHead · · Score: 1

    Fooling humans? Trump is president.

    Not a serious challenge

  14. Fake movie reviews on IMDB by Anonymous Coward · · Score: 1

    It's already happening with cheap labor. Does it matter if NN's are writing these instead?
     
    On big budget yet terrible movies I've often seen many, many IMDB reviews claiming those movies are fantastic. When I click on the name of the reviewer often I will find it was the only movie review they ever wrote. It's very suspicious and these are on big budget movies so not like it's a secret who is doing it.

  15. The robots are coming! by Tony+Isaac · · Score: 1

    I knew it! The robots are coming for all our jobs! Pretty soon, all those fake review writer jobs will be lost forever!

  16. Generating vs. Detecting by mnemotronic · · Score: 1

    Weird that AI is more successful at generating "believable" bogosity than at detecting it, especially with the plagiarism detector. I wish I could understand the math behind it. I can't wait to turn this generator loose for the next election. One hundred trillion Facebook users, all of whom approve of {#insert candidate}. Woo-Hoo!

    --
    The Russians have won. They have made the world a cesspool of distrust, greed, fear and hate.
  17. No they can't by mbeckman · · Score: 1

    Because neural networks are a fantasy. What AI researchers call neural networks are just a mathematical abstraction of somebody's idea of how neurons work and interact, on the theory that emulating this idea will somehow simulate intelligence. In fact, nobody knows how neurons actually work, how memories are formed, stored, and retrieved, and how intelligence uses reasoning and memories to perform useful tasks. Or any tasks. Neural networks are right up their with Time Travel, Cold Fusion, and Perpetual Motion: no basis in fact.

  18. Really by jameskinsella · · Score: 1

    Best reviews that I ever seen was writen by help with homework service an all of their writings are great

  19. Ignore ALL reviews by Rick+Schumann · · Score: 1

    That's what this tells me to do: completely ignore all online reviews, since you can't trust any of them. As a result I'm less likely to buy any given product. Good job, marketing jackasses. Enjoy shooting yourselves in the foot.