Slashdot Mirror

← Back to Stories (view on slashdot.org)

People Are Complete Suckers For Online Reviews (nypost.com)

Posted by EditorDavid on from the reading-the-reviews dept.

schwit1 shared an article from the New York Post: No reviews, no revenue. That's the key takeaway from a new study published in Psychological Science, which finds that if two similar products have the same rating, online shoppers will buy the one with more reviews... "[When] faced with a choice between two low-scoring products, one with many reviews and one with few, the statistics say we should actually go for the product with few reviews, since there's more of a chance it's not really so bad," wrote researcher Derek Powell of Stanford University, lead author of the report. In other words, when there's only a handful of reviews, a few bad ones break the curve and bring down the overall rating. "But participants in our studies did just the opposite: They went for the more popular product, despite the fact that they should've been even more certain it was of low quality," he wrote.

Matt Moog, CEO of PowerReviews, previously conducted a study with Northwestern University [PDF] that drew from an even larger data pool of 400 million consumers, which also found that the more reviews there are of a product, the more likely it is that a customer will purchase that product... He has also found that customers who read reviews often click the bad ones first. "They want to read what's the worst thing people have to say about this," he said... Most online shoppers (97 percent to be exact) say reviews influence their buying decisions, according to Fan & Fuel Digital Marketing Group, which also found that 92 percent of consumers will hesitate to buy something if it has no customer reviews at all.

32 of 162 comments (clear)

  1. More Complex by zieroh · · Score: 5, Insightful

    I think it's a bit more complex than that. Oftentimes, you can determine from poor reviews exactly what the shortcomings are, and decide if those shortcomings affect your intended use of the product. If a competing product has no reviews, then you have no way of knowing what the shortcomings are.

    --
    People who say "sheeple" have about as much sophistication as an AOL user, and in fact are probably actually AOL users.
    1. Re:More Complex by TWX · · Score: 5, Insightful

      Correct. What went wrong for other people? Why did it go wrong? Will this function impact me?

      As a case in point it's not really possible to buy an inexpensive TV that lacks "smart" features anymore. Smart features are very inconsistent from manufacturer to manufacturer. Thing is, if I never connect my TV to anything but an antenna, do those smart features matter? I need to read other users' experiences with a given TV or a range of candidate TVs to see what features work and what don't, and just because a TV gets poor reviews doesn't mean that the functions that I'll use are the ones poorly reviewed. It could be that the Internet connectivity stuff is what's garbage, or like the one I actually bought, the stock remote sucks but if I get a $10 remote from the previous model TV, I get 80% of the features back that normally require a cell phone with Bluetooth or other "smart" feature to work.

      --
      Do not look into laser with remaining eye.
    2. Re:More Complex by Mr+D+from+63 · · Score: 5, Insightful

      I think it's a bit more complex than that. Oftentimes, you can determine from poor reviews exactly what the shortcomings are, and decide if those shortcomings affect your intended use of the product. If a competing product has no reviews, then you have no way of knowing what the shortcomings are.

      I read reviews, but only really pay attention to ones that make sensible explanations of what they do or don't like. Positive reviews can be very helpful if you are looking for a certain functionality, but they have to be precise and not cheerleading. A lot of reviews often means there are more good ones to find, more useless ones as well. For the most part, when I read reviews and then purchase, I wind up getting pretty much what I expect. You can never eliminate the possibility of a surprise when buying products you've never tried before.

    3. Re:More Complex by Anonymous Coward · · Score: 5, Insightful

      Correct. What went wrong for other people? Why did it go wrong? Will this function impact me?

      It's striking me as somewhat ironic that a couple stanford researchers appear to have missed the significance of having thorough documentation.

      Because that's basically what you're getting with masses of reviews readily available.

    4. Re:More Complex by Wycliffe · · Score: 4, Interesting

      If a competing product has no reviews, then you have no way of knowing what the shortcomings are.

      Not only this, but I'm always suspicious of a product with very few reviews. If people are actually buying the product then there should be some reviews. If there are no reviews, I start to suspect it to be a fraudulent listing.

    5. Re:More Complex by omnichad · · Score: 4, Informative

      Negative reviews by morons actually tend to be extremely helpful. You know exactly why it was rated poorly (they didn't read directions / too dumb to use it). I find this especially true for restaurant reviews, where they describe something "wrong" with the food that really means that the food was made correctly and they've just never had a good version of that food.

  2. Obvious by burtosis · · Score: 5, Insightful

    If one product has 12 reviews and one 45k reviews, the 12 review has 5* and the 45k review has 4* I'd be more likely to buy the 45k review one. It's simply significantly harder to astroturf and bot 45k reviews than 12. Online typically all you have to go on is a shitty picture, product details that are nearly always incomplete and exaggerated, and reviews. The reviews are the least shitty way to not get ripped off. For the record, yes I know that many reviews on most every site are fakes, paid for the review, or bots.

    Plus sometimes the reviews are pure comedy gold and are worth reading on their own which can inspire some sales on their own.

    1. Re:Obvious by tomhath · · Score: 3, Informative
      TFA noted the same thing:

      The exception to that rule is if every one of the reviews is giving this place or product five stars. “If the rating is unusually high, that actually can have a negative impact,” said Moog, as shoppers suspect this is too good to be true. “What we have from our data is that the optimal rating is about 4.4 stars.”

    2. Re:Obvious by _Sharp'r_ · · Score: 4, Insightful

      Yeah, the study appears to naively assume all online products, sellers and reviews are completely legitimate and that reviews solely indicate a statistical level of quality and aren't influenced by other factors.

      Any product with tons of verified reviews means at least that the product has survived and sold enough to gather those reviews. That's an endorsement it's very difficult to fake.

      Now if they actually purchased and tested products themselves to determine which was better as part of the study, they might have a decent conclusion, but as all they did was make a statistical assumption then go judge people's rational behavior against their assumption, the study's conclusion is way off. They're trying to make a case about statistical uncertainty and they refuse to believe results based on people's actual experience with purchasing products online.

      This is a signal to noise issue. People are ignoring other factors and trusting lots of reviews because they're searching for the signal within the influenced-by-seller noise.

      --
      The party of stupid and the party of evil get together and do something both stupid and evil, then call it bipartisan.
    3. Re:Obvious by OzPeter · · Score: 3, Insightful

      If one product has 12 reviews and one 45k reviews, the 12 review has 5* and the 45k review has 4* I'd be more likely to buy the 45k review one. It's simply significantly harder to astroturf and bot 45k reviews than 12.

      Personally I look for reviews in the middle of the pack to get a good sense of what the product is actually like. I see a lot of 5s that appear to be people gushing about a product and a lot of 1s where it seems to be all about trashing a product. I feel that the 2s, 3s and 4s give a more balanced perspective of a product.

      --
      I am Slashdot. Are you Slashdot as well?
    4. Re:Obvious by 93+Escort+Wagon · · Score: 3, Interesting

      I am not sure that paragraph actually indicates that the author understands astroturfing - he seems to still be talking about the reader's state of mind.

      Also, I'd argue the author's understanding of statistics is flawed, since he apparently thinks a bell curve only has one side.

      --
      #DeleteChrome
    5. Re:Obvious by Kjella · · Score: 2

      Any product with tons of verified reviews means at least that the product has survived and sold enough to gather those reviews. That's an endorsement it's very difficult to fake.

      Exactly. Crap gets bad reviews, few sales and the product is discontinued. If a poorly rated product continues to sell it's probably low quality for a low price and the complaints basically boil down to expecting more than you'll reasonably get at that price point or it's a niche product rated poorly by the mainstream because they don't understand who's the target market. Basically it might not be a terrible product for you, if your requirements are low or you happen to fit the niche. A new product that's rated as crap is probably just crap for everyone, they usually start with cheerleader reviews by fans and friends if not outright fakes or paid endorsements so if the crap reviews have already trashed the rating it's a very bad sign. The more reviews, the more likely it's to be near the true rating.

      P.S. These guys should really do a study on IMDB movie scores... they might learn a thing or two about the real world.

      --
      Live today, because you never know what tomorrow brings
  3. More reviews mean more data points by Vermonter · · Score: 5, Informative

    If something has 1000 reviews, I know that the bulk of those are likely to be legitimate. If someone has 3 reviews, I have no idea how accurate the reviews are. More reviews = more sample data of user experience, and more data means that "wrong" reviews (reviews that don't reflect the user experience) are obscured. If you were presented with 2 studies where one used a sample of 5 people and the other used a sample of 5000 people, which results would you trust more? Reviews are just a less controlled study.

  4. Re:"...often click the bad ones first.." by Anonymous Coward · · Score: 5, Insightful

    I always check the bad reviews and only the bad reviews. Good reviews generally aren't helpful, they're either fake and posted by the seller or genuine but not useful. Seeing twenty 5 star reviews that say "it worked great for me" is great, but what I really want to know is how it failed for people and if I care about those failures.

    If all the bad reviews are by people who are clearly crazy or doing something stupid, I can be fairly confident in the product. If they instead reveal significant flaws, I may want to reconsider.

    As always, XKCD has a relevant cartoon about this.

  5. Worst restaurant on yelp! by Anonymous Coward · · Score: 4, Funny

    Yelp tried to extort them a few times, so this restaurant gives 50% off on pizza if you give them a 1-star review:

    https://www.yelp.com/biz/botto...

    http://www.bottobistro.com/REW...

    http://insidescoopsf.sfgate.co...

    http://time.com/money/3398188/...

  6. Re:As a producer, I thank /. reviewers here... apk by arth1 · · Score: 2, Informative

    [usual spam]
    APK

    That's a very good and pertinent example of what TFA is about, and why people skip good reviews. Either they are sockpuppets, or empty praise telling us nothing of value.
    It's the bad reviews that put meat on the bone. A review saying "it has problems with X, but here's my workaround" is worth a thousand reviews saying "It's the best thing since sliced bread!". Counter-intuitive as it may seem, constructive bad reviews facilitate more sales, and empty praise doesn't.

  7. Lies, Damn Lies and Statistics by Spazmania · · Score: 4, Insightful

    The three main kinds of mistruth.

    With more reviews, the buyer has a better idea -exactly- what's bad about the product thus has a better chance of making an educated decision about buying it.

    The article and study are examples of misusing statistics. The correlation between number of reviews and purchases niether tests nor demonstrates a causal relationship from the former to the latter, and even if it did it does not demonstrate the -claimed- causal relatinoship.

    --
    Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
    1. Re:Lies, Damn Lies and Statistics by war4peace · · Score: 5, Insightful

      Exactly. Furthermore there is a problem with the methodology:

      This bias was so strong that they often favored the more-reviewed phone case even when both of the options had low ratings, effectively choosing the product that was, in statistical terms, more likely to be low quality.

      That is incorrect.
      Say you have product A and product B, each with a score of 3 stars out of 5.
      Product A has 6 reviews, 3 of which are 1-star and the other 3 are 5-star - the average is obviously 3.
      Product A has 600 reviews, 3 of which are 1-star, 594 reviews are 3-star and the other 3 are 5-star - the average is obviously 3, again.

      Based on that information alone, you can NOT determine which product is more likely to be bad. You could however determine which product is more popular - but that's it.

      Furthermore, if both products have a low score, it is actually better to go for the more popular one simply because more people bought it therefore more troubleshooting, more info about how to overcome its shortcomings and more workarounds would be available online for that product. Shortly put, it would be more likely to find someone else who had a similar problem with the product and posted a solution online.

      --
      ...gis sdrawkcab (usually not responding to ACs; don't bother posting as AC)
    2. Re:Lies, Damn Lies and Statistics by clovis · · Score: 5, Funny

      And I suppose it's my turn to be the guy that posts the related xkcd :
      https://www.xkcd.com/937/

    3. Re:Lies, Damn Lies and Statistics by ShanghaiBill · · Score: 4, Insightful

      Also, the fewer reviews, the more likely those reviews are FAKE. Most of the statistical claims made in TFA are based on the implicit assumption that all the reviews are equally valid. So the real problem here is not dumb customers but dumb researchers.

    4. Re:Lies, Damn Lies and Statistics by Anonymous Coward · · Score: 2, Insightful

      It's more important to see latest reviews. Often scumbag manufacturers will replace a good product with a cheaper shittier version and ride the popularity due to the early reviews to the bank. That's why well established products should be really scrutinized.

    5. Re:Lies, Damn Lies and Statistics by Anonymous Coward · · Score: 2, Informative

      Not to mention all the reviews that are complete garbage. I remember a few years ago and I saw a bunch of 1 star reviews for a product, curious I checked and almost all of them were not only complaints against the shipper, but they were all along the lines of "product works perfectly fine but shipping took too long due to that hurricane. 1 out of 5 stars." That's right, they dinged the product (which actually had many sellers) because of shipping delays due to Superstorm Sandy.

    6. Re:Lies, Damn Lies and Statistics by Actually,+I+do+RTFA · · Score: 2

      You're supposed to post the multiple related xkcds. This one, especially the alt-text is also pertinent.

      --
      Your ad here. Ask me how!
    7. Re:Lies, Damn Lies and Statistics by David_Hart · · Score: 3, Insightful

      Also, the fewer reviews, the more likely those reviews are FAKE. Most of the statistical claims made in TFA are based on the implicit assumption that all the reviews are equally valid. So the real problem here is not dumb customers but dumb researchers.

      I'm willing to bet that it also doesn't factor in version/software changes, the fact that many reviews includes multiple models, and for every detailed excellent review, there are many stupid reviews (they ordered the wrong thing in the first place, do not know how to use it, had a problem with customer service, etc. stuff unrelated to how the product works).

    8. Re:Lies, Damn Lies and Statistics by Carewolf · · Score: 2

      Knowing almost all places have a few fake good reviews from friends and family, the place with more reviews is more likely to be genuine.

    9. Re:Lies, Damn Lies and Statistics by Anubis+IV · · Score: 2

      Say you have product A and product B, each with a score of 3 stars out of 5.
      Product A has 6 reviews, 3 of which are 1-star and the other 3 are 5-star - the average is obviously 3.
      Product B has 600 reviews, 3 of which are 1-star, 594 reviews are 3-star and the other 3 are 5-star - the average is obviously 3, again.

      Based on that information alone, you can NOT determine which product is more likely to be bad.

      Sure you can, since both you and the researchers have—I believe, wrongly—assumed that good products are just as likely as bad products, which is fine in theory, but doesn't hold up in the real world any better than a physicist's example that starts with "consider a perfect sphere in a frictionless vacuum". In the real world, bad products outnumber good products by a wide margin, and we're well-served in relying on past experience to shape our future decisions.

      With 600 reviews establishing a 3-star rating, we can be fairly certain that what we're getting with Product B is a 3-star product. With 6 wildly varying reviews for Product A, however, we can't pull any useful information from the reviews. That said, we know from past experience that any given product is more likely to be bad than good. As such, given the dearth of information to the contrary, we'd be well-served in assuming that Product A is likely to be a bad product, suggesting then that Product B is the better choice.

      Further, consider that good products tend to attract numerous reviews as they succeed and that, statistically speaking, it's unlikely you'll be among the first 6 (i.e. the first 1%) to review a product that will eventually go on to attract 600 reviews. In contrast, bad products tend to fade into obscurity before ever attracting many reviews. As such, for any given product with just 6 reviews, it's more likely that it's a bad product on its way to obscurity than a good product on its way to 600 reviews. Again, this would point towards B being the better choice.

  8. I check reviews for... by Anonymous Coward · · Score: 2, Insightful

    ...suggestions for competitors products. I then make my own determination after checking them out.

    1. Re:I check reviews for... by TheRaven64 · · Score: 4, Insightful

      Mod this up. Most reviews are useless when comparison shopping because the people writing them have a sample size of one. I recently bought a new electric shaver. Most of the reviews are from people who have owned maybe one other shaver in their life. For a product with 600 reviews, 500 of them are left within a few hours of the new one arriving, so all that you really know from them is that it came in a box and didn't break in the first use. The only useful reviews were the ones where someone actually compared it to others that are still available.

      --
      I am TheRaven on Soylent News
  9. Nothing New by boudie2 · · Score: 5, Interesting

    At one point in the 1970s it was my misfortune to be employed as a Filter Queen vacuum cleaner salesman (regrets, I've had a few). Anyways at the time I was taught that some people "have to be told to buy" something, even if it's from a perfect stranger. They can't make the decision on their own as most people are told what to do their whole lives. That and "Advertising is the best way to sell something, especially if it's no fucking good".

  10. Not all bad reviews are "bad" by petes_PoV · · Score: 2
    Many times I have seen 1 or 2 star reviews that complain about other aspects of the order. Packages arriving late, damage, being cancelled.

    These might not be the proper place to complain about non-product issues, but they happen and they drag down the overall rating of the product, itself. Who knows, maybe some bad reviews are hoping for an offer from the maker to improve their ratings.

    However, I do pay more attention to the bad reviews and the reasons given. If there is a pattern of failures, then I'll avoid a product. And I pay them more attention than I do to the good reviews, which even now often appear to be fake, exaggerated, written by idiots ("I've just received such'n'such, it looks wonderful though I haven't tried to use it yet - here's 5 *'s") or clearly from professional reviewers.

    --
    politicians are like babies' nappies: they should both be changed regularly and for the same reasons
  11. 1* reviews serve several purposes by The+Davii · · Score: 2

    1) I read the bad reviews first to see how many of the bad reviews are idiots. For example, I purchased a product that adapted VGA to video. There was a switch on the side of the box for NTSC/PAL. A number of N. American consumers indicated the box didn't work and that the picture was monochrome (black & white). Well, those folks obviously didn't flip the switch from PAL (as shipped) to NTSC because monochrome is a symptom of video format mismatch (simplying a bit to illustrate the point). So, I discount the 1* reviews with that reasoning that tells me they used the product incorrectly. 2) I look for the proportion of 5* and 1* reviews. If they're about equal, that's a danger sign. If they trail off in a nice log power type curve, so much the better. Those who mention IMDB being seeded with a lot of 10% reviews, beware because the studios know how much word of mouth matters today so they plant reviews that make Baywatch sound like The Godfather. Any "reviewer" who only gives 1 or 10* reviews is suspect in my book. 3) I look for reviews that closely match my use case. 4) I look for reviews that are clearly fake. 5) Any review that is 1* due to price I completely ignore (prices fluctuate and my price point may be very different than yours. Review the actual product, not the price. *I* decide whether the price is worth it. Unfortunately, Google Play's app store doesn't allow one to look at the negative reviews only. Very annoying. Maybe Google should look at this research and understand that low * reviews serve a purpose.

  12. Not just reviews by reboot246 · · Score: 2

    I use any resources I can find before I buy something that costs more than $50. Read reviews of the product on several different retailers' sites. See if any of them are written exactly the same. I've seen the exact same review on three different sites for the same product. I hope the guy got paid for writing them.

    If there's a link to a pdf of the manual, it's good to read the manual. A manual gives you some idea about how hard or easy the product is to install, put together, and use. It gives you a much more detailed description than you can find on a retailer's website. Check out the "troubleshooting" part to see what kind of problems you can expect and how easy they are to fix.

    If the retail site doesn't link to the manual, go to the manufacturer's site to find one. There's usually one there.

    If all I'm buying is lentils from Amazon, I just go by name brand and reputation. Reviews don't even come into play for a lot of items.