Slashdot Mirror

← Back to Stories (view on slashdot.org)

Amazon Overhauling Customer Reviews

Posted by Soulskill on from the shortening-your-soapbox dept.

An anonymous reader writes: Amazon says it's making some big changes to its product review system, one of the most heavily used on the internet and a vital part of Amazon's business. A machine-learning platform will endeavor to select helpful reviews with an emphasis on more recent ones. The average score will change as well: new reviews will be weighted higher than old reviews. Reviews from verified purchasers will have more influence as well, and also reviews voted up by other customers. "For example, sometimes a company will make small tweaks to a product or address some customer complaints, though this product isn't officially updated or renamed. With the new system, [Amazon] said, these small modifications should become more noticeable when shoppers are buying products." Because the review system is so important to customers, Amazon will be rolling out changes slowly, and watching for anything that breaks or gets skewed in unexpected ways.

15 of 116 comments (clear)

  1. Sounds like reasonable changes to me by FlyHelicopters · · Score: 5, Insightful

    I could be missing something, but frankly everything I read in the summery seems like reasonable changes to me.

    Someone who actually is known to have purchased the item, yea, their review should be worth more than random Internet person #4827341

    A review from last month is probably worth more than one from two years ago. The product may have changed.

    1. Re:Sounds like reasonable changes to me by Dutch+Gun · · Score: 5, Insightful

      A review by someone who chose a different product is likely to be more valuable in choosing a product, assuming that person can articulate why he/she chose the other product, because that means the person knows not only this product, but also other products on the market. By contrast, someone who chose the product he/she is reviewing has a very high probability of being familiar with only that product and not any others on the market. Given a choice, I'd take reviews from non-owners over reviews by owners any day.

      That's some pretty convoluted logic there, at least by my reckoning. If the user hasn't purchased the item in question, how exactly are you assuming he/she knows the product sufficiently that they're in a suitable position to review it, judging it's strengths and weaknesses? Can you assume they're not simply biased against a completing product they happen to own, for example? You're telling me that, for example, a Playstation 4 owner is in a better position to review an Xbox one, and we should trust their judgment as being more objective and fair than someone who actually purchased an Xbox one?

      Also, if the product sucks, assuming the product isn't so bad that folks return it, people who own the product are more likely to feel the need to give it better reviews to justify the money they spent.

      Wow... again. I'd bet that people who have purchased a product and are unhappy with it are actually *more* likely to review it harshly in an effort to punish the company for their poor product, and at least warn others against a crappy purchase. There are some old marketing saws that say similar things, I believe. At the very least, that holds true for me. I've purchased a couple of stinkers, and I made damn sure to leave a one or two star review, and explain in detail *why* it was such a terrible product.

      Heh, your expectations of human nature runs about 180 degrees opposite of mine, for whatever reason.

      --
      Irony: Agile development has too much intertia to be abandoned now.
    2. Re:Sounds like reasonable changes to me by vikingpower · · Score: 4, Insightful

      Disagree in the case of books. An old review is absolutely not worth less than a recent review, especially if e.g. a classical engineering or math text is 2 decades old.

      --
      Religous speak to God. Insane are spoken to by God. When all shut up, one can finally hear Shostakovich in peace
  2. Stop grouping revisions by magusxxx · · Score: 5, Interesting

    The other problem is that they'll group different models on the same page. So the reviews you see are for all the models together. In many cases Version B is way better than Version A. But, you still see the bad reviews without realizing it's unrelated to what you're going to buy. They also need to address the paid bad reviewers. I looked up stuff just last week and the same one star review, word for word, was listed on three different items. And these were three totally unrelated items. One in electronics, one in cookware, and the third in camping equipment.

    --
    Care killed the cat, but satisfaction brought it back.
  3. Amazon Vine by Smerta · · Score: 3, Interesting

    I'd be interested to know if the data-crunchers at Amazon have looked at the Amazon Vine reviews, as a group, to see if they are slanted positive.

    Amazon Vine is the program where a certain select demographic of Amazon customers receive free stuff (including items such as 60" TVs, laptops, etc.), with the understanding that they will objectively review the product and post the review on Amazon. My experience is that almost every Amazon Vine review is 4-5 stars. I'd also be curious to see if Amazon looks at the spread of reviews from Vine reviewers -- by that I mean, "Do reviewers in the Vine program rate free Vine products higher than other products they've bought?"

    The implication being that Vine reviews (many of them) probably feel that a good review of a product that Amazon wants to sell is "quid pro quo". I strongly suspect that Amazon wants exactly the opposite of quid pro quo; they want early Vine reviews to weed out marginal or bad products.

  4. Re:Another company trashes its most useful product by magusxxx · · Score: 5, Interesting

    Well, yes and no... http://www.amazon.com/gp/custo... ... This person's one star review was posted three years ago and has garnered 76 responses. The most current was 20 days ago. It's amazing seeing the "You're right" - "No, you're wrong" conversation. Even when the company, right from the beginning, stated, "This is how your testing methods are faulty...", people are still saying this one star review has convinced them not to buy the product. [BTW: 88% of 4,121 people gave this product a 4-5 star review. ] So, while this review has survived the test of time, the only usefulness it has achieved is to show how bad testing methods are readily accepted if it's buried in enough data.

    --
    Care killed the cat, but satisfaction brought it back.
  5. The worst reviews on Amazon by MillionthMonkey · · Score: 5, Insightful

    "I have to give this book one star because I ordered it and it never arrived on time even though Amazon said it left the facility six days before it was supposed to get here!"

    "This book is typical LIBTARD crap and if you buy it you're a stupid egghead."

    "I haven't read a book in five years so when this book came out I decided to buy it. This isn't the book I thought I was ordering, this is crap written by a different guy with a similar name! Buyer beware!"

    Is it really that hard to get a computer to pick these out?

  6. Re:Caveat Emptor by Anonymous Coward · · Score: 5, Insightful

    You must have missed the bit that says: "Reviews from verified purchasers will have more influence as well".

    Similarly, if they're applying machine learning to the whole system, they could fairly quickly pick out people who seem to do nothing but review items on Amazon, or people who create accounts and do nothing but review items, etc.

    Gaming the system will still happen, but it'll be harder.

  7. Transcript of a recent meeting at Dice HQ by Anonymous Coward · · Score: 3, Insightful

    Boss: What the hell? Slashdot's revenue dropped again? What happened?

    Middle management #1: That is strange, our beta design was supposed to increase traffic.

    Middle management #2: Yeah strange right? We spent weeks making sure the beta was difficult to use as hell, then we shove it down the user's throat, how could our traffic tank after that? HOW?

    Middle management #3: It can't be our fault, my 3 years old son was playing with beta before the launch and he absolutely loved it, he just learnt how to use a mouse and he was clicking around rapidly, he was so excited by the design he even clicked on the ads, if everyone did that our views and revenue should have tripled by now.

    Boss: Well we got to do something, any ideas?

    Middle management #1: Hmmm... well I heard there is something call 'social media', I haven't looked at it yet but it looks like people love sharing things on it, maybe we can use that?

    Middle management #2: Yeah I heard about that too, my daughter said she uses it to share elmo photos.

    Middle management #3: Oh I got an idea! Let's put a bunch of social media share link on the site!

    Middle management #1: Sounds good to me, but if everyone is already doing it we need to do something a little different.

    Middle management #2: How about... Oh I know, let's remove the most useful and popular 'read more' link, and replace it with a bunch of share links. I swear the users are so fucking stupid they won't be able to tell the difference.

    Middle management #3: Yeah! Those geeks, they don't know much about computers, they are just going to click on the same place over and over again, and come back for more!

    Boss: Geek site for retards? That is fucking brilliant! Let's do it!

    1. Re:Transcript of a recent meeting at Dice HQ by AmiMoJo · · Score: 5, Informative

      Yup. Seriously, was putting the share button where the "read more..." link used to be designed to make people accidentally click on it? Do you guys do any kind of user testing at all, or were you fully aware that's the most clicked on link on the site and just wanted to abuse it?

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    2. Re:Transcript of a recent meeting at Dice HQ by kimvette · · Score: 3, Interesting

      Indeed; when /. readers want to share something we generally know how to copy & paste. Now, there are always exceptions to the rule but they are in the vast majority.

      Besides, /. already HAD the share feature; it was just horribly broken. Fix share, put it back where it was, and bring back the Read More link.

      Now, I realize that slashdotters are your product, not your customers, but by not listening to us you're pushing us to continue to choose reddit or even (ugh!) fark over /., thereby reducing the value of product making /. less attractive to your actual customers (advertisers).

      Hey I have an idea - why not take this whole trend one step further and start posting stories about the kardashians, other train wrecks like Lindsay Lohan, Britney Spears, and other child stars who delved into hard drugs or alcohol and ruined their lives, thereby increasing readership, and the all-important ad clicks?

      --
      The Christian Right is Neither (Christian nor right). See: Matthew 23, Matthew 25, Ezekiel 16:48-50
    3. Re:Transcript of a recent meeting at Dice HQ by 93+Escort+Wagon · · Score: 3, Insightful

      Seriously, was putting the share button where the "read more..." link used to be designed to make people accidentally click on it?

      I am reasonably certain you are exactly correct.

      --
      #DeleteChrome
  8. Good for the consumer? by hsa · · Score: 3, Insightful

    Yeah, right. Amazon is adjusting the average score from the average score to some propietary algorithm. Yeah, there are talks about tweaks and the marketing makes it sound totally reasonable.

    Amazon sells products. Amazon likes to sell products. Products with higher rating sell better. Products with poor rating sell worse. Amazon would like to sell more products.

    I can bet $1000 right now that the "average rating" is going to go up.

  9. May finally be time to abandon ship... by bradley13 · · Score: 3, Informative

    As Dice transforms /. into yet another random site (video section, meaningless pictures attached to articles, and now social media buttons), it become clear that I am no longer part of their target audience. I hate those crappy social media share buttons - they're nothing but trackers in another form.

    Hello Soylent

    --
    Enjoy life! This is not a dress rehearsal.
  10. good step, but... by The+Grim+Reefer · · Score: 3, Insightful

    I wish they would find a way to separate reviews of different formats. It gets annoying trying to determine which edition of a classic movie is being reviewed on Amazon. They lump VHS, DVD, & Bluray along with all editions together. It can be interesting reading someone's opinion on classic movies, but I'm more interested in format/edition information about Citizen Kane than what someone thinks about it. Some movies have had multiple Bluray releases, and some are considerably better than others. The newest is not always the best.

    This can also be the case with older music. There was a studio back in the late 1960's/early 1970's that did not have the recording speed of their equipment calibrated properly. It actually recorded at a slower rate than it should have. Some remastered versions took this into account, but several newer ones did not. This is a case where edition specific reviews are very important to me.