Slashdot Mirror
Should You Trust Website Customer Reviews?
crevette asks: "I was looking on Amazon for some gizmo when I got a review from G. Cooke, TX, who is in the top 10 reviewers. Out of curiosity, I checked her reviews... She has 658 reviews, many on the same day, which include everything from knife sets to a plastic duck! She reviews many books on the same day... She must be spending hundreds of dollars on useless stuff every month. Worst of all, most of her reviews are 5 stars. Do you think those people are paid by Amazon or some company? Do you trust them? If not (like I tend to think) what can we do about it?"
go to Google's new beta product search to find cheap reviewers!
fp
icblf
It's a public website; anyone can post a review. Trust it as much as you do Slashdot. :)
I read a bunch of Cooke's reviews. She has nothing bad to say, nothing critical, and all her reviews read like an ad. Yeah, I'd say something's wrong here.
and so should you.
I don't need no instructions to know how to rock!!!!
Most of the reviews that I see are either very positive or very negative. There are very few reviews that label a product as simply okay or adequate.
This is probably due to the fact that only people that have had extremely positive or extremely negative experiences with a particular product will actually take the time to post. Well, other than a few people with too much time on their hands that want to be listed as top reviewers.
.sig wanted. Inquire within.
The point I'm making is that the reviewers aren't always participating in a community, but also acting out a personal agenda as well, which other customers may take in as fact.
How about ignoring the reviews if you don't trust them? This isn't difficult.
Might explain why she reads so much....
Name: G. Cooke
Nickname: pagebypage
E-mail: grospoin@aol.com
Reviewer Rank: 8
About me: So many good books, so many wonderful things....so few hours. For me, reading and trying is sheer pleasure, a boon companion, and sometimes work as reviewing is my profession. I hold membership in the National Book Critics Circle, and write for newspapers across the U.S. Thanks to all who have expressed an interest in my reviews, and thanks to Amazon for providing a forum in which we can exchange ideas.
Figuring out her Email address, then Googling it reveals Ms. Cooke probably does nothing but writes reviews allllllllll day long.
Man, what a horrible way to waste time. Well, back to reloading Slashdot....
sig--we don't need no goddamn sig
This person's writing style is descriptive and consise. Far too good for a "normal" user.
This says to me: Someone is being paid to write good reviews for any product out there. A mini-Marketing agent if you will.
It's somewhat reminiscent of the Microsoft PR agent who "switched from Mac to Windows", or it's like the "grassroots pro-Microsoft" campaign that Bill sponsored in the opening days of the Anti-Trust court proceedings.
Trying to appear unsolicited and innocent, but in reality, it's just someone's job.
Money is the root of *most* work.
If you can't trust Amazon.com user opinions, what makes you think you can trust answers on Ask Slashdot? Many of us rattle off about every issue out of ignorance and get modded up despite that ignorance. Can you trust us?
The reviews, that I recall, spoke of how the comics confronted the uber secret freemason's and so forth. I believe there were quite a few Kafka and Nietsche references as well. Hilarious.
Now I have a new google project for the day. Somebody somewhere must have saved 'em!
If brevity is the soul of wit, then how does one explain Twitter?
90% of the reviews I see fall into one of these categories:
1. 5 star reviews from crazy fan-types who absolutely love some movie/musician/cartoon, etc.
2. Anti-fans who post 1 star reviews about stuff they can't stand being popular.
3. Fans who've turned into anti-fans, claiming that item X "isn't as good as their previous efforts" and that the creator "has sold out."
I've come to rely on Amazon.com not only for good deals, but also for basic product information. They carry such a wide range of items and have detailed information about everything.
I also admit I take the product reviews very seriously when I am going to buy something, especially if it's not cheap. I do this even if I don't plan on buying it from Amazon.
I doubt I am the only person like this. It would seem only logical that companies would do anything they can to get favorable reviews about their products on Amazon.
I guess you just have to read as many of the reviews as possible and make a judgement call. I do tend to look for negative reviews as they're definitely not paid for! However, those also must be taken with a grain of salt.
I rate Amazon's reviews at five stars! Nothing could be better!
-G. Cooke, TX
Review * 1/2
I rate Slashdot at a star and a half. Definitely take anything you hear there with a grain of salt.
-G. Cooke, TX
Before any good/useful course of action can be taken, there are several steps to take:
1) Determine if there is some 'shady' reviewing going on (although this looks like a pretty solid set of circumstancial evidence).
2) Find out who's responsible for it.
If it is amazon, itself, that is doing this sort of thing, then the only actions that can be taken are public outcry, and possibly an e-mail campaign. After all, they may be using it to drum up business, and as we have all learned from spammers: if it works, they'll keep on doing it.
On the other hand, if it is some third party submitting reviews, it might be possible to get Amazon to remove all reviews by said person...
Just a thought
hmmmm?
.
I was once looking through some of my old favorite Sci-Fi books on Amazon, and noticed a few "top 100" reviewer reviews. I noticed that they all had one thing in common. Two paragraphs. The first paragraph was so generic and "ooh ahh neato" that it could have been about anything. The last paragraph looked like it was paraphrased right from the back/inside cover.
So I went to one of the top 10 reviewers. She claimed to be a librarian who speed reads one book a day, and rewviewed each one. ALL her SciFi book reviews looked just like one another, and all of them had 5 stars out of 5, even some of the worst SciFi I've ever read in my life.
They don't just need meta-moderation. They need personalized meta-moderation. I want to select the group of people whose reviews I trust, and the people whose reviews of reviewers I trust. Maybe the "tragedy of the commons" is ok for Slashdot, but I'd sure hate to have that affecting the reviews that I see for actual products. I want other people like me to review the products that I buy.
This problem ranks right up there along with eBay auctions and the fact that they "close" at a given point in time. In the real world, an auction continues as long as people are making bids. eBay should extend an auction by 5 minutes or an hour or a day each time someone bids on an item. That'd get rid of "last minute bid services". (I'd suggest a 5 minute extension - because then there's a natural time for everyone interested in an item to "gather" together and do the final bidding.)
.
Forget the star/number/etc. rating and read the reviews. People that have actually used a product will have real comments about the good, bad, and ugly of it. Ignor reviewers that say "Wonder product! I am completely satisfied." or "Waist of money!". They don't tell you anything. Stuff like "The battery door broke after a few days, but the company Fed-Ex'ed me a new one.", or "Works great with Bob's widget.12.tgz drivers!.", give you actual information about what happened when they used the product. Judge for yourself how many stars they're worth.
I will rarely take the word of any one reviewer, top 10 or not. They often have simple factual mistakes about products. But if you read a dozen or so comments you can usually put together a decent impression and collect useful data. Review spam campaigns like the Rush Limbaugh-led attack on Michael Moore's book are rare.
I would certainly rather have the reviews than not. I've bought many cool things from Amazon that I never would have considered or even found (music mostly) if not for the reviews and comments.
Mig
Check out Henry Raddick's stuff -- I think the guy's got a dry British sense of humor and he can be really funny:
Henry Raddick's reviewsQuick sample:
From the beginning Epinions.com thought, "That would be great/cheap content to just have people post their opinions about products, but why would you trust random people?" So they baked in a rather complex "web of trust" into their website from Day 1.
If you see someone whose reviews reflect your own opinions, you can add them to your list of trusted people. Then when you see a list of reviews, your trusted people's reviews are at the top. Furthermore, your trusted people also have people they trust and you are likely to turst those people too, just maybe not quite as much. So your trusted people's trusted people's reviews bubble up near to the top, and so on.
Also, if you see some reviewer who you think is way off base, you can block them and never see their reviews again. It's a clever scheme, and if you use the site enough, you can tailor it to serve you decent reviews quickly. And it's all based on your opinion of other people's opinions, unlike Amazon which just bubbles up reviews from people who write a lot of reviews. I think quantity of reviews is hardly a good metric to use.
eBay should extend an auction by 5 minutes or an hour or a day each time someone bids on an item. That'd get rid of "last minute bid services". (I'd suggest a 5 minute extension - because then there's a natural time for everyone interested in an item to "gather" together and do the final bidding.)
Shit, I should patent that.
.
This is nothing new, I tend to take peer reviews in a forum setting with a grain of salt.. it is too easy for a company to post positive information about their products or negative reviews of the competition.
I still like google to find more professional reviews of computer hardware and consumer electronics like PDAs. I was looking up information on the palm m130 vs. the m515 today, for example, and found google invaluable. After reading about four reviews on different sites, I feel that I have a good understanding of both products. The highest ranked reviews were actually very professional and well written too, something that is harder and harder to find now..
There was an article on CNet last Friday about automatic recommendation systems. What happens when systems automatically recommend things?
In a incident that highlights the pitfalls of online recommendation systems, Amazon.com on Friday removed a link to a sex manual that appeared next to a listing for a spiritual guide by well-known Christian televangelist Pat Robertson.
The two titles were temporarily linked as a result of technology that tracks and displays lists of merchandise perused and purchased by Amazon visitors. Such promotions appear below the main description for products under the title, "Customers who shopped for this item also shopped for these items."
Basically, the gist of it is that people shopping for the televangellist's spiritual guide, and Amazon's recommendation system suggested that people who bought this also bought another book called The Men's Guide to Anal Sex.
I'm speculating, but I would guess that such a system could be hacked by ordering both books, and then shortly later cancelling your order. The order cancellation probably does not remove the association of these two items in the recommendation database tables.
Those who would give up liberty in exchange for security and DRM should switch to Microsoft Palladium!
One of the technical book publishers I was talking to recently was telling how they discovered whenever they would release a new title to Amazon (programming or software development usually), that they would immediately get a rash of very bad reviews -- all of which came about the same time from the same IP address which happened to belong to a rival publisher. He told me that now it's almost accepted industry practive to have your employees post bad reviews of your competitors products online.
Myself, I look for the reviews that are by people who clearly have actually read the book/used the product. There is sort of an art to picking them out.
And yes, most of my reviews are positive. Much of this is experience, negative reviews tend to get negative votes very quickly. Also the stuff I love is the stuff I want to write about. Occasionally I'll post a negative review of something I think deserves a health warning, but usually it'll be due to some technical rather than artistic consideration - "historical" CDs that aren't labelled as such, that kind of thing.
My advice is don't make a decision on the basis of one review. Note the ones that describe what the reviewer loved and see whether or not that would be something that you would love too. I usually put enough in my reviews to ensure that even if I write a rave review, someone who wouldn't like it will learn enough from what I've written to realise they wouldn't like it.
As for the reviewer that's the topic of this discussion, I have no idea. Why not read the reviews, see if they're actually useful, and if they are, then make the decision on that basis?
This isn't the kind of thing you have to get off your backside about. You can just talk about it on Slashdot. There's no need to write to your senator or congressman.
You are not alone. This is not normal. None of this is normal.
Lots of good stuff here, just browse through them:
Henry Raddick's Reviews
Here's an example, for the book "Surviving Divorce: A Handbook for Men" by Gay Search:
A well-written and challenging book which I bought for my Uncle Sandy as he attempts to cope with the aftershock of divorce. Unfortunately he thought the author's name was a coping strategy being suggested and he refused to read it.
"Moderate drinking can help prevent amputated limbs" -- Abigail Zuger, NYTimes, 12/31/02
I know I have seen one review from the #1 reveiwer -- Harriet Klausner -- and it was awful. It's for the book The Scar by China Mieville, and it's terrible. I think she read the back cover, the first few pages, and a few other reviews to make her own review. She ends the review with this sentence: "Award winning China Mieville (see Perdido Street Station) is bound to more than just receive nominations; she is going to win many trophies for this strong story."
...
As most of you know, China Mieville is NOT A SHE. Anyone who actually had the book couldn't miss this fact, since there is a big picture of him on the back flap.
They do have some moderation, though. When I first saw the review it was one of the featured reviews on the main book page, but now that 12 people have said it was "unhelpful" it has fallen to the third page.
Anyway, that should tell you something about how good the top reviewers are
-- Hobbits suck!
Obvious answer - Nope, don't trust Amazon reviews. I'm quite sure if there are too many negative reviews of an item, they remove them. How many items have you found with an average of less than three stars?
When the "listmania" stuff first came I made a couple of lists one evening when I was bored, one of them was "10 worst films ever", featuring Armageddon, Waterworld, Independence Day and so on. It was quickly removed.
Still, the filters and the reviewers sometime let through stuff, and people can still take the piss out of some products if they
are creative...
"What really makes David Hasselhoff stand apart from his contemporaries is his magnificent voice. Some critics have compared his resounding tenor to that of Mario Lanza or Johnny Hartman, but I would compare it more to a wounded jackal getting whipped by a screaming pornfilm fluffer.
So enjoy this comprehensive collection of Hasselhoff's greatest. You won't regret a moment of it!"
Check it out, there are 449 reviews in that style...
Being bitter is drinking poison and hoping someone else will die
Slashdot should make a "public" account with login "Slashdot", and every single Slashdot user should go on and review one item.
We'd be the number 1 reviewer in no time. The current #1 reviewer has 4052 reviews.
- Vincit qui patitur.
I ordered a book from an Amazon "zShop".. these zShops.. aka 3rd party sellers... are rated by customers and given comments. Well, I ordered a used book and apparently somewhere in the text description was "This book is not the one listed!" and it mentioned a different title. I just saw the picture which was the book I wanted, saw the condition as "Like new".. and ordered it. To cut to the chase, after I got the wrong book, I demanded they refund shipping as well as the price of the book. They refused. I gave them a horrible review on Amazon. Amazon only shows the 10 most recent reviews on the individual "zShop page" unless you "click for more reviews"... Mind you, for this particular zShop.. lastpagebooks specifically, the last review on them was quite some time ago. The next day, I see my review is suddenly #30 or so, with a bunch of one line "This store is great! A++++++++!" comments, 5 stars for each. And my comment is suddenly lost in obscurity. Apparently Amazon has no problem with this, or at the very least, no solution.
Not to criticize or defend this review system, I think I have come up with what could be a reasonable explanation for many of the reviews being on the same day - a batch queue. Amazon does not allow you to post reviews in realtime. You post a review, and they review the review, and then post it. The way I imagine it would work is that someone reads through hundreds and hundreds of reviews in a month and then clicks on a button to force them live from a queue. One reason I know they do in fact review the reviews is that sometimes you will see somewhere where someone posts a price or a URL in the review, but Amazon brackets it out like [removed] or something like that.
--Jon
She gives five stars to _another_ men's electric razor on November 26.
She gives five stars to an iron on October 1.
She gives five stars to _another_ iron on November 23.
She gives five stars to a cordless vaccuum on August 11.
She gives five stars to _another_ cordless vaccuum on September 7.
She gives five stars to a regular vaccuum on August 6.
She gives five stars to _another_ regular vaccuum on October 13.
Come to your own conclusions. My feeling is that she is either:
A: a professional product reviewer, in which case Amazon should include a disclaimer that she is being paid for her reviews,
B: a compulsive liar / attention-seeker,
C: a collection of reviewers all publishing under one pseudonym, in which case Amazon should include a disclaimer that she is not a real person.
D: the marketing department for Amazon / Target, in which case Amazon should include a disclaimer that she is being paid and is not a real person.
When Ever asking people for an opinion about someting most people seem to have a very static 2 sided view on things thus 5 Stars and 1 Stars.
Lets use Slashdot as an example.
What ever Microsoft Does 1 star.
What ever Linux Does 5 Stars.
(They both did the same thing)
Marketing 1 Star.
Getting a Job 5 Stars.
Apple 2.5 Stars.
Half of the people are 5 Stars
The other Half of the people are 1 star.
And even look at moderation How many moderations do you see as 5 and -1 (1 and 2 happen without moderation) but there is usually only a little bit or 3s and 4s.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
If anyone read the book "21 Dog Years, Doing Time @ Amazon.com"
Mike Daisey discusses how he was a customer service call center lacky but also reviewed childrens toys for the site. He did this from home on his own time, The funny thing was he would fall far behind his reviews and have closets full of unreviewed Fisher Price and Playskool toys laying about the house.
Good thing a fat guy with no children who's never been a parent was reviewing childrens toys.
can be found here. As you can see, Harriet Klausner has reviewed over 4,000 items, which seem to be mostly books. I challenge you to find a book that she didn't like! Four stars is the lowest score that she has ever given as far as I can tell. She must be on vacation or something because she has only posted two reviews this month. Also, notice that her reviews are all three paragraphs long and the second paragraph tends to start with the word "However". Maybe she is just a book reviewing bot. She certainly doesn't write interesting reviews.
Lasers Controlled Games!
http://www.amazon.com/exec/obidos/tg/cm/member-rev iews/-/AA9IP6AYACFK5/102-2168050-5628139
Henry Raddicks reviews need to be read in full to get some of the jokes he mentions. He has a whole cast of characters (homophobic uncle, poor dog he's always trying some crazy theory on) that he references in many of his reviews..
Examples...
Handbook of Meat Product Technology
An admirably thorough guide to the tools of the production-line meat processing trade. The superb colour photographs particularly made it a perfect gift for my 15 year old daughter who is showing alarming signs of not becoming vegetarian.
God, Why Did Dad Lose His Job?
A truly wonderful guide which has enabled me to explain my recent sacking for vandalising company property to my children in terms of a minor act of redemption. First rate.
Okay, running Gail Cooke through an Anagram Generator, I get 'A GECKO OIL'. As we all know a gecko is a type of reptile, or should I say snake? We all know that selling 'Snake Oil' conjures up a marketing scam. So, Gail Cooke = Snake Oil.
Thank you. Is there like a prize or something for like the best post ever? I now kneel as you bestow thy Karma upon thee.
"This isn't a study in computer science, its a study in human behavior"
Most all reviews are worthless because they simply lack any context. I've made this comment about the Slashdot book reviews too.
When Joe Blow say this product is great or that product is crap, it really means NOTHING, and putting any faith into these reviews is a major mistake. Unless the reviewer gives some background about themselves and their history with the product or category of products you have no way of knowing how they relate to you.
This is especially true of technical book reviews. Without having some idea about the persons knowlege and skill level and what experience they have, there is no real value in the review. For this reason, when I write book reviews I also include a brief resume listing things like years of experience in that field, certifications, other books that I have read on the subject, etc.
There's lots of noise in the reviews, but you can still filter out a useful signal. Many reviews are also obviously dumb, such as the guy that ordered a CD player, loved it, but decided he didn't need it after all, so he gave it one star. Brains entirely optional. After reading through ten or twenty reviews of the CD player, you'd find recurring mentions of good or bad battery life, skipping/no skipping etc, which is usually all you need.
So there's this Slashdot user named "Anonymous Coward" that posts several thousand times PER DAY! Very little of what he or she writes is useful. I'm having a hard time believing that one person can generate this much garbage.
If these suspicious persons have reviewed so many products, they may have a reviewed a few of which you have personal knowledge. Those are golden data points.
If there aren't any such datapoints, maybe somebody whose reviews you DO agree with has reviewed products the suspicious person has also reviewed. Those are, shall we say, silver data points.
Obviously what I'm working up to here is an automated system for finding short paths in networks of reviewers. Not a Web of Trust, but a Web of Agreement, built upon the mutual information (the degree of agreement) between various reviewers.
I have seen "mutual interest" systems such as for music. Those are useless because of shills that (pretend to) like everything, but a system built on mutual information would find those reviewers to have 0 information content.
My wife is pre-med. I'm the one who's hoping to be driving the Lincoln SUV carting kids back and forth to soccer practice (pulling a Bass boat as well). You've just got to pick 'em right.
Google Viewer - View search results as scrolling web page images
Google Webquotes - View search results with quotes about them from other sites
Check it out at Google Labs
Should You Trust Website Customer Reviews?
Heh..that's like determining your self esteem level by your Slashdot moderation points..
-ted
In early '99, they were getting about 5000 reviews in a day. Each review is checked twice, once by a filter(dirty words), and the other by eyes(relavence). There were about 20 catalogue people then. They post all book listings, CD's, products, answer crazed author questions(my fav part)...and if they have time, post reviews.
If you guys think Amazon is trying to pursuade you with reviews...your high on conspiricy juice. They are pure fluff for your sake. We barley had enough time to add new products to the site, let alone check reviews. Trust me, Amazon would LOVE to get rid of the review system, cuase it is a pain in the ass to maintain and labor intensive. But they keep it cuase customers wanted/want it.
And no, we didn't re-order the reviews and put the good ones at the top. It is a pure FIFO (First-In-First-Online) system. People just usually post if they really hate, or really like something. So the reviews are more rants than critiques and tend to be very biased...just like /.
P.S. I still smell like the Art Bar too!
Informative??? This is supposed to be a JOKE. Froogle has nothing to do with reviewers. It's a products-for-sale search engine. "Find cheap reviewers at..." -- get it? JOKE!
The nice thing about the reviews, whether they are valid or not, is that they lead to more books being shown if your recommendations list. Why is this good? Remember: if you buy something from Amazon based on its recommendation, and you didn't like it, they'll take it back.
Get off my launchpad!
Well they don't sell anything, they just do a pricewatch-type thing on the prices for products and get paid for some of the referrals. And they let you review things they don't get any money off of (see the Restaurants section for example).
I haven't checked out Epinions extensively in a while, but I worked there very briefly a few years ago and their ultimate goal was to let the community build the product hierarchy, which is good for them (they don't have to pay experts full-time to build/prune the product trees) and good for the consumer (you could review just about anything, as long as the community was fast and responsive). That is, as long as certain people were in charge of keeping the product hierarchy clean and correct (think dmoz.org). I don't know if they have implemented this yet, though.
They definitely don't hide bad reviews of things (see the cell phone service provider section, for instance). They pride themselves on having a great deal of unbiased reviews. They just believe that if people are given access to good reviews on a product, it will help steer them towards the right product for them, which the person will eventually buy. Sure, sometimes a bunch of bad reviews will lead people to not buy something they might normally, but that isn't usually how things play out.
Yes, reviewers are paid to give rave reviews, just like search engines are paid to increase web page rankings.
Once, after I gave a rave review, I got an email several months later from John Wiley & Sons, offering me $50 to review a similar book of theirs. (This was back when Amazon.com put email addresses next to reviewers' names.)
I don't remember if the paid review was to be submitted to a bookseller's site, or to more conventional book review media.
However, I was a 'Catalogue Specialist' (the people who post new listings) at Amazon for a few months and I can confirm you don't have a clue. You seem to think that Amazon has a whole staff of people making sure that product rankings are high, when in fact, they could give a shit what the ranking is. The only people I've ever heard of bitching about ranking and reviews were authors, complaining about their books negative listing.
In fact I have a memorable and funny story to illustate just how much Amazon could care less about the reviews and rankings.
So what was the point of that story...to a) illustrate how Amazon could really care less about ranking and b) that there is no review/rating conspiricy at Amazon.
I have a feeling the reviews in 'question' were like this, "This thing sucks, and it took an extra day to get here." Or some similar crap. In that case, yeah it gets rejected cuase it's a space waster.
We all know about "Google Bombing". This is the same thing, "Amazon Bombing", and involving public perceptions and trust being leveraged over the Internet. Amazon is a huge retail presence, and on top of that they have a public interface to their product review/promotion API! Cha-ching...the sound of money.
But the same thing happened when the snake oil salesman rode his wagon into town in the 1800's. There would be a plant, a shill, in the crowd who on cue would get all hysterical about the presentation and appear buy 6 bottles. Cha-ching...the oil flowed.
But then again, some celebrity going on about how great some gadget or pill is isn't so different; just playing on your goofy, unquestioning trust.
Trust is the ultimate sales API, and goes straight to the brain stem. Your instincts are used against you and...cha-ching...everywhere the sound of money.
=^..^= all your rodent are belong to us
This lady is not an Amazon employee or on their payroll, she is just looking for validation for her meaningless life by publishing reviews and GIVING them to web sites. GIVE is the important qualifier here.
Your reply is off the mark. The poster was talking about big releases and not about some little known dummies guide.
And with all due respect, had there been a conspiracy taking place in Amazon, I have the feeling that you were not high enough to be in on it.