Slashdot Mirror
Should You Trust Website Customer Reviews?
crevette asks: "I was looking on Amazon for some gizmo when I got a review from G. Cooke, TX, who is in the top 10 reviewers. Out of curiosity, I checked her reviews... She has 658 reviews, many on the same day, which include everything from knife sets to a plastic duck! She reviews many books on the same day... She must be spending hundreds of dollars on useless stuff every month. Worst of all, most of her reviews are 5 stars. Do you think those people are paid by Amazon or some company? Do you trust them? If not (like I tend to think) what can we do about it?"
go to Google's new beta product search to find cheap reviewers!
fp
icblf
It's a public website; anyone can post a review. Trust it as much as you do Slashdot. :)
I read a bunch of Cooke's reviews. She has nothing bad to say, nothing critical, and all her reviews read like an ad. Yeah, I'd say something's wrong here.
and so should you.
I don't need no instructions to know how to rock!!!!
Most of the reviews that I see are either very positive or very negative. There are very few reviews that label a product as simply okay or adequate.
This is probably due to the fact that only people that have had extremely positive or extremely negative experiences with a particular product will actually take the time to post. Well, other than a few people with too much time on their hands that want to be listed as top reviewers.
.sig wanted. Inquire within.
The point I'm making is that the reviewers aren't always participating in a community, but also acting out a personal agenda as well, which other customers may take in as fact.
Might explain why she reads so much....
Name: G. Cooke
Nickname: pagebypage
E-mail: grospoin@aol.com
Reviewer Rank: 8
About me: So many good books, so many wonderful things....so few hours. For me, reading and trying is sheer pleasure, a boon companion, and sometimes work as reviewing is my profession. I hold membership in the National Book Critics Circle, and write for newspapers across the U.S. Thanks to all who have expressed an interest in my reviews, and thanks to Amazon for providing a forum in which we can exchange ideas.
Figuring out her Email address, then Googling it reveals Ms. Cooke probably does nothing but writes reviews allllllllll day long.
Man, what a horrible way to waste time. Well, back to reloading Slashdot....
sig--we don't need no goddamn sig
If you can't trust Amazon.com user opinions, what makes you think you can trust answers on Ask Slashdot? Many of us rattle off about every issue out of ignorance and get modded up despite that ignorance. Can you trust us?
90% of the reviews I see fall into one of these categories:
1. 5 star reviews from crazy fan-types who absolutely love some movie/musician/cartoon, etc.
2. Anti-fans who post 1 star reviews about stuff they can't stand being popular.
3. Fans who've turned into anti-fans, claiming that item X "isn't as good as their previous efforts" and that the creator "has sold out."
I rate Amazon's reviews at five stars! Nothing could be better!
-G. Cooke, TX
Review * 1/2
I rate Slashdot at a star and a half. Definitely take anything you hear there with a grain of salt.
-G. Cooke, TX
I think the question the author is trying to ask is how can we get truthful reviews on a product. Considering what marketers are willing to do, I would not put 'padding' their reivews past them.
.
I was once looking through some of my old favorite Sci-Fi books on Amazon, and noticed a few "top 100" reviewer reviews. I noticed that they all had one thing in common. Two paragraphs. The first paragraph was so generic and "ooh ahh neato" that it could have been about anything. The last paragraph looked like it was paraphrased right from the back/inside cover.
So I went to one of the top 10 reviewers. She claimed to be a librarian who speed reads one book a day, and rewviewed each one. ALL her SciFi book reviews looked just like one another, and all of them had 5 stars out of 5, even some of the worst SciFi I've ever read in my life.
They don't just need meta-moderation. They need personalized meta-moderation. I want to select the group of people whose reviews I trust, and the people whose reviews of reviewers I trust. Maybe the "tragedy of the commons" is ok for Slashdot, but I'd sure hate to have that affecting the reviews that I see for actual products. I want other people like me to review the products that I buy.
This problem ranks right up there along with eBay auctions and the fact that they "close" at a given point in time. In the real world, an auction continues as long as people are making bids. eBay should extend an auction by 5 minutes or an hour or a day each time someone bids on an item. That'd get rid of "last minute bid services". (I'd suggest a 5 minute extension - because then there's a natural time for everyone interested in an item to "gather" together and do the final bidding.)
.
Forget the star/number/etc. rating and read the reviews. People that have actually used a product will have real comments about the good, bad, and ugly of it. Ignor reviewers that say "Wonder product! I am completely satisfied." or "Waist of money!". They don't tell you anything. Stuff like "The battery door broke after a few days, but the company Fed-Ex'ed me a new one.", or "Works great with Bob's widget.12.tgz drivers!.", give you actual information about what happened when they used the product. Judge for yourself how many stars they're worth.
I will rarely take the word of any one reviewer, top 10 or not. They often have simple factual mistakes about products. But if you read a dozen or so comments you can usually put together a decent impression and collect useful data. Review spam campaigns like the Rush Limbaugh-led attack on Michael Moore's book are rare.
I would certainly rather have the reviews than not. I've bought many cool things from Amazon that I never would have considered or even found (music mostly) if not for the reviews and comments.
Mig
Check out Henry Raddick's stuff -- I think the guy's got a dry British sense of humor and he can be really funny:
Henry Raddick's reviewsQuick sample:
From the beginning Epinions.com thought, "That would be great/cheap content to just have people post their opinions about products, but why would you trust random people?" So they baked in a rather complex "web of trust" into their website from Day 1.
If you see someone whose reviews reflect your own opinions, you can add them to your list of trusted people. Then when you see a list of reviews, your trusted people's reviews are at the top. Furthermore, your trusted people also have people they trust and you are likely to turst those people too, just maybe not quite as much. So your trusted people's trusted people's reviews bubble up near to the top, and so on.
Also, if you see some reviewer who you think is way off base, you can block them and never see their reviews again. It's a clever scheme, and if you use the site enough, you can tailor it to serve you decent reviews quickly. And it's all based on your opinion of other people's opinions, unlike Amazon which just bubbles up reviews from people who write a lot of reviews. I think quantity of reviews is hardly a good metric to use.
Mutant Dog
Click on the Bil Keane icon for the archive.
If brevity is the soul of wit, then how does one explain Twitter?
eBay should extend an auction by 5 minutes or an hour or a day each time someone bids on an item. That'd get rid of "last minute bid services". (I'd suggest a 5 minute extension - because then there's a natural time for everyone interested in an item to "gather" together and do the final bidding.)
Shit, I should patent that.
.
One of the technical book publishers I was talking to recently was telling how they discovered whenever they would release a new title to Amazon (programming or software development usually), that they would immediately get a rash of very bad reviews -- all of which came about the same time from the same IP address which happened to belong to a rival publisher. He told me that now it's almost accepted industry practive to have your employees post bad reviews of your competitors products online.
Myself, I look for the reviews that are by people who clearly have actually read the book/used the product. There is sort of an art to picking them out.
And yes, most of my reviews are positive. Much of this is experience, negative reviews tend to get negative votes very quickly. Also the stuff I love is the stuff I want to write about. Occasionally I'll post a negative review of something I think deserves a health warning, but usually it'll be due to some technical rather than artistic consideration - "historical" CDs that aren't labelled as such, that kind of thing.
My advice is don't make a decision on the basis of one review. Note the ones that describe what the reviewer loved and see whether or not that would be something that you would love too. I usually put enough in my reviews to ensure that even if I write a rave review, someone who wouldn't like it will learn enough from what I've written to realise they wouldn't like it.
As for the reviewer that's the topic of this discussion, I have no idea. Why not read the reviews, see if they're actually useful, and if they are, then make the decision on that basis?
This isn't the kind of thing you have to get off your backside about. You can just talk about it on Slashdot. There's no need to write to your senator or congressman.
You are not alone. This is not normal. None of this is normal.
Lots of good stuff here, just browse through them:
Henry Raddick's Reviews
Here's an example, for the book "Surviving Divorce: A Handbook for Men" by Gay Search:
A well-written and challenging book which I bought for my Uncle Sandy as he attempts to cope with the aftershock of divorce. Unfortunately he thought the author's name was a coping strategy being suggested and he refused to read it.
"Moderate drinking can help prevent amputated limbs" -- Abigail Zuger, NYTimes, 12/31/02
Obvious answer - Nope, don't trust Amazon reviews. I'm quite sure if there are too many negative reviews of an item, they remove them. How many items have you found with an average of less than three stars?
When the "listmania" stuff first came I made a couple of lists one evening when I was bored, one of them was "10 worst films ever", featuring Armageddon, Waterworld, Independence Day and so on. It was quickly removed.
Still, the filters and the reviewers sometime let through stuff, and people can still take the piss out of some products if they
are creative...
"What really makes David Hasselhoff stand apart from his contemporaries is his magnificent voice. Some critics have compared his resounding tenor to that of Mario Lanza or Johnny Hartman, but I would compare it more to a wounded jackal getting whipped by a screaming pornfilm fluffer.
So enjoy this comprehensive collection of Hasselhoff's greatest. You won't regret a moment of it!"
Check it out, there are 449 reviews in that style...
Being bitter is drinking poison and hoping someone else will die
Yeah, I guess it was a poorly worded question on his part. Still, people need to use common sense when they read these things. I do use online reviews to make buying decisions, and I've found that the quality of the reviews depends a lot on what you're trying to purchase.
For instance, technical books tend to have good reviews on Amazon. This is probably because it's more difficult for a marketer to fake that stuff. I've also used reviews on others sites before buying a lawn mower, snow thrower, and a computer parts supplier. I had to dig through more noise, but the results have all been positive.
Probably the worst reviews that I've found have been for video games. It's nothing but blind fans, posting mindless garbage about how much this game rules.
What I usually do is look for trends in the negative reviews. If there is a common theme emerging, then I might stay away from that product or company. The same can be said for the reviews that people get on Ebay. If someone has a few negatives, and they all say that he didn't ship on time, there might be a problem there.
A reviewer who takes the time to use proper grammar and explains his thoughts logically will obviously carry more weight.
Again, it's just common sense, with a little luck thrown in.
I ordered a book from an Amazon "zShop".. these zShops.. aka 3rd party sellers... are rated by customers and given comments. Well, I ordered a used book and apparently somewhere in the text description was "This book is not the one listed!" and it mentioned a different title. I just saw the picture which was the book I wanted, saw the condition as "Like new".. and ordered it. To cut to the chase, after I got the wrong book, I demanded they refund shipping as well as the price of the book. They refused. I gave them a horrible review on Amazon. Amazon only shows the 10 most recent reviews on the individual "zShop page" unless you "click for more reviews"... Mind you, for this particular zShop.. lastpagebooks specifically, the last review on them was quite some time ago. The next day, I see my review is suddenly #30 or so, with a bunch of one line "This store is great! A++++++++!" comments, 5 stars for each. And my comment is suddenly lost in obscurity. Apparently Amazon has no problem with this, or at the very least, no solution.
She gives five stars to _another_ men's electric razor on November 26.
She gives five stars to an iron on October 1.
She gives five stars to _another_ iron on November 23.
She gives five stars to a cordless vaccuum on August 11.
She gives five stars to _another_ cordless vaccuum on September 7.
She gives five stars to a regular vaccuum on August 6.
She gives five stars to _another_ regular vaccuum on October 13.
Come to your own conclusions. My feeling is that she is either:
A: a professional product reviewer, in which case Amazon should include a disclaimer that she is being paid for her reviews,
B: a compulsive liar / attention-seeker,
C: a collection of reviewers all publishing under one pseudonym, in which case Amazon should include a disclaimer that she is not a real person.
D: the marketing department for Amazon / Target, in which case Amazon should include a disclaimer that she is being paid and is not a real person.
When Ever asking people for an opinion about someting most people seem to have a very static 2 sided view on things thus 5 Stars and 1 Stars.
Lets use Slashdot as an example.
What ever Microsoft Does 1 star.
What ever Linux Does 5 Stars.
(They both did the same thing)
Marketing 1 Star.
Getting a Job 5 Stars.
Apple 2.5 Stars.
Half of the people are 5 Stars
The other Half of the people are 1 star.
And even look at moderation How many moderations do you see as 5 and -1 (1 and 2 happen without moderation) but there is usually only a little bit or 3s and 4s.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
If anyone read the book "21 Dog Years, Doing Time @ Amazon.com"
Mike Daisey discusses how he was a customer service call center lacky but also reviewed childrens toys for the site. He did this from home on his own time, The funny thing was he would fall far behind his reviews and have closets full of unreviewed Fisher Price and Playskool toys laying about the house.
Good thing a fat guy with no children who's never been a parent was reviewing childrens toys.
Okay, running Gail Cooke through an Anagram Generator, I get 'A GECKO OIL'. As we all know a gecko is a type of reptile, or should I say snake? We all know that selling 'Snake Oil' conjures up a marketing scam. So, Gail Cooke = Snake Oil.
Thank you. Is there like a prize or something for like the best post ever? I now kneel as you bestow thy Karma upon thee.
"This isn't a study in computer science, its a study in human behavior"
So there's this Slashdot user named "Anonymous Coward" that posts several thousand times PER DAY! Very little of what he or she writes is useful. I'm having a hard time believing that one person can generate this much garbage.
If these suspicious persons have reviewed so many products, they may have a reviewed a few of which you have personal knowledge. Those are golden data points.
If there aren't any such datapoints, maybe somebody whose reviews you DO agree with has reviewed products the suspicious person has also reviewed. Those are, shall we say, silver data points.
Obviously what I'm working up to here is an automated system for finding short paths in networks of reviewers. Not a Web of Trust, but a Web of Agreement, built upon the mutual information (the degree of agreement) between various reviewers.
I have seen "mutual interest" systems such as for music. Those are useless because of shills that (pretend to) like everything, but a system built on mutual information would find those reviewers to have 0 information content.
Google Viewer - View search results as scrolling web page images
Google Webquotes - View search results with quotes about them from other sites
Check it out at Google Labs
Should You Trust Website Customer Reviews?
Heh..that's like determining your self esteem level by your Slashdot moderation points..
-ted
In early '99, they were getting about 5000 reviews in a day. Each review is checked twice, once by a filter(dirty words), and the other by eyes(relavence). There were about 20 catalogue people then. They post all book listings, CD's, products, answer crazed author questions(my fav part)...and if they have time, post reviews.
If you guys think Amazon is trying to pursuade you with reviews...your high on conspiricy juice. They are pure fluff for your sake. We barley had enough time to add new products to the site, let alone check reviews. Trust me, Amazon would LOVE to get rid of the review system, cuase it is a pain in the ass to maintain and labor intensive. But they keep it cuase customers wanted/want it.
And no, we didn't re-order the reviews and put the good ones at the top. It is a pure FIFO (First-In-First-Online) system. People just usually post if they really hate, or really like something. So the reviews are more rants than critiques and tend to be very biased...just like /.
P.S. I still smell like the Art Bar too!
However, I was a 'Catalogue Specialist' (the people who post new listings) at Amazon for a few months and I can confirm you don't have a clue. You seem to think that Amazon has a whole staff of people making sure that product rankings are high, when in fact, they could give a shit what the ranking is. The only people I've ever heard of bitching about ranking and reviews were authors, complaining about their books negative listing.
In fact I have a memorable and funny story to illustate just how much Amazon could care less about the reviews and rankings.
So what was the point of that story...to a) illustrate how Amazon could really care less about ranking and b) that there is no review/rating conspiricy at Amazon.
I have a feeling the reviews in 'question' were like this, "This thing sucks, and it took an extra day to get here." Or some similar crap. In that case, yeah it gets rejected cuase it's a space waster.