Slashdot Mirror
Should You Trust Website Customer Reviews?
crevette asks: "I was looking on Amazon for some gizmo when I got a review from G. Cooke, TX, who is in the top 10 reviewers. Out of curiosity, I checked her reviews... She has 658 reviews, many on the same day, which include everything from knife sets to a plastic duck! She reviews many books on the same day... She must be spending hundreds of dollars on useless stuff every month. Worst of all, most of her reviews are 5 stars. Do you think those people are paid by Amazon or some company? Do you trust them? If not (like I tend to think) what can we do about it?"
The point I'm making is that the reviewers aren't always participating in a community, but also acting out a personal agenda as well, which other customers may take in as fact.
I heard an NPR story on the person who has the second most reviews with Amazon (somewhere in the 4000's i think) He was legite. These people are addicted to posting reviews and i am sure they post most of them at the same time. I don't know how they get their products buying or promos.
Figuring out her Email address, then Googling it reveals Ms. Cooke probably does nothing but writes reviews allllllllll day long.
Man, what a horrible way to waste time. Well, back to reloading Slashdot....
sig--we don't need no goddamn sig
I remember seeing some news story about this guy who's competing with another lady for the most number of Amazon reviews. He basically sits around and reviews tons of stuff that he has himself, and he does buy a good amount as well. I don't think there was any implication of anyone sponsoring him since all his reviews were fairly random. I think it's basically a case of one of those people who collect things obsessively, in this case it's reviews. In any case I usually do trust reviews from product specific websites (like a digital camera site), not something as varied as Amazon.
I will rarely take the word of any one reviewer, top 10 or not. They often have simple factual mistakes about products. But if you read a dozen or so comments you can usually put together a decent impression and collect useful data. Review spam campaigns like the Rush Limbaugh-led attack on Michael Moore's book are rare.
I would certainly rather have the reviews than not. I've bought many cool things from Amazon that I never would have considered or even found (music mostly) if not for the reviews and comments.
Mig
There was an article on CNet last Friday about automatic recommendation systems. What happens when systems automatically recommend things?
In a incident that highlights the pitfalls of online recommendation systems, Amazon.com on Friday removed a link to a sex manual that appeared next to a listing for a spiritual guide by well-known Christian televangelist Pat Robertson.
The two titles were temporarily linked as a result of technology that tracks and displays lists of merchandise perused and purchased by Amazon visitors. Such promotions appear below the main description for products under the title, "Customers who shopped for this item also shopped for these items."
Basically, the gist of it is that people shopping for the televangellist's spiritual guide, and Amazon's recommendation system suggested that people who bought this also bought another book called The Men's Guide to Anal Sex.
I'm speculating, but I would guess that such a system could be hacked by ordering both books, and then shortly later cancelling your order. The order cancellation probably does not remove the association of these two items in the recommendation database tables.
Those who would give up liberty in exchange for security and DRM should switch to Microsoft Palladium!
And yes, most of my reviews are positive. Much of this is experience, negative reviews tend to get negative votes very quickly. Also the stuff I love is the stuff I want to write about. Occasionally I'll post a negative review of something I think deserves a health warning, but usually it'll be due to some technical rather than artistic consideration - "historical" CDs that aren't labelled as such, that kind of thing.
My advice is don't make a decision on the basis of one review. Note the ones that describe what the reviewer loved and see whether or not that would be something that you would love too. I usually put enough in my reviews to ensure that even if I write a rave review, someone who wouldn't like it will learn enough from what I've written to realise they wouldn't like it.
As for the reviewer that's the topic of this discussion, I have no idea. Why not read the reviews, see if they're actually useful, and if they are, then make the decision on that basis?
This isn't the kind of thing you have to get off your backside about. You can just talk about it on Slashdot. There's no need to write to your senator or congressman.
You are not alone. This is not normal. None of this is normal.
Slashdot should make a "public" account with login "Slashdot", and every single Slashdot user should go on and review one item.
We'd be the number 1 reviewer in no time. The current #1 reviewer has 4052 reviews.
- Vincit qui patitur.
Very true. This is a problem with nearly any measurement that involves voluntary surveys, such as course surveys in universities or those little comment cards on the restarurant table.
I personally look for the reviews with 2-4 stars for stuff on Amazon, specifically hoping to get a more balanced review. If they happen to be a "Top X" reviewer, fine as long as they're relatively detailed in the good and bad of the product. Lots of stars is hardly the only factor we should use!
I was just doing some christmas shopping yesterday on Amazon, and it was also the first time I've ever actually bought something on Amazon. (I know, where have I been??).
:)
I was reading a bunch of the customer reviews, and noticed how a lot of the "featured" ones also had the "Top 500 Reviewer", "Top 10 Reviewer" (etc) graphics next to their review. So then I went to all of these people's review pages and noticed how all of these "Top" reviewers reviews were totally skewed towards the positive. 9/10 are 5 stars, with an occasional 4 star. And Gail Cooke was one of the ones that stood out.
Before I commited to anything, I went to epinions first to read some "real" reviews. Most of the reviews on amazon are just garbage.
Pointless post, I know, just weird that I was going through the exact same thing last night.
Joseph?
"Only in their dreams can men truly be free 'twas always thus, and always thus will be."
--Tom Schulman
She gives five stars to a can opener on August 1st.
She gives five stars to _another_ can opener on the very next day, August 2nd.
ok, she's a professional reviewer...so, does this mean she's posting lots of reviews on amazon to gain experience? or name recognition, or what? she's certainly not getting paid for it...i hope...
"Facts are meaningless. You could use facts to prove anything that's even remotely true." - Homer Simpson
If anyone read the book "21 Dog Years, Doing Time @ Amazon.com"
Mike Daisey discusses how he was a customer service call center lacky but also reviewed childrens toys for the site. He did this from home on his own time, The funny thing was he would fall far behind his reviews and have closets full of unreviewed Fisher Price and Playskool toys laying about the house.
Good thing a fat guy with no children who's never been a parent was reviewing childrens toys.
can be found here. As you can see, Harriet Klausner has reviewed over 4,000 items, which seem to be mostly books. I challenge you to find a book that she didn't like! Four stars is the lowest score that she has ever given as far as I can tell. She must be on vacation or something because she has only posted two reviews this month. Also, notice that her reviews are all three paragraphs long and the second paragraph tends to start with the word "However". Maybe she is just a book reviewing bot. She certainly doesn't write interesting reviews.
Lasers Controlled Games!
http://www.amazon.com/exec/obidos/tg/cm/member-rev iews/-/AA9IP6AYACFK5/102-2168050-5628139
Henry Raddicks reviews need to be read in full to get some of the jokes he mentions. He has a whole cast of characters (homophobic uncle, poor dog he's always trying some crazy theory on) that he references in many of his reviews..
Examples...
Handbook of Meat Product Technology
An admirably thorough guide to the tools of the production-line meat processing trade. The superb colour photographs particularly made it a perfect gift for my 15 year old daughter who is showing alarming signs of not becoming vegetarian.
God, Why Did Dad Lose His Job?
A truly wonderful guide which has enabled me to explain my recent sacking for vandalising company property to my children in terms of a minor act of redemption. First rate.
I've heard credible stories (from the students themselves) of being given extra credit in classes for posting a favorable review of the Professor's book to Amazon.
If your children ever found out how lame you are, they'd murder you in your sleep
Edmond H. Wollmann
If these suspicious persons have reviewed so many products, they may have a reviewed a few of which you have personal knowledge. Those are golden data points.
If there aren't any such datapoints, maybe somebody whose reviews you DO agree with has reviewed products the suspicious person has also reviewed. Those are, shall we say, silver data points.
Obviously what I'm working up to here is an automated system for finding short paths in networks of reviewers. Not a Web of Trust, but a Web of Agreement, built upon the mutual information (the degree of agreement) between various reviewers.
I have seen "mutual interest" systems such as for music. Those are useless because of shills that (pretend to) like everything, but a system built on mutual information would find those reviewers to have 0 information content.
There are also people like me who rate things as a method of tracking what they've already seen/read/used.
I find Netflix's rating engine to be a perfect method of tracking what movies I've already seen so I don't go and accidentally rent the same one twice. This has led me to have an awful lot of ratings (~2000 DVDs), but I can be fairly confident that everything in my rental queue is new.
Of course, rating things (1-5 stars) is very different from reviewing something. Reviewing something requires you to think rather hard about what you liked or didn't like about a product and is usually done because you have a strong emotion about it and feel the need to tell the world about it. Rating something is so much easier that it can be done without much thought at all, making it much more common.
Of course that's just my opinion, I could be wrong.
The world is neither black nor white nor good nor evil, only many shades of CowboyNeal.
In early '99, they were getting about 5000 reviews in a day. Each review is checked twice, once by a filter(dirty words), and the other by eyes(relavence). There were about 20 catalogue people then. They post all book listings, CD's, products, answer crazed author questions(my fav part)...and if they have time, post reviews.
If you guys think Amazon is trying to pursuade you with reviews...your high on conspiricy juice. They are pure fluff for your sake. We barley had enough time to add new products to the site, let alone check reviews. Trust me, Amazon would LOVE to get rid of the review system, cuase it is a pain in the ass to maintain and labor intensive. But they keep it cuase customers wanted/want it.
And no, we didn't re-order the reviews and put the good ones at the top. It is a pure FIFO (First-In-First-Online) system. People just usually post if they really hate, or really like something. So the reviews are more rants than critiques and tend to be very biased...just like /.
P.S. I still smell like the Art Bar too!
Also you sometimes see people posting lots of really positive reviews of reviews, presumably to kiss up to each other, but having designated epinions editors whose reviews of reviews are weighted more heavily seems to have helped that.
That said, I then found these editors' opinions to vary pretty wildly. Some gave me really good reviews, while others gave me lukewarm reviews when the work and thought on my end were essentially equal. It was an interesting illustration of how subjective these things are. Ah well, at least my ratings don't go on my permanent record...
Why? Because I thought it was important for potential buyers to know that this book was written by an admitted, convicted pedophile. I thought that some buyers would want to know this.
Apparently, Jeff Bezos loves pedophiles (he's a pedophile-phile) because he rejected the review three times. I gave up after that.
Best Buy can have you arrested
She copies and pastes her reviews for the same books on B&Bi sbnin quiry.asp?isbn=0525946896
http://search.barnesandnoble.com/booksearch/
I always sort by "Lowest rating first". It can give you a better idea, IMHO, to read these along with the top ones.
My favourite feature of the epinions.com site is that one can quickly see the top-rated items in a category, and then go and glance at the 'cons' section of each review of the interesting ones, and therefore quickly know-about anything that would clobber one's intended use for the thing...
If I care about rapid-fire photography .. and particular good-rated cameras have no capability to do it .. probably that non-capability will be noted in one of the reviews 'cons' section, and .. then I know ( having not had to trust a single+solitary reviewer or a 'review'-site )... .. to make sure that the market doesn't choose objectively, eh?
Particularly important when amazon.com seems to be interested in suppressing non-favourable reviews
( yes I know that permitting correct perception damages established entity's profits, and yes I know that that is 'evil' to market/economist types, but human-real worth is worth more to me than monetary appearances are )
Also, the reviews on epinions give me the 'gotchas' to look out for, so I understand the requirements ( for a given kind-of-item ) better! ... ) cannot evolve, but rather would only devolve, as marketing displaced, totally, vital perception.
Many thanks to everyone who makes public valid reviews, because without such we ( or 'the market'
Messages to/for me ( in me journal )
I make the occasional exception for very popular books which I think are overrated.
Danny.
I have written over 900 book reviews
Well they don't sell anything, they just do a pricewatch-type thing on the prices for products and get paid for some of the referrals. And they let you review things they don't get any money off of (see the Restaurants section for example).
I haven't checked out Epinions extensively in a while, but I worked there very briefly a few years ago and their ultimate goal was to let the community build the product hierarchy, which is good for them (they don't have to pay experts full-time to build/prune the product trees) and good for the consumer (you could review just about anything, as long as the community was fast and responsive). That is, as long as certain people were in charge of keeping the product hierarchy clean and correct (think dmoz.org). I don't know if they have implemented this yet, though.
They definitely don't hide bad reviews of things (see the cell phone service provider section, for instance). They pride themselves on having a great deal of unbiased reviews. They just believe that if people are given access to good reviews on a product, it will help steer them towards the right product for them, which the person will eventually buy. Sure, sometimes a bunch of bad reviews will lead people to not buy something they might normally, but that isn't usually how things play out.
However, I was a 'Catalogue Specialist' (the people who post new listings) at Amazon for a few months and I can confirm you don't have a clue. You seem to think that Amazon has a whole staff of people making sure that product rankings are high, when in fact, they could give a shit what the ranking is. The only people I've ever heard of bitching about ranking and reviews were authors, complaining about their books negative listing.
In fact I have a memorable and funny story to illustate just how much Amazon could care less about the reviews and rankings.
So what was the point of that story...to a) illustrate how Amazon could really care less about ranking and b) that there is no review/rating conspiricy at Amazon.
I have a feeling the reviews in 'question' were like this, "This thing sucks, and it took an extra day to get here." Or some similar crap. In that case, yeah it gets rejected cuase it's a space waster.
The original poster has a great eye!
You can't prove that Gail Cooke is a professional copywriter but you can say that she writes exactly like one.
How's this for evidence:
In a country famous for its difficulties in teaching reading and writing, Ms. Cooke has learned to write in short, concise sentences using well-chosen adjectives and without any of the most common spelling and punctuation errors. She even ends everything with an airy flourish of praise.
If she isnt a professional copywriter, she should be one.
Big hint: Could Amazon be using people like her to write ad-copy into their reviews? Could they also use others? Say, someone with a more masculine voice and approach? Perhaps someone with a love for computers and digital cameras?
Bigger hint: I have a two-sentence resume. It says that I spend a lot of time at home and, that I am rotten to the core.
To mail me, remove the 'mailno' from my email addy.
"Yeah. It smells, too..."
... to keep nice stuff coming in from the people who makes it. If I build something and want it to be reviewed in order to get some attention, I better be sure not to be trashed... El Alex Dinamo
Another difference is that most of the books I review are obscure. Most of Amazon's books are obscure too, but their reviews definitely cover popular titles better - how many thousand reviews of the Harry Potter books are there?
Danny.
I have written over 900 book reviews
It's nice that you worked for Amazon, just at the time when I, as an investment analyst in real life, criticized Amazon for allowing bad reviews for products ;)
In one of the other replies, you spoke about not enoug manpower to handle all the incoming reviews. That may be true, but tell me: is it so hard to create a script that will automatically flag all low-rating reviews for personal attention while chack 4+ star reviews for profanities only and then publish them immediatelly? Otherwise, how do you explain that out of 11 products I gave a negative review as the first customer, five have never appeared on the site and the other six appeared with a 1-2 weeks delay? How do you explain that the turnaround for any positive review of mine was below 24 hours? Having written over 300 reviews so far, I have some more recent statistics to show.
Don't get me wrong: I don't blame Amazon. As I indicated at the beginning: censoring out negative reviews at Amazon is a very sound business decision, and I would be very surprised if Amazon wasn't doing it. Thus, trusting the rating at Amazon is foolish, and nobody with a genuine interest in a good purchase should be doing it.
I'm wondering if this Gail Cooke is a real person at all. Remember David Manning of The Ridgefield Press? That was the Make-Believe Movie Reviewer that Sony created to give their films glowing reviews.The internet breeds such things.
...
I am reminded of a section of George Orwell's 1984 where Whinston Smith (the protagonist) has to rewrite a bit of history and decides to create a dead hero to be the subject of Big Brother's rant instead of the commitee long since dismantled.
"What was needed was a piece of pure fantasy. Suddenly there sprang into his mind, ready made as it were, the image of a certain Comrade Ogilvy, who had recently died in battle, in heroic circumstances. There were occasions when Big Brother devoted his Order for the Day to commemorating some humble, rank-and-file Party member whose life and death he held up as an example worthy to be followed. Today he should commemorate Comrade Ogilvy. It was true that there was no such person as Comrade Ogilvy, but a few lines of print and a couple of faked photographs would soon bring him into existence.
Comrade Ogilvy, who had never existed in the present, now existed in the past, and when once the act of forgery was forgotten, he would exist just as authentically, and upon the same evidence, as Charlemagne or Julius Caesar.