Are Amazon Vine Reviews of Technical Books a Joke?

First time accepted submitter jasax writes "As an Amazon frequent buyer, I rely quite a lot on reviews of the books I want. However, some caution is in order: the (bad) quality of Amazon's reviews and reviewers under the Amazon Vine program has already been news in Slashdot. Today I was shocked by a practical result of that program. This second edition (published in 2012) of a very specialized system identification book has 12 reviews: the oldest (dated 2007) certainly targets the first edition. The remaining 11 reviews are all from 'Vine Reviewers' (VRs). All seem to be ignorant of what 'System Identification in the Frequency Domain' really is. None of the reviews is tagged with a 'Verified Amazon Purchase'; most (if not all) are 'small talk reviews' peppered with technical phrases cloning the publisher's book description, and some of the reviews are ridiculous, to say the least. If this sample of reviewing by VRs really is the norm, then the bottom line is that the Vine program is totally irrelevant and unreliable — at least for technical books."

Comments

They're just useless

[rsilvergun]

how the heck are you going to fit a review of a technical manual in a 6 second video? That's ridiculous.

Oh, _Amazon_ Vine.... Well, carry on then...

Re:They're just useless

[red crab]

I'd say ditto for food reviews, movie reviews etc generally published on magazines and newspapers. They are written by youngsters with just a good command over language who don't actually understand the subject. Or worse the reviewers get sometimes paid to write favorable reviews. In Amazon Vine's case, reviews are sponsored by the seller of the product, you can hardly expect them to be honest in any way.

Re:They're just useless

[K. S. Kyosuke]

how the heck are you going to fit a review of a technical manual in a 6 second video?

You either put on an extremely happy face and oink with delight, or you put on a really scared face and squeal in horror. The viewers will sort it out.

Re:They're just useless

[inflex]

Marketing... that's the crux. My wife is a writer, we've got about 10 books out there ( different pen names ), and honestly, unless you're either really lucky, or you pump the market constantly with a lot of effort you invariably won't float to the top, regardless of the contents of your books ( or products ). Even when you have a good run and make it in to the top-100 rankings on Amazon for a week or two, once you fall out of that it's a fast and long fall back down to irrelevance. We see a lot of other writers pouring money in to other pockets for reviews, promotions, blogs and social-name-dropping, it doesn't always pay off, but if you don't do it you really don't stand much of a realistic chance.

Binding, hardcover, etc.

[Dan East]

I guess it is indeed a warning sign when a reviewer spends more time talking about the physical attributes of a book than the contents of the book itself.

Re:Binding, hardcover, etc.

[MatheoDJ]

Perhaps there are Vine reviewers farming out their writing and receiving free products in exchange. I've been experimenting with Amazon's Mechanical Turk, and a fair number of the assignments seem to be fake reviews of some sort, with improved SEO as the endgame. I've been offered HIT's (Amazon's shorthand for the small tasks often worth as little as $0.01) that require a person to write small articles or reviews, 150-500 words, using incredibly specific technical jargon or product lingo. Sometimes it's Ph.D. level stuff. I can't imagine anyone having the technical prowess and educational background necessary to write intelligently about such subjects being interested in making $0.75 to do so, but maybe that's enough money to inspire someone in Bangladesh to write meaningless positive reviews peppered with jargon, and small enough change to inspire someone in America to outsource a large number of reviews in order to increase their Vine ratings.

Re:Binding, hardcover, etc.

[Chrisq]

I guess it is indeed a warning sign when a reviewer spends more time talking about the physical attributes of a book than the contents of the book itself.

I don't know, someone once told me that a book said "never judge a book by its contents", and it had a really nice cover.

Re:Binding, hardcover, etc.

[StikyPad]

I liked how your post was concisely packed into a single sentence. It took up just the right amount of room on my screen, and I could scroll it up or down using my mouse wheel.

Re:Binding, hardcover, etc.

[hairyfeet]

This kind of crap is why I don't trust Vine reviews. Instead I try to find an ordinary Joe, they are usually easy enough to spot, not the fanciest writers, not all flowery in their prose, but you get an honest review from somebody that bought the thing because they wanted/needed it and are just giving their opinion.

Likewise when i buy stuff there I try to give as straight forward a review as possible and point out any limitations because i have often found that a thing might work really fantastic in one particular role but be just terrible in another. For an example of what I mean one Amazon seller had a load of OEM PCIe graphics cards for dirt cheap. After picking up a couple and slapping them in some boxes in the shop i wrote a nice glowing review that pointed out these cards were GREAT if you had an older system you want to upgrade from XP to Win 7 and needed a graphics card, or if you were building an HTPC that was strictly gonna be a media box? Wonderful for that role, but anybody that thought these cards would be good for gaming anything newer than Portal 1 was gonna be disappointed because these cards just weren't made for that.

So are Amazon Vines reviews bullshit? I personally think so, for me they are useless because too often they come off like a paid ad or as in TFA written by somebody that doesn't truly understand the product, give me just a normal guy that bought the thing to do a job and is just giving his thoughts on it any day of the week. may not be the most flowery of prose but it gives me a much better idea of the pros and cons than the Vines reviews.

Re:Binding, hardcover, etc.

[emt377]

I can't imagine anyone having the technical prowess and educational background necessary to write intelligently about such subjects being interested in making $0.75 to do so, but maybe that's enough money to inspire someone in Bangladesh to write meaningless positive reviews peppered with jargon, and small enough change to inspire someone in America to outsource a large number of reviews in order to increase their Vine ratings.

Perhaps someone out there has a review generator, and they've found a means to automate the site interaction. That could well be worthwhile, or an interesting experiment at least.

Re:Binding, hardcover, etc.

[chipschap]

I have been a Vine reviewer for several years, and I take great care with my reviews. I do sometimes review a technical book, but I make sure it is in an area in which I have sufficient expertise. While I'll review a book about audio engineering, I won't review one about, for instance, organic chemistry.

My reviews are not fake. I make every effort to give a fair evaluation. I write positive reviews and negative reviews (even though it's well known that a reviewer's ranking on Amazon goes down with negative reviews, as they are generally not well received by readers).

I don't know if I'm typical of Vine reviewers; the ones I've interacted with on the Vine forums seem about the same as me. Of course, there are bound to be bad apples in the group. In addition, Amazon has been making review requirements for Vine members ever more strict, and this undoubtedly has lead to decreased quality. But I do sincerely hope things are not quite as bad as portrayed here, or the program should indeed be discontinued.

The bigger issue, reviews of niche products...

[antifoidulus]

I think the bigger issue here is the usefulness of niche products, no matter if they are Amazon Vine or whatever. The size of target audience for this book is MAYBE in the 10s of thousands, and there are probably what, only 1 or 2 other books that would really even be considered "competitors" to this book. With those parameters, are any Amazon reviews going to be all that useful?
You are almost certain to get a review from someone not in the target audience or who got over-ambitious and ended up not understanding the source material, a review from the author's brother in law just so it looks like people are buying etc. Any sort of useful review is going to probably come in the form of a long blog post/magazine article, and thus isn't likely to be present on Amazon....So what you are left with is someone commenting on how flexible the book's spine is.

yeah it's a joke

[Anonymous Coward]

Reviews need ratings so they can be flagged as spam.

Re:yeah it's a joke

[faffod]

Amazon has a rating system for reviews. Sure you can't flag something as spam, but that would be abused by the tolls so it's a balance they have to strike. Also, Amazon allows for comments to be added for reviews. With so few reviews on this book, it should be quick enough to make the reviews as not helpful and add in some cases add a comment on the egregious ones. In other words, the review system is only as good as the average of the reviewers - get some good reviewers to balance out the bad ones and it works well enough.

Amazon needs their head read.

[CuteSteveJobs]

Their books reviews on Amazon proper could be a great resource for anyone deciding whether to get a book, but they're full of crap reviews like "FAST SHIPPING! RECOMMEND THIS SELLER! A+++++" or "COVER DAMAGED AND ARRIVE 2 LATE 4 CLASSES. ME MAD!" They have a system where you can vote down these reviews, but why not just tell lamers "This isn't eBay" Problems been pointed out to them by many but they have twiddled their thumbs while their database has filled up with crap.

So they should fix what they already have before launching new programs like Vine. Giving favored reviewers free crap is hardly going to inspire independent reviews anyway.

Re:Amazon needs their head read.

[lxs]

Also, I've noticed that some sellers have "shipping estimates" that they don't always adhere to.

Do you know what an estimate is? hint: It doesn't mean "guarantee" or "goal".

Re:Amazon needs their head read.

[GryMor]

If I'm looking at a book, the reviews had better be about the book rather than one of the 100+ sellers of the book. If you want to review that particular seller, there is a separate spot for that, and low enough service ratings will tank their visibility across all of their products. "Cover damaged, terrible service" on a review of Excession is useless spam. The same review on the seller itself is helpful.

Re:Amazon needs their head read.

[Rich0]

They really need to fix the GUI for submitting reviews so that you're prompted for both at the same time, and it is made clear which pertains to the product vs the seller with more specific questions to answer.

Read the "bad" reviews

[oldhack]

Read the three- and two-star reviews. They give you both good and bad. Large part of five- and one-star reviews are crap.

A++

[Anonymous Coward]

Excellent article. Would read again.

Its astroturf on a vine

[Bob_Who]

absolutely no cred...

Physician, heal thyself

[gsiarny]

Is this really a meta-review which indicts an entire system based on 12 reviews of one book?

If only the summary had reported that the reviews relied too much on anecdotal evidence...

Easy Shill Detection

[ReallyEvilCanine]

I expect someone will come up with a collection/extraction script to list all the names of these shill and paid "Vine Reviewers".

well...

[Ryanrule]

Amazon is walmart. Proceed with that information.

Re:well...

[kamapuaa]

You mean they sell a lot of different things? What point are you making?

Never?

[dbc]

Never shopped at Amazon? I find that more astounding than finding a honest man in congress. I'm just curious as to why... Is it a political statement? You live where shipping would be outrageous? You spend too much time on Slashdot to have any time to shop?

I remember when I made my first Amazon purcahse. It was some O'Reilly Nutshell book, I can't remember which. I had to install a web browser on my workstation at work to do it. I got the latest and greatest web browser available -- Mosaic -- which I downloaded as a tarball from UI and did ./confgure;make all. Amazon has been around for a while... finding someone who has never bought anything from them is both amusing and amazing.

Vine rules require reviewing all products

[Anonymous Coward]

As I had no idea what the "vine reviewers" was about, I read the amazon definition. And it says that: "Once they are invited into the program, to maintain their status as a Vine Voice, they must review *all* of the selected products within 30 days of receipt...".

So once in the program you get a nice flow of interesting free books related to your topics of interest. But apparently you *must* enter a review for each book that Amazon sends you, or the flow of books stops. The outcome is obvious - particularly for technical books that not every reader will understand. But even for normal books, people can sometimes just be busy - resulting in rushed/content-free reviews.

So not the reviewers' fault really, just stupid rules from Amazon. Requiring reviewers to review *half* of the books sent would probably produce a better outcome..

Re:Gaming the system

[jtownatpunk.net]

Shenanigans! Posting on an internet forum but never shopped at Amazon. [cough]bullshit![/cough]

Re:Gaming the system

[Rich0]

These are Vine reviewers. They all DID use the product, in the sense that they were mailed a free copy of the book to review. The problem is that they're just random people - it isn't like Amazon has a Vine review program for PhDs in electrical engineering.

This is like asking a chemist to review a dynamometer or asking an auto mechanic to review a mass spectrometer. No useful information is likely to come out. If you're interested in reviews of such products you'd prefer them to come from experts in the field who can either spec to the details of their performance or at least how well they work in daily use.

What you should know about Vine reviewers

[Anonymous Coward]

by a Vine reviewer (posted anonymously due to my Vine NDA terms).

I was invited into Vine several years ago. Since then, I have received about $10000 worth of merchandise for review. The items I've received have ranged from Post It notes and gel pens to a high-end DSLR and some decent hardware (NAS, external and internal hard drives, mice, keyboards, routers and printers)

(1) Each month, Vine reviewers are offered two chances to select items to review. The first list is a short one (10 to 20 items) that Amazon descrives as "targeted" to members. The amount of tampons, diapers and hair care products that have been offered to me (a middle-aged bald male) suggests the targeting is *very* general. Targeted items are offered to multiple members (but not universally), and the high end items are gone in seconds.

(2) A week later, all of the non-selected items remaining from the targeted list are offered to the entire Vine community. Same rapid response to high end items takes place. You have to navigate hundreds of choices instead of dozens.

(3) The demographics of who were selected are all over the place.
--Some had long histories of purchasing from Amazon before being invited into Vine with both many and few (or no) previous reviews.
--Some had short histories of purchases with amazon.
-At the time I was selected, I had reviewed about 40 products (all most exclusively computer hardware or related literature) over a seven-year period
--Attempts by the Vine community to determine the total population of Vine reviewers (both on amazon-hosted Vine forums and in informal groups away from the amazon mothership) suggest the Vine reviewer population can be counted in hundreds (probably the low hundreds)
--I have some informal relationship with Vine members away from the mothership. While the sample size is WAY small to be meaningful, geographically we seem to be distributed consistent with the US population distribution in general. Generationally we seem to be mostly Boomers, but there are clearly a few Gen Y and millenials sprinkled in. Some of us clearly have high incomes, and some of us clearly do not.

(4) Originally Amazon required that we submit reviews on 80% of all items we selected.

(5) This year, Amazon changed the requirement. We must now review 100% of the items we select within 30 days of receipt. We can request a 10 day extension (and can do so 10 times per year).

(6) We know vendors pay Amazon to have items made available to Vine reviewers. We *think* this costs them around $5000, but this is based on a single data point from a Vine member who worked for a company that took a pass on paying that amount to be included in the Vine selections.

(7) For Vine reviews, the "Customer Review from the VIne Program" is essentially equivalent to an "Amazon Verified Purchase". Except that the reviewer received it at no cost...and is now obligated to review it within 30 days.

I can't speak for others, but I value my credibility as a reviewer, my % of helpful votes and my amazon reviewer ranking. Because of this, I never select books about technology issues (or technology items) that I do not already know something about (or am in the process of developing knowledge of). Similarly, I don't select or review genre fiction, since I don't read this for pleasure.

Yeah, some Vine reviews suck. So do some non-Vine reviews. As with ANY online resource, you always have to apply your own filters to help gauge the credibility.

And yes, I have happily awarded 1 and 2 star ratings to Vine products that sucked (and 4 and 5 star reviews to products that kicked ass).

Here is my extraordinary Joe

[justthinkit]

Dr. Lee D. Carlson

Why yes, Cardinal, I did glance at you

[Hognoxious]

When one is called "100 Best Christian Cake Recipes" and another is "Butt-plugs and trap-ons: the definitive guide" you mostly want to keep your markets separate.

Note that I said mostly.

I'm a Vine reviewer - there's a flaw in the system

[Skynyrd]

If the subject is complex enough, the reviewer may not understand the book, and be forced to give it a superficial review.

Amazon is making the problem worse. When I first started writing reviews for Vine, I had to review 75% of the items I received in order to be able to receive additional items. Seems fair. Then it was raised to 80%, and was recently raised to 100%.

If I don't review it in 30 days, I am barred from ordering another item until I review it.
So, if somebody orders a book, and finds it way over their head, they still have to review it. Now we have a problem.

Guilty as charged

[wezelboy]

I've totally written crappy vine reviews. Quite a few in fact. I'm sorry.

The bottom line is when I get my Vine newsletter, most of the time all of the stuff that I might consider buying/reading is already gone. I don't know if Vine has 'favorites' program where certain people get access to items before others, but most of the time Vine is a bust for me. I'm left with the items/books that are leftover- which often means technical books. I'll get these things if I have an interest in the topic, but I just don't have the time to delve into the finer points of signal processing. It will sit on my shelf and gather dust until I find something I want on Vine that is actually available, then I have to write a review to be able to get that thing.

I've pretty much given up on Vine. It's a waste of time, and not worth the guilt of writing a crappy review. Even if you write a heartfelt review, people will mark it as unhelpful anyway. Call me jaded.