Slashdot Mirror
Does Software Need a Siskel and Ebert?
theodp writes "Over at Scripting News, Dave Winer laments the lack of serious software reviews in the NY Times. That wasn't always the case, recalls Dave. 'When they started doing software reviews in the early '80s it was with the usual Times flair,' says Winer. 'But somewhere along the line they stopped taking tech seriously. It's as if they would only review Saturday morning television shows. How could television like The Sopranos or Breaking Bad take root in the culture if there was no criticism that discussed it? Yet that's where we are today with software.' So, does software need a Siskel and Ebert (or A.O. Scott and Manohla Dargis for you highfalutin NYT readers!)?"
Software is everywhere.. systems are too disimilar. The fact that Mac OS != Windows alone without including Linux means this task is Herculean. We do have people who review software more seriously but their in more specilized formats. If you want to do something more open and with a wider target audience like S&E then I don't see how it could work with Software
Just another second banana
Most reviews are shills ... companies have whole departments dedicated to getting bloggers to post sham reviews ...
No. Siskel and Ebert rocked don't get me wrong, but we have a thing now called the Internet and google which can pretty well give you any info you want on most software out there. Anything in a newspaper or magazine is going to be influenced by $$$ anyway while the Internet is typically pretty damn raw
He declared games to be even more creatively bankrupt than movies, and came up with the Boulder Pledge. ("Under no circumstances will I ever purchase anything offered to me as the result of an unsolicited e-mail message. Nor will I forward chainletters, petitions, mass mailings, or virus warnings to large numbers of others. This is my contribution to the survival of the online community.")
The really funny thing to me is that computer games are pretty much the only sector of software with something even approaching a regular review/rating system, and they have long acknowledged that their "Roger Ebert" is either not writing reviews or hasn't been born yet. For other software you have to rely on advertisements disguised as reviews in PCMag et al.
Perhaps Statler and Waldorf...
In the 1980s I did not go to the NYT for software reviews, I went to Byte and other serious magazines for that information.
Time Bomber the Book coming soon.
I knew that the future of reliable reviews at the NYT was over when David Pogue gave MS Word 6.0 for the Mac a good review. It's almost universally seen as one of the worst software upgrades in history.
I emailed him and told him I was disappointed.
I think you're trolling, but as a FOSS advocate myself, I wish you were wrong.
It's much like the problem of racism. After an advocate sees enough incidents of racism, every decision they don't like is suspected of being racial discrimination. Similarly, the myth that "you get what you pay for" is so pervasive that FOSS is often discriminated against, and there's a lot of money aimed at keeping it that way.
FOSS advocates like myself often suspect a bias in bad reviews, partly because we've seen companies like Microsoft pay their shills to bash FOSS, and partly because even honest reviewers don't have any investment in the software they get for free. They'll often dismiss it at the slightest problem when a paid-for product would get a second chance. There's also the familiarity bias, where the latest version of a program will be rated highly because the reviewer's already familiar with older versions, but an alternative has slight differences that the reviewer doesn't understand. While the two packages may be equal to a new user, the reviewer will rate the one they're most familiar with higher. Since FOSS usually has a minority market share, this bias is often against it.
The best way to avoid the rabid hordes of FOSS advocates is to have a professional writing style. Before writing any reviews, show a history of technical knowledge and a willingness to thoroughly examine everything new. In the reviews themselves, explain where and how you got the software (disclosing any conflicts of interest), and preferably also document how much prior experience you have with that program's other versions and competitors. In short, show us that you acknowledge your own faults.
You do not have a moral or legal right to do absolutely anything you want.
It's absolutely impossible to do real software reviews of many software products without risking getting sued. This is due to the industry using NDA's for software that prohibit unapproved reviews. NDA's are why on release day you will all of a sudden see a plethora of reviews on release day. Reviews off of sites like Amazon are largely worthless due to the sheer number of shills and the most popular reviewers getting large quantities of merchandise for free.
One merely needs to look at what happens with video games to know why. If you work for a video game magazine and give a scathing negative review you won't get selected to review the next product from that publisher. After a while you end up being unemployable as video game reviews have to be ready for release day. It doesn't take long to realize you have to carefully write about a game without pissing off the publishers. The net result is that pretty much every game review web site effectively becomes a shill for the publishers as they can't afford to miss out on day zero releases.
Take your favorite site and select all their reviews and put them on a bell curve. Most (average) software should fall somewhere in the middle of their scale. In practice you will find many sites will give average reviews of a 7 or 8 on a 10 point scale. An honest site will fit the bell curve, a dishonest site will quickly be exposed by the bell curve distribution being shifted towards better scores. These problems are why some sites make claims about refusing to sign NDA's, they are showing that they have more integrity to give honest reviews.
This can even extend through to things like operating systems where many beta or rtm releases have excluded the right to review the product without approval in exchange for getting an early release. One simply needs to review the history of Operating System releases to see the effect of reviewers that are afraid to piss up companies. Look back at Windows Me, Vista, Mac OS's before 10 and so on and you can find a plethora of initial approving reviews (ZD Net in particular comes to mind).
The problem gets even worse with actual commercial software. Read your fine print from Oracle or any other commercial product and you will almost certainly find the license prohibits benchmarking and other similar activities that could be used for a review - especially for trial versions. In addition to license issues the cost for commercial software makes it unfeasible to purchase.
Trying to review enterprise class software becomes even more unfeasible as you can't simply install it. In order to properly set it up you need a consultant who knows the product fairly well and that is cost prohibitive for a company that isn't even going to use it. Since enterprise software tends to include language in the EULA that prohibits unapproved reviews no consultant, who naturally depends on having a good relationship with the publisher, is going to help you if you might say critical things about it.
So how do you get a real review of a product that your considering investing a lot of money in? Go to a conference or users group for the software, find an admin who's been using it and take them out to a nice dinner for an off the record review of how the product actually works.
Part of the problem is that a critic can sit down for two hours watch a movie and write a meaningful review. This is not possible when it comes to software.
Let me use a real life example: I was an early proponent of Java since my first few interactions with it in 1994 were positive. Only when I was is deep in the bowels of the beast did I start to see the problems: flawed parameter passing model, the "everything has to be an object" religion (which ironically is violated by built in data types), the "you must write a preamble bigger than COBOL's to have a well designed piece of code", the horrible graphics library that if first shipped with, etc.
After that I realized that maybe moving to Java is not such a good idea after all. I think the popularity of C#/Haskell/Scala/C++11/Python are a result of this realization.
It's more like a vendor telling their client FUD myths so the FOSS option is never considered. Then the vendor only has to compete against their closed-source competitors (if any), and this FUD usually comes from the closed-source leader.
As a personal example, I did some IT work for a small radio station. Our studio computers (to be used by hosts while recording shows) had simple requirements, boiling down to "a web browser and a text editor". A vendor representative told me outright that my plan for Linux desktops was unsuitable, because supposedly Linux can't run web browsers, only servers. He then offered to sell me Windows 7 Ultimate. That's the sort of manipulation that occurs daily, keeping the "FOSS isn't a serious option" perception in the purchasing public's mind.
You do not have a moral or legal right to do absolutely anything you want.
Instead of Siskel & Ebert maybe we need more of a Mystery Science Theater 3000 for software...
With the exception of entertainment and the rare 'culturally relevant' application, the vast majority of software is primarily a tool to get its job done, rather than an item of artistic merit in its own right. The New York Times reviews are — for the most part — cultural reviews; they're not the appropriate venue for most software reviews.
With that said, there are those exceptions where one can speak about the artistic or cultural merits of a piece of software, and my strong impression is that the Times has never really stopped speaking about those. The difference between the '80s and today is that at that point, there was so much less understood and so much more that was new in the world of software that a lot of what came out was of cultural relevance and worth talking about on those merits.