Slashdot Mirror
Where Do You Go for Worthwhile Product Reviews?
An anonymous reader asks: "What's the deal with reviews and product comparisons? My boss wants independent comparative reviews of proxy and web servers to use to make/justify his decision. We all know that what the vendors write about their own (and competitive) products, so I tried searching for 3rd party reviews. I can find heaps of articles on the web telling us how great IIS is or how good Microsoft's Proxy server is, but nothing showing a back-to-back comparison of Squid vs. Sun Java Proxy vs. Microsoft Proxy, and the same for Apache and IIS. What's happening here? Where can I find an honest back-to-back product comparison?"
I was a student at the time. I was interested in race relations, so what better way to get the 'inside scoop' on the whole deal than to see the KKK in action. Know your enemy, and all that. After the rally I sat down with the Grand Cyclops and asked him point blank, 'Why can't I find any good information regarding the superiority of any given race over any other given race?'
Your question reminded me of that, for some reason.
Everyone's use will be different... For production use, I've rarely found independent reviews that test what I want tested, in the conditions I want, doing the same things I'm looking to do.
For your example case, I'd personally test each product in-house, drawing up conditions and test plans ahead of time. If you're planning a significant deployment, vendors will generally supply product for you to evaluate. Sometimes if you ask nicely, too.
Just my two cents... And yes, I get that it may not be feasible. Its labor and time-intensive. But in-house testing and evaluation almost always beats 3rd party reviews, in my book.
Lie about it, and recommend whichever vendor gives you the most kickbacks.
Slashdot - where whining about luck is the new way to make the world you want.
Search in google the product you want to review then add the following phrase ", problems"
:)
I'm sure you would get all the bad side, then weigh which one of the products are the lesser evil
sample query: iis, problems
I've given up searching "$PRODUCT review".
If you're lucky, a magazine will have a comparative review and will have taken roughly equal amounts of ad revenue from each of the competing vendors. Useful search terms include "shootout" and "versus".
Anecdotal evidence from the tech community can be a heuristic if you're wondering about general bugginess and hassle factor. If you need real benchmarks, the only ones that mean a thing are those you run yourself.
Are you running a mixed shop or a single-vendor one? Don't underestimate the pain of interoperability and equipment management hassles if you've never experienced them.
Work as hard as you can to pin down what you need: good scaling on SMP machines? Easy management? Particular features? Good local talent pool for running/fixing it? Low purchase price? Support contracts? The more questions like that you answer, the clearer the choice will be and the easier the web searching will be. "Apache scale SMP OR cluster" is likely to get more informative results than "Apache IIS comparison".
If you are worried about security, then abandon all hope of useful information from the press, concentrate more on lockdown and scheduling updates then on the choice of product (but never install IIS 5), and keep an eye on the news.
Cultivate sysadmins in other places who have environments about your size and with similar needs.
If you go to a related forum you can often post asking about a couple products and often get a reply from someone who used both. Forums can be pretty useful and users can have great expertise. I suspect some manufacturers even let forums do problem solving for them. I got much better advice for a new ASUS mobo from a user in a photography forum then I could get from the ASUS forum. If you go to a forum for webmasters you can get all sorts of advice on servers. I don't trust a lot of published reviews in magazines that take advertising. Recently I was shopping for an LCD monitor and reviews often use wrong specs and don't seem to understand the product very well (they won't even mention if a panel is S-IPS or S-PVA etc). Certain brands seem to get a lot of wiggle room and a look at the ads on the bottom of the page usually shows why. One good site for head-to-head comparisons for monitors is http://www.lesnumeriques.com/duels.php?ty=6&ma1=52 &mo1=149&p1=1606&ma2=36&mo2=105&p2=1041&ph=6 .
I know monitors are not what you asked about but I still think forums are best bet. You may be lucky not many reviews exist because I find its a good way to get hung out to fry.
I can only wish it was that simple. But other than keeping quitting and moving to fresh small startups (which probably don't exactly pay a lot for an admin), it's not really practicable.
And even as a startup, if your work isn't purely developping your own product, you end up doing stuff for various clients. Which have their own ideas set in stone, based on reading some IT-for-retards ragazine or on a golf round with the nice salesman from MS/IBM/whatever. And we all know that you can't trust those techies with their techno-babble speak, whereas a salesman would never tell a lie
An as soon as the company grows past a certain size, and doubly so for companies whose primary product aren't programs or IT services, well, my favourite metaphor is: clue is heavier than air. The higher you go up the hierarchy pyramid, the thinner it gets. If clue were oxygen, you'd see higher level managers blue in the face like they're Smurfs.
To their defense, it's not their job to know the finer points and differences between web servers, but then it also shouldn't be their job to take such low level decisions. So you have a bunch of people taking decisions about stuff that they knew nothing about, and it wasn't their job to know anything about. What really makes it worse is having several layers of shielding against the effects of bad decisions. He made some "strategic decision" to go all-IIS, and can claim credit for any positive results (even coincidental or immaginary), but it's _your_ fault if something goes wrong with it or it takes too long to port your application to it. And whenever such shielding is in place, out goes the incentive to get any real clue or to refrain from taking bad decisions.
But, to get back on topic, you'll find very few large companies where such shielding from responsibility isn't in place. So you're limiting your employment oportunities drastically if you only accept jobs from the few who aren't led by people who don't take their IT info from ads and salesmen.
Probably a more realistic thing to do is realize that, in the end, few things matter _that_ horribly much. Some people have a penchant for blowing minor differences out of proportion, and make mountains out of molehills. There _are_ product issues that matter, and there _are_ awfully bad management decisions, but there's also a lot of stuff which really isn't as critical as the "either something is perfect or it's complete crap" gang makes it sound. If some proxy is 5% faster than another, pfft, it doesn't even start to matter. You'll want plenty of margin for when you get slashdotted anyway, but 99% of the time it'll be _way_ under-used. Having 5% or even 10% less unused capacity isn't the end of the world.
And once you do a realistic assessment of how bad it really is, a lot of things aren't _that_ horrible after all. So management picked a less than optimal proxy. Who cares? Compared to some other decisions I've seen various managers take, this doesn't even start to matter. If you're going to quit a job solely because of something like that, methinks you need to rethink your standards. And maybe look for an OCPD support group in your area.
A polar bear is a cartesian bear after a coordinate transform.
Some things I learned include...
1. The only job a real programmer will take must involve Ruby on Rails
2. Never buy a MS product
3. Filesharing music is "fair use"
4. Programmers should not create closed source programs EVER.
5. Linux sucks, BSD sucks, MacOS sucks, and Windows sucks. (I am posting from an IBM/360)
6. The only safe browser is Lynx