Slashdot Mirror
How Should One Review a Distribution?
Chilliwilli asks: "Why are are good distro reviews so few and far between? Every review I've read recently seems to follow this unoriginal pattern. Big cheers about a nice easy graphical install followed by one or two driver problems blamed on hardware manufacturers. Then the rest of the review seems to be everything worked out of the box. Menus contained usual items. Software versions are X, Y and Z. See OSNews for many examples of such reviews. From the reviews I've currently read all distros seem pretty much the same, is there a reliable source for interesting, impartial and full reviews? Are there any guidelines for distro comparisons? What should people really be looking at when reviewing a distribution? I guess the broader question is what sets distros apart?"
Well, from the perspective of a new or inexperienced user, most distributions are more or less the same. It doesn't really matter what kernel you have, or what KDE you're running to such users; they don't really look at what people who are more familiar with Linux might look at. Most mainstream news sources are run by people who are relatively inexperienced with Linux distributions--hence shallow reviews.
On a side note, what sets distributions apart is different for every experienced user, I think. For example, I run Linux on my desktop, because I'm a hobbyist. As such, I constantly remove and install lots of different packages, and so Debian happens to be the right distro for me. Apt-get allows me to search through a huge archive of binaries and install fun things, then remove them cleanly because of reverse dependency checking. Gentoo, on the other hand, is right for the user who has more time than I do to play around, and wants the very latest versions of software, as well as the coolness factor of having compiled it oneself. Fedora or Mandrake is targeted towards the user who has -less- time than I do to play around, and just wants a nice, workable system right out of the box. Basically, there's nothing that sets distributions apart for everybody. It depends on your specific needs.
Then, you can find out what factors might be important to that target group. Say, you're reviewing distros for Joe Noobie. Using this, you might concentrate on things that might be important to that class of user. (How to get up and running. Such as, where can the distro be obtained? Is it downloaded, purchased, or does it come on a computer you can order? What's your prior experience with this distro, if any?) Then, you would concentrate on things that your class of user might want to accomplish. (Email, text messaging, browsing, watching movies, downloading and properly installing spyware, to make their computer suck, making them feel right at home, Windows-style, etc.)
Finally, to make the review interesting, different, and thought provoking, I would detail the steps I took to get form point A to point B in the review (special commands you might have had to type, or insights you have on how to get something done), and explain it in such a way that will encourage feedback, further experimentation, other reviews, and maybe even (hopefully) improvements in the product.
Well, in all truth, after you are done with the pretty installer, and you have updated the applications you use to current versions, the biggest difference between distributions is the packaging system and custom graphical admin tools provided by the distro. To a certain extent, Linux is Linux is Linux. This is why developers can write one program that will run on most any distro.
To properly review a distribution probably takes longer than most people who do such reviewing have time for. If you need to write something in three days, you've got time to install a distro, but not enough to fuck with it for three months and see how easy it is to keep it running and happy when you are adding weird custom shit, new versions of important system files, and applications that the distribution vendor never intended to integrate.
I am distro-shopping myself right now. Not sure what I'll do.
First off, I like testing a distro a few weeks after it was released, to see what the update procedure is like.
I also like to see you easy it is to install "non-standard" software, such as MP3 players on RedHat, etc.
But all in all, the only real way to do a review is the way the car magazines sometimes do: run the distro for a few months as your main machine. Then all problems will become clear.
Another caveat is to have more than one person review; for example, a Macintosh user may expect a computer to work much differently than a Gentoo user would. Many different opinions need to be taken into account; it is unlikely that there is a one-size-fits-all in the distro department.
Fellowship 9/11
Instead I'd like to see reviews that focus on how easy it is to administrate the system. Is there a wide variety of prebuilt packages? Are they easy to install? If I'm new to linux, what tools are on the system to assist me? What hardware does it support? Those sort of things.
Prevent email address forgery. Publish SPF records for y
It's pretty much in who the distro is aimed towards. You have to review a distrobution based on how well it fits into the the demographic at which it is aimed. To review Gentoo and compare it to Mandrake wouldn't be a fair comparison to either one. Mandrake, Redhat/Fedora, and Suse are aimed more towards the mass consumer market. These distros should have a few qualities:
1) They should be easy to set up.
2) They should work as close to 100% out of the box as possilbe
3) Their inner workings should be nearly invisible to the regular person
Basically, these distros should be compared to OS X and Windows on their installation and hardware detection. It should drop you into Linux easily, and with a fully working machine within an hour.
Next, you have the more hardcore user that you are aiming at with distros like Gentoo, Slackware, and Debian. These tend to be a little more difficult to set up (in comparison to the previous group). If a regular person picked up a copy of one of these at CompUsa and went to install it, they probably will be scratching their heads a bit, and they also probably will get some sort of error. The goals of these distrobutions tend to be the same, yet with a much higher emphasis on the customization factor. That's pretty much the tradeoff a distrobution makes. Ease of use vs. Customization. As one goes up, the other tends to go down. That's what makes Linux great. It's the fact that I can control how exactly how my machine is set up. Either I choose to do it all on my own, or I choose to let someone else decide for me what is installed on my machine and how it is configured. I have installed Mandrake, Fedora, Debian, and right now am waiting on the compiling of my first ever Gentoo install. I think each has their own sets of plusses/minuses and I recognize that.
(Yes, I also realize that each one of these distrobutions has various "flavors" that break the stereotype of that distrobution, such as live cd's, etc.)
Comment removed based on user account deletion
I'd like the review to include whether or not it's strictly free/open software of dependant on proprietary components.
...flavours of linux. There are philosopical differences, but I would guess most people dont give a flying fuck... my analogy runs thusly:
:)
If you prefer a restaraunt meal, served up by chefs with their own ideas; you pick redhat or suse or whatever...
If you like to potter in the kitchen with "meals in a bag, just add vegetables", then use Debian...
If you like to spend 4 hours at the market choosing meat, vegetables and spices to cook your own killer meal to your taste; pick gentoo (gee, guess my bias
If you are a survivalist or a mad hippie who likes to farm it, grow it and kill it yourself; then slackware is the choice for you; this used to by my distro of choice...
But, like food, you need to try it yourself to see if you like it... reviews rarely help unless the reviewers come round and tear your tasting apparatus out of your head and jam it in their own before they go off to eat...
just my pointless $0.02...
err!
jak
"To properly review a distribution probably takes longer than most people who do such reviewing have time for."
Amen. Most distribution "reviews" are one page praise pieces these days, written by people who honestly have no qualifications in the first place to write a distribution review.
Could you imagine reviewing Windows, Office Super-Deluxe, and a hundred other bits of major software in a _day_? Of course not. You'd have to spend WEEKS. Yet, lo and behold, the majority of idiot reviewers do the install on a single machine, blame the distro for anything that goes wrong, and then go nitpicking (or, alternatively, ignore all flaws and praise the distro anyways because they use it). What happened to the rest of the damned review?
I'd also like to see some relatively unbiased reviews. For the love of G-d, please do not write a review if you're in love with the system in the first place, because you use it on your personal box. It just ends up as a piece of evangelism that wastes the three minutes of my precious life.
To summarize:
1. Limit the scope of the review to:
A. Certain users (and do proper and formal usability tests with them).
B. Certain pieces of software within the distribution (but be certain to test them thoroughly!). If this means you limit it to the installer and certain generic OS tasks, than so be it.
2. Make sure you are _qualified_ to write the review. This should involve some formal educational background in usability engineering at the very least. No one's interested in uninformed opinions.
3. Don't review the distribution you use and love. Your review will be hideously biased, whether you try to make it fair or not. Example: This is the primary reason why all Gentoo reviews seem to gloss over the horrifying install (in my experience).
4. On a similar note, give every distro a fair shake. The fact that it doesn't work just like your favorite distribution should not be a point against it. I'm sick and tired of hearing "but it doesn't have apt-get, so it sucks".
5. Avoid absolutes such as "this is the best" or "this is the worst". Make note of pros and cons, and let the reader decide. You can give recommendations if you want.
6. If you alter the system by installing non-standard software, make note of this (ie, apt on Fedora or SuSE). If you're doing weird configuration, make note of it on the review, too.
7. Thoroughly inform yourself of the features of the distribution, and make note of the fact that you're not reviewing the distribution on 1000 machines at once (if the distribution was intended to scale like that). Example: This is the primary reason why RHN always seems to get bashed in RHEL reviews - people make believe it's just up2date, and miss the extremely useful remote management functionality.
8. Avoid getting into comparison situations. If it's hard to install software, say so, but don't damn Mandrake for not being Slackware (or vica versa).
Those are some things to look for, anyways. Like I said, too many idiots taking too short a time to review far too much.
-Erwos
Plausible conjecture should not be misrepresented as proof positive.
In the software world, it's easier for me to:
In the "other" realm, it's easier to:
For the record, I think you're right.. software SHOULD be easy to use, but I think that there's danger in making things TOO easy -- you trade power for simplicity (the learning curve on Start->Find is much less steep than that of piping stuff through grep, but the grep solution is often much more USEFUL).
S
I have seen quite a few distro reviews, and most of them start with either "the graphical installer is nice" or "there's no graphic installer!".
Debian doesn't currently have a pretty graphical installer...
*sigh*
When will people get this? It's not the lack of a "graphical installer" that keeps people from wanting to install debian. It's seriously not. I don't use debian, and mostly because of the installer, or issues that arise from the installer. But, hell, I installed redhat 6.0. It didn't have a graphical installer. I've installed gentoo on many boxes, from stage 1 to stage 3 installs, and that's certainly not graphical, and I actually enjoyed it. I've installed solaris 5.7, 8, and 9, on a variety of SPARCs and x86 boxen, and survived without blowing my top.
The thing people hate about the Debian installer is that it's annoying as all holy fuck.
Even if you choose the "install the standard system" options, it still asks you fourteen million questions, such that, after a few minutes, you just start accepting the default answer. It wants you to baby sit it, to set up a support group for it and walk it through in 12 steps. I can't even count the number of times it was all "read this page of information", and then, at the bottom, it tells me "i'm going ahead with this", and it's not like you have an option, so why the fuck read the page?
Not to mention the damn thing talks to you as if the computer is sentient! "I'm going to go ahead and try to detect your network", "Would you like me to install the development packages?", "I have a terrible itch right under PCI slot 3, can you scratch it with your tounge for me?"
I know the whole philosophy of debian is built around CHOICE and FREEDOM. But, at some points, just make a fucking decision for the consumer, will you? You're probably not going to alienate any zealots if you just go ahead and autodetect the network adaptor without creating a committee and waiting for someone to second the motion.
Everytime someone brings up the graphical installer vs. the world arguement, the test case is always Debian. It's unfair; Debian's installer sucks for completely autonomous reasons. If you'd like to see a relatively well-done command line installer, look at RedHat 6.x, or even the ansi installers for redhat 7.x, or Solaris' install, which is perfectly functional without being annoying.
~Will
sig?
Does the user come from Windows (9x or NT), from Mac OS (9 or X) or directly from some other Linux distribution. What are the expectations of these different users? Does the user know what a command line is? Etc.
I never saw a review that gave a certain appreciation (or rating) for a certain type of user...
I'm wondering how you think Microsoft could possibly support hardware (SATA) that came out after XP was released on the boot CD. Likewise, I can't use a 2001 Redhat/Debian/SuSE/whatever CD to install onto a SATA drive. Be happy they at least included a way to load the drivers for new/non-standard storage hardware during the intial install.
The only advantage of linux in this case is more frequent updates, nothing more.
And frankly, if it took you 4 hours to install the "service packs," all one of them, you either have no idea what you're doing or a slow connection, neither of which are Microsoft's responsibility.
I'm all for pointing out real problems with MS and Windows, but lets stick to real problems, or we all just look like zealous idiots.