Slashdot Mirror
How Should One Review a Distribution?
Chilliwilli asks: "Why are are good distro reviews so few and far between? Every review I've read recently seems to follow this unoriginal pattern. Big cheers about a nice easy graphical install followed by one or two driver problems blamed on hardware manufacturers. Then the rest of the review seems to be everything worked out of the box. Menus contained usual items. Software versions are X, Y and Z. See OSNews for many examples of such reviews. From the reviews I've currently read all distros seem pretty much the same, is there a reliable source for interesting, impartial and full reviews? Are there any guidelines for distro comparisons? What should people really be looking at when reviewing a distribution? I guess the broader question is what sets distros apart?"
One shouldn't.
Seriously, at the present time there is not much reason for Slashdotters to read linux distribution reviews. The distributions vary mainly in philosophy, *not* in software.
Anyone who cares already knows the basic differences between Debian, Fedora, Gentoo, et cetera. The only other details are what software is installed by default. But who leaves the default install in place? Even Windows users install and upgrade software.
I have seen quite a few distro reviews, and most of them start with either "the graphical installer is nice" or "there's no graphic installer!".
Debian doesn't currently have a pretty graphical installer but I find that it's not necessary because I never need to reinstall, and it's functional and works over a serial console.
I know people who refuse to even give Debian a try because the installer (and the base install) isn't pretty enough.
Anything is possible, except skiing through revolving doors.
Well, from the perspective of a new or inexperienced user, most distributions are more or less the same. It doesn't really matter what kernel you have, or what KDE you're running to such users; they don't really look at what people who are more familiar with Linux might look at. Most mainstream news sources are run by people who are relatively inexperienced with Linux distributions--hence shallow reviews.
On a side note, what sets distributions apart is different for every experienced user, I think. For example, I run Linux on my desktop, because I'm a hobbyist. As such, I constantly remove and install lots of different packages, and so Debian happens to be the right distro for me. Apt-get allows me to search through a huge archive of binaries and install fun things, then remove them cleanly because of reverse dependency checking. Gentoo, on the other hand, is right for the user who has more time than I do to play around, and wants the very latest versions of software, as well as the coolness factor of having compiled it oneself. Fedora or Mandrake is targeted towards the user who has -less- time than I do to play around, and just wants a nice, workable system right out of the box. Basically, there's nothing that sets distributions apart for everybody. It depends on your specific needs.
Then, you can find out what factors might be important to that target group. Say, you're reviewing distros for Joe Noobie. Using this, you might concentrate on things that might be important to that class of user. (How to get up and running. Such as, where can the distro be obtained? Is it downloaded, purchased, or does it come on a computer you can order? What's your prior experience with this distro, if any?) Then, you would concentrate on things that your class of user might want to accomplish. (Email, text messaging, browsing, watching movies, downloading and properly installing spyware, to make their computer suck, making them feel right at home, Windows-style, etc.)
Finally, to make the review interesting, different, and thought provoking, I would detail the steps I took to get form point A to point B in the review (special commands you might have had to type, or insights you have on how to get something done), and explain it in such a way that will encourage feedback, further experimentation, other reviews, and maybe even (hopefully) improvements in the product.
Well, in all truth, after you are done with the pretty installer, and you have updated the applications you use to current versions, the biggest difference between distributions is the packaging system and custom graphical admin tools provided by the distro. To a certain extent, Linux is Linux is Linux. This is why developers can write one program that will run on most any distro.
To properly review a distribution probably takes longer than most people who do such reviewing have time for. If you need to write something in three days, you've got time to install a distro, but not enough to fuck with it for three months and see how easy it is to keep it running and happy when you are adding weird custom shit, new versions of important system files, and applications that the distribution vendor never intended to integrate.
I am distro-shopping myself right now. Not sure what I'll do.
First off, I like testing a distro a few weeks after it was released, to see what the update procedure is like.
I also like to see you easy it is to install "non-standard" software, such as MP3 players on RedHat, etc.
But all in all, the only real way to do a review is the way the car magazines sometimes do: run the distro for a few months as your main machine. Then all problems will become clear.
Another caveat is to have more than one person review; for example, a Macintosh user may expect a computer to work much differently than a Gentoo user would. Many different opinions need to be taken into account; it is unlikely that there is a one-size-fits-all in the distro department.
Fellowship 9/11
Instead I'd like to see reviews that focus on how easy it is to administrate the system. Is there a wide variety of prebuilt packages? Are they easy to install? If I'm new to linux, what tools are on the system to assist me? What hardware does it support? Those sort of things.
Prevent email address forgery. Publish SPF records for y
It's pretty much in who the distro is aimed towards. You have to review a distrobution based on how well it fits into the the demographic at which it is aimed. To review Gentoo and compare it to Mandrake wouldn't be a fair comparison to either one. Mandrake, Redhat/Fedora, and Suse are aimed more towards the mass consumer market. These distros should have a few qualities:
1) They should be easy to set up.
2) They should work as close to 100% out of the box as possilbe
3) Their inner workings should be nearly invisible to the regular person
Basically, these distros should be compared to OS X and Windows on their installation and hardware detection. It should drop you into Linux easily, and with a fully working machine within an hour.
Next, you have the more hardcore user that you are aiming at with distros like Gentoo, Slackware, and Debian. These tend to be a little more difficult to set up (in comparison to the previous group). If a regular person picked up a copy of one of these at CompUsa and went to install it, they probably will be scratching their heads a bit, and they also probably will get some sort of error. The goals of these distrobutions tend to be the same, yet with a much higher emphasis on the customization factor. That's pretty much the tradeoff a distrobution makes. Ease of use vs. Customization. As one goes up, the other tends to go down. That's what makes Linux great. It's the fact that I can control how exactly how my machine is set up. Either I choose to do it all on my own, or I choose to let someone else decide for me what is installed on my machine and how it is configured. I have installed Mandrake, Fedora, Debian, and right now am waiting on the compiling of my first ever Gentoo install. I think each has their own sets of plusses/minuses and I recognize that.
(Yes, I also realize that each one of these distrobutions has various "flavors" that break the stereotype of that distrobution, such as live cd's, etc.)
Comment removed based on user account deletion
A review should show how the distro fits comparing it to where its suppoed to go.
You should look at how the distro follows the Linux way (or the Unix way). For example, look for the odd little things that someone added because they thought it was a good idea. Maybe it was or maybe it isn't. For example every major distro now aliases "rm" to "rm -i" which isn't the unix way (at least according to Kernighan). The real unix way is alias "del" to "rm -i" and teach people to use del if they want to be asked so they don't learn bad habits.
Another thing is does the keyboard short cuts work? If I have a windows theme, does the keyboard work that way and if I select a mac theme, will it work that way too? Can I mix and match so it looks like Windows and has mac bindings? Is there clear help showing new keyboard options if I pick a better theme?
Remember computers are a tool. They are there to serve a useful role. It doesn't matter how nice they look or work if they don't end up getting the job done. After an upgrade, I should be able to get my work done faster however my tests show that isn't going on.
Reviews should reflect the ability of the distro to work as the tool its suppoed to be.
Let's face it. One of the main reasons that the specialty magazines and sites that do these reviews exist is to make people feel so good about their previous "purchase" that they'll "buy" more. It was true of the old car (and gun) magazines and is true of the computer mags and sites today. Linux, open source, are being merchandized just as aggressively as the sports car was years ago. So it should surprise no one that there's 'nary a negative word to be said about a particular software product being reviewed, whether an O/S or an application. Although not as easy to navigate as a magazine review, the various mail lists set up to support particular distros are probably the best source of info on them. After browsing the archives awhile for comments, problems and solutions you can get a pretty comprehensive picture of what's what. Of course it also helps to have a few junk boxes around that you can load up with the latest release from each publisher to experience the thing for yourself. *That's* a tradition that comes right out of the beginnings of the personal computer movement...
That's just one axis too. You might have someone who wants to set up a web server but is a complete computer newbie (you wouldn't recommend them to use Slackware.) Or you might have someone who wants to use a desktop who is an elite hacker (you wouldn't recommend them to use Mandrake.)
So there is the axis of purpose, and the axis of experience. And that's just a start. A certain distribution might be perfect for the purpose, perfect for the user's level, but require money.
And so on, and so forth.
Karma: It's all a bunch of tree-huggin' hippy crap!
I'd like the review to include whether or not it's strictly free/open software of dependant on proprietary components.
...flavours of linux. There are philosopical differences, but I would guess most people dont give a flying fuck... my analogy runs thusly:
:)
If you prefer a restaraunt meal, served up by chefs with their own ideas; you pick redhat or suse or whatever...
If you like to potter in the kitchen with "meals in a bag, just add vegetables", then use Debian...
If you like to spend 4 hours at the market choosing meat, vegetables and spices to cook your own killer meal to your taste; pick gentoo (gee, guess my bias
If you are a survivalist or a mad hippie who likes to farm it, grow it and kill it yourself; then slackware is the choice for you; this used to by my distro of choice...
But, like food, you need to try it yourself to see if you like it... reviews rarely help unless the reviewers come round and tear your tasting apparatus out of your head and jam it in their own before they go off to eat...
just my pointless $0.02...
err!
jak
"To properly review a distribution probably takes longer than most people who do such reviewing have time for."
Amen. Most distribution "reviews" are one page praise pieces these days, written by people who honestly have no qualifications in the first place to write a distribution review.
Could you imagine reviewing Windows, Office Super-Deluxe, and a hundred other bits of major software in a _day_? Of course not. You'd have to spend WEEKS. Yet, lo and behold, the majority of idiot reviewers do the install on a single machine, blame the distro for anything that goes wrong, and then go nitpicking (or, alternatively, ignore all flaws and praise the distro anyways because they use it). What happened to the rest of the damned review?
I'd also like to see some relatively unbiased reviews. For the love of G-d, please do not write a review if you're in love with the system in the first place, because you use it on your personal box. It just ends up as a piece of evangelism that wastes the three minutes of my precious life.
To summarize:
1. Limit the scope of the review to:
A. Certain users (and do proper and formal usability tests with them).
B. Certain pieces of software within the distribution (but be certain to test them thoroughly!). If this means you limit it to the installer and certain generic OS tasks, than so be it.
2. Make sure you are _qualified_ to write the review. This should involve some formal educational background in usability engineering at the very least. No one's interested in uninformed opinions.
3. Don't review the distribution you use and love. Your review will be hideously biased, whether you try to make it fair or not. Example: This is the primary reason why all Gentoo reviews seem to gloss over the horrifying install (in my experience).
4. On a similar note, give every distro a fair shake. The fact that it doesn't work just like your favorite distribution should not be a point against it. I'm sick and tired of hearing "but it doesn't have apt-get, so it sucks".
5. Avoid absolutes such as "this is the best" or "this is the worst". Make note of pros and cons, and let the reader decide. You can give recommendations if you want.
6. If you alter the system by installing non-standard software, make note of this (ie, apt on Fedora or SuSE). If you're doing weird configuration, make note of it on the review, too.
7. Thoroughly inform yourself of the features of the distribution, and make note of the fact that you're not reviewing the distribution on 1000 machines at once (if the distribution was intended to scale like that). Example: This is the primary reason why RHN always seems to get bashed in RHEL reviews - people make believe it's just up2date, and miss the extremely useful remote management functionality.
8. Avoid getting into comparison situations. If it's hard to install software, say so, but don't damn Mandrake for not being Slackware (or vica versa).
Those are some things to look for, anyways. Like I said, too many idiots taking too short a time to review far too much.
-Erwos
Plausible conjecture should not be misrepresented as proof positive.
Most Linux distro reviews I've seen go like this:
To review a distro properly, you need to use it for at least a month, IMHO. You need enough time to discover that security updates are a pain to install. Enough time to find out that installing third party packages is impossible because the distro uses a beta version of GCC.
In other words, you need to give more than first impressions: anyone can do that, and it's not terribly helpful.
Pretend that something especially witty is here. Thanks.
In the software world, it's easier for me to:
In the "other" realm, it's easier to:
For the record, I think you're right.. software SHOULD be easy to use, but I think that there's danger in making things TOO easy -- you trade power for simplicity (the learning curve on Start->Find is much less steep than that of piping stuff through grep, but the grep solution is often much more USEFUL).
S
I'd have to bet Slackware.
:)
That's what I run. If I want something, I usually go to the source, and install what they have to offer, rather than waiting for anyone to do anything for me. Why should I wait for a mainstream distro to patch the hell out of something, and distribute it to me with convoluted configuration options, when I can get it exactly like the author intended it? Slack makes this fairly easy. One of the first things I do when I install a Slackware system, is to download the new kernel sources, and compile it myself. Not that there's anything technically wrong with Slackware's kernel, I just like to know that my system is running exactly the way it should, without any extra drivers, or fluff.
We're building up a distro like this. It's going to be called "LMLinux". There's a really brief overview at http://lmlinux.com . We almost have the first alpha release done, but not quite. Our package management backend is just about complete, which looks rather nice.. Everythings stored in a MySQL database, for easy reference. It will make for very easy work, for anyone to browse to a few lookup pages, or for a package manager to find things.
I'm all about doing things from source, and doing it the way the author intended. But hey, if people like patchwork systems, let them get another distro with a kernel that has various patches reverse-engineered to work in the wrong version, etc, etc..
In our ideal world, you have the option of installing the package from our server, or grabbing it from the author yourself, and installing it yourself, without worrying that the next automagic update will hose whatever you've done.
I've done work for RedHat and Debian people, who absolutely freak out at the idea that I'd even think of downloading something myself, and installing it myself.
With all the complaints I see about x distro not having this, or not doing that, I kinda giggle. My Slackware installs do everything. Farther up in the thread is someone complaining about a lack of NTFS support by whatever distro. NTFS? Mine does it, I compiled it into the kernel, if there will be NTFS drives to read. I watch any video type I want, I've installed xine. It's not rocket science, read the readme.
And, you're absolutely right, you get all the free support you need in the mailing lists for the program, or checking on dejanews (eerr, groups.google.com)
Serious? Seriousness is well above my pay grade.
Does the user come from Windows (9x or NT), from Mac OS (9 or X) or directly from some other Linux distribution. What are the expectations of these different users? Does the user know what a command line is? Etc.
I never saw a review that gave a certain appreciation (or rating) for a certain type of user...
Well, now that I have a debian team member in a thread that I am responsible for, I feel the need to quantify my response.
I mean no personal disrespect. None at all. To be honest, despite my tone in the previous post, I actually like debian once it's running. It's easy to install and upgrade, and I appreciate that. Having to deal with redhat day in and day out, I am so sick of RPM dependancy hell that I've pretty much washed my hands of RedHat, and apt-get (as well as emerge and similar tools) are fantastic and much appreciated.
It's just the installer that gets me.
And my problem is that I work at a webhosting company. I have to set debian up on machines fairly regularly. Not regularly enough to justify setting up a slipstreamed automated install, but regularly enough to be annoyed by it. And it frustrates me even more, because I'm usually setting it up on customer machines, so I get to do the grunt work, and then I don't even get to use the computer (to its credit, debian takes little "administration" time (read babysitting)). But that means I miss out on the parts of debian that I like. The saving grace of the install is the local 10/100 mbit mirror.
So, I'm disgruntled with debian's installer. I'm sick of telling it that, yes, I do in fact want a 2.4 kernel because I might want to do something with the lartc (linux advanced routing and traffic control) kernel stuff. I'm sick of having to open the damn case and install an 8139 card in order to download the drivers for the real network adaptor that redhat 9 picked up 2 years ago. Blah blah.
My conclusion has been that Debian is an excellent linux distro with a great philosophy of user control and choice which occasionally leaks over into some places where it's a nuisance.
But, having said that, thank you for your work.
~Will
sig?
Then install a Linux from Scratch.
The Tao of math: The numbers you can count are not the real numbers.
I'm wondering how you think Microsoft could possibly support hardware (SATA) that came out after XP was released on the boot CD. Likewise, I can't use a 2001 Redhat/Debian/SuSE/whatever CD to install onto a SATA drive. Be happy they at least included a way to load the drivers for new/non-standard storage hardware during the intial install.
The only advantage of linux in this case is more frequent updates, nothing more.
And frankly, if it took you 4 hours to install the "service packs," all one of them, you either have no idea what you're doing or a slow connection, neither of which are Microsoft's responsibility.
I'm all for pointing out real problems with MS and Windows, but lets stick to real problems, or we all just look like zealous idiots.
To be blunt... The single biggest difference between all distros is whos ego each install of the distro satisfies.
Of course you can look at any two distros and say "look, A-Distro uses RPM but AD-istro uses DEB", or "A-witty-Linux-acronym" uses Kernel 2.4 while "A-wittier-Linux-acronym" uses 2.6. But the sum of all distros all differ in the egos.
In all, most FLOSS development is driven by the wish to become famous (or infamous) within the community. That is maybe also why there are an infinite number of softwares at versions 0.x - it is sexy as hell (and gives a lot of cred) to implement a cool thing, but it is incredibly un-sexy to make it work for everybody and have an intuitive user interface...
Now, don't get me wrong here. Many different and differing distros is a good thing. Not as good as one distro flexible enough to work for everybody, but good none the less. And I am personally very grateful for the variations, as I found a very narrow "distro" called Paul's Boot CD that did exactly what I needed a few weeks back.
But I long for the day when I hear of the Linux distro that promotes itself as "nothing special, nothing fancy, just simple, flexible and intuitive"...
Yes Linux is better in how it handles hardware(ONE reboot AFTER install is complete is all I ever seem to have to do with a linux install, windows has at least 2 for JUST the os, leet alone dirvers, updates, etc.).
But it's lacking in several other areas that would scare developers away.
The big one is will it run out of the box, right now the way compatability between distros and even versions of the same distro work the odds are against it. The would probably have to ship a game with a spare cd containing all the variations on the binaries needed just to work on most of the mainstream distros.
And as much as I laud and love the way Linux distros install in one go without reboot hell, and deal well with hardware changes, Games need good vidcard drivers and that requires getting ati and nvidia on board with optimized linux drivers Though this last point is somthing of a chicken/egg problem as is the next point.
Linus still does not have installed user base to make porting a worthwile effort for many game/app developers.
The concept behind the LSB was a good one and a step in the right direction even if the implementation had its detractors.
Mycroft
https://signup.leagueoflegends.com/?ref=4c3ed6600b6ea