Slashdot Mirror
Linux and GNU at their best
Mapc writes "...Couple of years ago (in 1990) there was a study called fuzz which checked quality of UNIX utilities. It has been revisited since then - in 1995 (and in May 1998?). Interestely enough the study shows that GNU utilities and Linux utilities are the best made ones (lowest - 7-9% fail ratio)
The paper is here , the original fuzz program lives on the ftp as well.
Test it on Solaris 7 and tell us the results"
This is great to have this test report available now. I will be attending a meeting this week where I will present the argument for Open Source software and why we should embrace and use it.
It would be really interesting to see the results of tests conducted against MicroSoft products.
"ghostview" gives me a bunch of errors when I try
to look at the paper. Any clues on how to get it
to work?
uhm, not to sound rude, but did you uncompress the file first?
:)
(never know... some people forget such trivialities
What else to expect from software developed by fanatics who is not limited by either budget or time schedule?
Workaround for GS 3.33:
/CoreDistVersion get 2000 ge {
Find the line:
currentdistillerparams
and replace with the line:
false {
No problems on Solaris with Postscript viewer
hehehe
Where are *BSD utilities ?
You say it like it's a bad thing. :)
Seriously, That really isn't an excuse to have non-quality software. There have been commercial companies that have proven that in spite of deadlines and budget constraints. Quality software can be made. It's largely a matter of priorities, or what is important to you.
I noticed this too, they only tested SysV products. Seriously tho, what was to be expected from the likes of AIX, HP-UX and company. With the exception of Digital Unix which has since been sold out to The Man, it's just a test of a bunch of crap. I'm not trying to start a holy was (like the previous response to this question) but I can't be too impressed with Linux or GNU when it's competition is that as it was. Anyone worth their weight in root beer would know that GNU utils would rank a hell of a lot higher than everything else on that list (insert OSS reasoning that we all know here).
To summarize, AIX, HP-US, Ultrix, etc. suck, would like to see a more extensive test, ignore FUD in previous response.
Jackson
Sounds good to me! I'm a non-coder that sure wouldn't mind doing this type of thing. There's got to be others as well.
Eliot Landrum
elandrum(at)bigfoot.com
(didn't want to bother making an account today)
The purpose of a man page is to provide a short reference for people who know what they are doing. I fail to see why that is no longer useful.
If one is using unsafe languages such as C or C++ to write these utilities, one should expect a high occurrence of these types of errors.
If it were true that it doesn't render in
ghostscript (it does render fine on my machine),
it would have been doubly ironic. Both the Fuzz Test and the Alladin project (ghostscript is a branch from Alladin) are from the same University.
I'll agree that GNU info is crap, but I don't agree with anything else said. Incedentally, the reason for the "-" and "--" parameters being different is that "-" is the one-letter cryptic options, and "--" is for the full-word versions of those same options. When GNU wanted to make mnemonic full-word options, they couldn't use the standard "-" for it. (Because the de-facto standard is that something like "-opt1" is equivilent to "-o -p -t -1".)
I prefer to use a coverage tool to test my code .
It is a kind of white box test, so it is perfect to test open source programs.
Take a look on
ftp://cs.uiuc.edu/pub/testing/GCT.README
I don't mind the -- options so long as there's a - option to handle it, and I'm not making an emergency boot disk or otherwise working with extremely tight space requirements.
However, don't complain about GNU utilities until you work with HPUX, AIX, SCO, and Solaris with only vendor provided software. You want to view a binary file with your pager? Sorry, less isn't an option. Want to list the files that *don't* contain your pattern? grep -L unsupported. Having a vi which has line-length and file-length limitations is incredibly annoying; how do they expect that you'll edit files with >2048 characters per line? I run into files with lines this long daily, and AIX refuses to let me mess with them.
I really find it cute that under Solaris 2.4, no matter what corruption fsck found, it never thought I might want it to put unreferenced inodes in lost+found. Suddenly I understood why they weren't concerned that a removed lost+found directory wasn't readily replacable, without doing a newfs.
The Unix Philosophy tells you to do one thing, and do it well. The commercial unicies listen to the first part, GNU listens to the second part. Which part do you find more important?
Sure, I'd much prefer to have some software that followed that entire sentence. But we frequently don't have that option, and I'd rather put up with a few bloated features rather than software that doesn't handle its job competently.
I don't use emacs, because vim-4.6 does one thing well, and emacs just does stuff well. Um, depending on who you talk to. That gets into my rant about GNU software coding styles, and how utterly wrong it is. But that's off-topic.
Maybe the >> operator is fixed in that respect by the use of Strings, but that is like plugging a little leak in the Titanic. Array access/pointer error/allocation errors occur commonly in C/C++ programs, and they could all be prevented by using a more advanced language.
Check out Lyx, and Klyx (the latter being a
K desktop aware version). Nice WYSIWYM
(What You See Is What You Meant) editors for
SGML documents.
...that fails?
It does indeed work fine on Aladdin Ghostscript 5.50 (MGv, the best GS frontend, works quite nicely too, I might add :).
Brian Fundakowski Feldman
...which was the point alluded to before.
Funny. Quality is impossible in business, but unstoppable as a non-business activity.
What part of "gestalt" don't you understand?
Are there other tests in the GNU arsenal?
What part of "gestalt" don't you understand?
The BSD utilities are in the BSD distributions. More to say, there source code is under ftp.freebsd.org, ftp.openbsd.org and ftp.netbsd.org.
They are usually a lot simpler than their GNU counterparts.
Isn't the failure rate of UNIX utilities affected by whether you have root access or not? Since most Solaris boxes are owned by companies with very few root users and most Linux boxes are owned by individuals with root access of course the failure rate is going to be lower.
GNU Ghostscript 4.03 (1998-5-1) works fine with this document. Be sure you're running the latest version before reporting bugs.
What would be useful is to have a small team of people doing this sort of testing continually, rather than every five years, to provide bug reports and feedback to coders. Most people find formal testing a chore so they don't bother, but it looks like this sort of "low-knowledge" testing could be done by anybody for any application.
Why don't we go through that list of utilities the good Professor said failed under Linux and get those bug reports filed, rather than hope that the coders see this paper for themselves? This looks like a perfect means by which non-coders can give something back to the Open Source community.
Worth a thought?
You can view HTML in a command-line environment as well, either with Lynx, a HTML to text converter or just from more (crude but effective). Those old man pages need converting.
I had the good fortune to see this paper presented after the 1995 revisiting. Professor Miller gave an excellent talk, and is the best CS professor I ever had.
--Gus
Also, there is no mention of any 1998 revisit to the study, nor did Prof. Miller mention it while I was working for him in the first half of last year. The "last modified" date on the file is in 1995, so there was probably not a revisit last year.
--Gus
The failure rate of Unix utilities under these conditions has nothing to do with root access. These tests measured the ability to handle strange and unusual input data; they could be run as any user.
--Gus
This is not really a cause for celebration. 7-9% failure rates for an incredibly primative test is far too high. That commercial vendors are even worse does not make it any better. This kind of stuff damages the reputation of *nix as a whole. If anything it should be a call to arms. It would be interesting to get the source code and see how
it performs on an up-to-date linux system. I wish I had more time to do a bit of this sort of testing/fixing.
---- Backwards compatible -- If it's not backwards it's not compatible
Regarding the duality of options, frankly, '-i' means very little to a new user, but '--initialize' provides a hint as to what the option might accomplish. The idea behind long options is to provide a human-parsable alternative to the shorter versions. The overhead of adding this support is extremely low, and the benefits for first-time users more than pay for it.
Info format is awful. You'll hear no argument from me. But think about when it was proposed; the only alternative really was roff. Man pages fail to solve one question in an important two-part problem: how do I learn this, and how do I look up information after I've learned it. Man pages solve the latter. SGML manuals, properly written, can solve both, providing both a tutorial, and a reference.
Attracting newbies isn't the goal; helping people use the system is, on both counts.
-Ed Felix qui potuit rerum cognoscere causas.
Those failure rates are old, and if run on the current GNU tools, you should get better results (since some of the failures have been fixed since then).
It would be interesting to re-run the tests with the latest tool versions.
failure reading a paper about the quality and low failure rate of GNU utilities on a GNU utility.
-- adraken
I just thought you might want to know that there is a project specifically aimed at fixing the bugs described in this paper (the 6-9% failure rate). (In fact, many of them have been fixed.)
If a thing is not diminished by being shared, it is not rightly owned if it is only owned & not shared. S. Augustine
Here.. I converted the file to PDF 3.0 if anyone is interested. If you need it in 2.x e-mail me. It's only 50kb in 3.0 format.
http://lonestar.texas.net/~la ndrum/fuzz-revisited.pdf
my web usage meter has gone up a bit since i posted this, so i guess ya'll have used it.
:)
yesterday my total usage for the week was 18.45 megabits (my isp is weird and measures it in bits) and today my total for the week is 72.43 megabits.. since my other sites on there basically suck, i guess this thing is popular
Can I get his report as a zipped RTF, or as an acrobat file, or HTML, or anything other than a Tar/Gzipped _Postscript_ file?
Kris.
First of all this test is updated for 1995 if it's true that it wasn't revisited in 1998.
GNU utilities got a LOT better since then, and measuring by the size of patch clusters for solaris, sun didn't make much progress.
And secondly, the source is on this ftp as well, fetch it and try it on anything you want.
* When I get a new computer I first of all wipe native utilities, and then install GNU utilities.
They're much more consistent, faster and better.
Linux wouldn't be what it is today if not GNU.
I think that the GNU people deserve a big compliment.
--Roman , remove spamless. to mail me.
What ? GNU the worst set of utilities ?
I don't agree a single bit with that. Without all the wonderful GNU utilities, my life would be really painful.
The -- for long options is a nice feature. Why do we need two options, you ask? Why not? What's wrong with it? I think its well worth it. They do all your normal non-GNU utilities do and more. I don't use long options that much, but I don't have a problem with 'em. I think the typical newbie you want to attract will find programs easier to use with long options.
Well, I wouldn't completely agree about the info stuff either. While its true that GNU should put more attentions on the manuals, documentation on an hypertext source is really better when you want to learn how to use *new* stuff. As I see it, both things, manual and info, are useful and have different objectives. May be info is not the format to use, but hypertext is the thing. And I have found info useful anyway.
So, you see *data* saying GNU behaves better than anything else and you say they're the worse set of utilities out there ? You think they cheated or do you simply care about "-- and -" and "manual pages missing" more than having a utility do what its meant to do?
Azulejo.
The BSD utilities ARE the Gnu utilities. At least on the newer freebsd systems this is true.
I suggest proponents of "creeping featurism" read Gancarz's "The Unix Philosophy" -- and someone send the FSF a copy while you're at it