Slashdot Mirror
Ulrich Drepper On The LSB
Sam Lowry writes "In a recent post at his livejournal, Ulrich Drepper criticizes the LSB standard and urges the distributions to drop it." It's an interesting piece; Ulrich raises some good points.
← Back to Stories (view on slashdot.org)
Who is Ulrich Drepper, and why should I care about what he says on his LiveJournal?
-mkb
Some other random dude says this isn't true over on his MySpace!
I've been using Linux for many years, and the problem of obtaining software packages drives me to the end of my nerves. Every single time I try to get a package that isn't something extremely common like Apache, I run into major, major problems. Honestly, I don't care how the problem gets fixed. Distribute a binary with everything compiled in for all I care. Distributions distribute every package known to man anyway. :)
./configure nonsense. It sucks.
Something needs to be done. Even with the source, half the time I have to make all sorts of include changes. What is so hard about providing a common build and install process? If you get Apache, OpenOffice, and Mozilla to adopt a convention, everything else will follow. Why not have something like Apache Ant that simply installs either to a user directory or to a common directory and links to every user directory? Then provide a nice GUI on top of it, where it will either compile if the source is there and then install, or just install otherwise? How hard could that be? Forget this
Regardless, this is a perfect example where sometimes it really does make sense to have "management" provide leadership by imposing structure. Ideally, they would be serving and representing the interests of users and helping to overcome the disinterest of joe programmer who doesn't do the psychologically difficult work of catering to someone other than themselves. The "scratch an itch" metaphor breaks down when other people don't know how to "scratch" themselves and need the help of a division of labor to serve their needs. Before you say that they should learn how to "scratch", think that as a community, society, and economy we all scratch eachother's itches in an incredibly diverse number of ways. This comes about because of intentionally trying to fulfill a demand. In the case of the Linux stack of Free/Open Source software, the developers have not taken responsibility for how their product is consumed.
Dude thanks! I finally know how to install this game on Linux. The last time I tried, I ended up causing my mother's computer to wardial her friends from her recipe club.
Current mood: Sad :(
It's interesting because it's true.
The poster points out some of the same frustrations many non-linux people have when they try to use the OS. Keep in mind, that anyone switching to Linux still has to do work. This means any switching to Linux research is going to occupy spare time. That time better be spent getting Linux to do my work better, not me making Linux work at all.
You've been incrementally learning Windows for 10 years now. Every time you change versions you have to go through another learning-curve bump. "Where did they put "ODBC Drivers" now?". If you were suddenly presented with learning Windows on a tabula rasa, your learning curve and frustration level would be just as high as they are for a Windows user moving to Linux for the first time.
If you're a programmer, let me ask you this: How many text editors have you had to learn? Isn't it a pain in the ass learning a new one? "Hell, I already know 43 editors, I have no desire to learn another one". This does not make any of the editors you already know superior to the one you don't, nor does it make the new one inferior just because you don't. Different isn't a priori bad, it's just different.
No folly is more costly than the folly of intolerant idealism. - Winston Churchill
Let's forget once and for all about binary compatibility. Bury it. Because it does not really benefit most people. There is one very well-known operating system which implements as near full binary compatibility as you can get -- and it's generally regarded as a disaster.
..... but the new stuff probably will be more like the natural stuff so nothing will need to be changed}.
What matters is source compatibility. And right now GNU/Linux has that in spades. Not just GNU/Linux, but the BSDs, Mac OSX, Solaris and even Windows have it. If the source code is properly written, and properly packaged, then it will compile on any machine that is up to the job of running it. If you make any really drastic changes -- the standard C library for instance -- you might well have to recompile some applications. Is that a major hardship? I don't think so. Back when we changed from round-pin 5 and 15 amp plugs to rectangular-pin 13 amp plugs, people had to have their houses rewired. When we went from artificial gas to natural gas, people had to have their cookers and heaters modified. When Channel Five launched, many VCRs needed their RF output shifted. These were all necessary changes for the better {ironically enough, we probably will be going back to artificial gas in future
Binary compatibility was never more than a nasty hack, fudged in for the benefit of those who want to lock up the source code of their software. These people are pure evil. By not sharing their code with you, they are just one very tiny step removed from stealing from you. It had the beneficial {at least, it was beneficial when processors were slow and disk space small} side effect that you did not have to spend CPU time and disk space compiling applications locally; but now that disk space and processor power are cheap, the benefits of pre-compiled applications are diminished substantially.
There's even a good argument to be made in favour of deliberately introducing binary incompatibility. If programs compiled on my computer would only ever be able to run on my computer, and any program compiled on anyone else's computer would never be able to run on mine, then there would be no such thing as viruses or buffer overrun vulnerabilities. {Unfortunately, this raises the question of how to ever get any computer up and running}.
Je fume. Tu fumes. Nous fûmes!
Were there one available, I would still be unlikely to use it. The fact remains that after you've seen through all the marketing hype, XML remains inappropriate for many tasks, and configuration files are right at the top of the list. You only have to look at Jabber or Tomcat to see some perfect examples of that.
"The invisible and the non-existent look very much alike." -- Delos B. McKown
All other points raised are shown to be consequences of this.
The specific example he cited is a rather enormous bug (a thread which is detached can by definition not be joined. "Detaching" a thread means telling the system that you are not interested in its exit status... and join()ing is reading the exit status).
(This doesn't mean that other examples are as clear cut. It could still be that most tests do actually show genuine glibc bugs, and that he just picked up the right example to bolster his point.)
that the audit raises many bugs
that other distros "somehow" achieve their compliance but he's not sure how
I'd say, if Ulrich is right about the test cases, the situation should be fixed by removing/rewriting the dodgy test cases althogher. Deliberately running distros with non-standard shared libraries or on dog-slow hardware to make them succeed the tests is pointless. If that is indeed how "somehow" some distros achieve to pass the tests, Ulrich is indeed right on the mark that it would make the test suite completely meaningless. You are not certifying a distribution, but you are certifying a distribution tweaked to run the tests...
Better fix the suite, and run the distro under "normal" conditions (i.e. the same as normal users would do).
He's not whining that it's hard. He's whining that it's impossible, because the tests don't match the either the standards or common practice. He's whining that distros must be somehow faking compliance, because they ship *his software* which doesn't "pass" the buggy tests.
.001 seconds). And the same cases are particularly hard to test programs' assumptions about.
His argument is: no set of Linux software could pass the LSB suite by actually consistantly giving the desired results, because there's no libc that consistantly gives those results (when run on sufficiently fast hardware to expose the bugs in the tests, for example); yet distros do claim to pass the suite; therefore, the LSB is not ensuring compatibility, because it certifies things that don't work by their rules.
Furthermore, he argues that programs that don't work tend not to work because they rely on undefined behavior. Certifying that the environment behaves in accordance with the standard doesn't help, because the software developer's environment and the user's environment may do different things in some cases, while both comply with the standard. Unless the programs are tested for doing non-standard things, they won't necessarily work. And the undefined behavior is undefined for a reason: you can't improve the system without changing it (especially when the thing not defined is which takes longer: executing a certain function or waiting
The sections that you dismiss as whining are actually providing examples, which is important in engineering (or science). There are theoretical flaws in any process; it is always important to know whether those situations ever actually occur. If he didn't have an example of a program relying on undefined behavior which should vary between systems, one could say that nobody would actually write code like that and think that it worked; but it turns out that people actually do write such code, and these people happen to include the people writing LSB tests, which is why they're flawed tests.
And while he happens to be right in this case, I don't think very highly of him. He's clearly very bright, but the poster above who said that Ulrich had a bigger ego than Theo was spot on. Too often, he lets his ego and NIH syndrome get in the way.
For example glibc is the only major C library that doesn't support the new buffer proctected string functions originally written by OpenBSD (at least last time I checked). These fuctions are faster, safer, and easier to use then the POSIX ones and are supported not just on BSDs but almost every commercial UNIX. Source compatability alone would dictate including them.
Drepper however has repeatedly refused to include them because they work and they make it too easy to not code buffer overflows (no this is not a joke). According to Drepper programmers should be good/smart enough not to mess up something so simple as a string buffer so including a defacto standard that makes it easy to get it right is inappropriate. WTF?
The policy of the United States is worse than bad---it is insane. -- Ludwig von Mises, Economic Policy(1959)
The fact remains that after you've seen through all the marketing hype, XML remains inappropriate for many tasks, and configuration files are right at the top of the list.
In fact, it's the opposite: XML makes a lot of sense for configuration files. For instance, suppose that you need to write a script that automatically adds a line to /etc/X11/xorg.conf or a similar configuration file. If a file like that is in XML, this is trivial: you can write a XSL transformation or use any of a billion tools to apply the change in a correct way. But if it's in some ad-hoc file format (as it is right now), you either have to write a parser and unparser (which would have been unnecessary if it had been in XML; and how do you know for sure that your code is entirely correct?) or use some hacky combination of sed/grep/etc. to perform the change (which is, alas, the "Unix way"). The latter will of course fail unpredictably in lots of cases. E.g., are you handling those sections correctly? Comments? What if the line was already present? And so on.
Of course, XML is a horribly bulky format. But who cares? It's not like configuration files will take up a lot of disk space either way. The important thing is to have a universal standard format that can be easily manipulated using standard tools so that you don't have to implement parsers and printers all the time or approximate them using broken sed/grep hacks.
While Ulrich has his faults, the above is completely false. The reason they weren't accepted into glibc was IIRC:
1) They are non-std. and did not have a usable standard like definition apart from the implementation and had no tests (Solaris implemented them slightly differently, for example, and Input Validation in C and C++ from oreilly also screwed it up -- and that was written by people selling a Secure codeing in C book).
2) It doesn't solve the problem better than asprintf() which had been around for years (although also non-standard), as you still have problems with truncation (and both APIs have the problem of requiring the programer to correctly pass around the meta data about the string -- Ie. it's size/length).
3) Given the above, and the fact the implementation is "free" then anyone wanting to use them can just include the source in their apps. and rely on autoconf (and they'll also be guaranteed to have the "correct" implementation).
ustr: Managed string API with ave. 44% overhead over strdup(), for 0-20B