Slashdot Mirror
Ulrich Drepper On The LSB
Sam Lowry writes "In a recent post at his livejournal, Ulrich Drepper criticizes the LSB standard and urges the distributions to drop it." It's an interesting piece; Ulrich raises some good points.
← Back to Stories (view on slashdot.org)
The LSB has nothing to do with glibc? Really? Strange. I always thought the LSB was designed to ensure binary compatibility between distributions, and hence has quite a lot to do with glibc.
Personally, I still think the LSB has some value, but Uli's concerns are valid. IMHO, they seem to point to problems with the current LSB test suite that should be fixed, rather than leading to the conclusion that the whole concept is broken, though. In its current form, there is little value to be had in LSB compliance, true. But it needn't always be that way. A decision needs to be made to either fix the LSB or abandon it altogether. Uli prefers the latter approach. I favour the former.
"The invisible and the non-existent look very much alike." -- Delos B. McKown
I'll grant I'm not familiar with all the politics and the specific methodology by which a Linux distro tests or achieves LSB compliance, but this blog entry sounds a lot like whining. Ulrich whines that it's hard, that the audit raises many bugs, that it's tedious, that other distros "somehow" achieve their compliance but he's not sure how, that the audit process itself has bugs, and that the LSB group must be pushing this agenda down people's throats.
If it were truly well-thought-out, I'd see either one of two lines of discussion. One would list philosophical proofs that the concept of LSB was unsound for specific philosophical reasons X, Y and Z. The other would list technical proofs that the implementation of LSB standards was unsound for specific technical reasons A, B and C. No whining that it's hard. No whining that other distros do it differently. No whining that bugs are found. No whining that there's politics involved. Just solve the problems found, improve the process of finding problems, or show why the problems or the process is untenable.
[
Then I wish he'd put a XML parser into glic so that no-one has an excuse for not using XML for configuration files and for data export / import.
thank God the internet isn't a human right.
I think the intention was that vendors could ship a binary-only package and have it work on any LSB-compliant distribution. Hence the need to specify a package format that should work everywhere, and hence RPM. It's a nice thought, but it might have been easier to ignore packaging and just specify that tar and gzip commands shall be available.
-- Ed Avis ed@membled.com
for these two trolls which are posted on every article about Linux. And yet some clueless moderator mods them up despite the fact that they are both wrong and offtopic.
After ditching some old Linux installs a few months ago (a Gentoo system that had gotten hopelessly snarled up and a YDL with a broken RPM database) I tried out a few different options. Conclusion: the most important thing in Linux is a good package archive. The other 10,000 Linux annoyances mostly need to be solved once. The package stuff is just going to keep on biting and biting at you.
Gentoo and Debian both have superb packaging collections, which is why they both hang on despite Debian's having been walking dead for three or four years now, and Gentoo's, errr, peculiarities. Kubuntu apparently has a superb package collection, which is why it has taken off. The other stuff is just an endless frustration for the apps I use. (Fedora, Mandrakeiva, SuSe fanboys -- YMMV, obviously.)
What I'm listening to now on Pandora...
The other segment is for desktop OSs that run on generic multi-source hardware. That is over 90% of the market, and that is where the BSDs, Windows and Linux compete.
The hardware part of this market segment is not dominated by anyone, there are low entry barriers and lots of players. The OS part is dominated by MS, but with increasing competition from the BSDs and Linux. Whether this will turn into a real threat, its too early to say. Apple is not a player here, and, right or wrong, evidently doesn't intend to be. In this market segment, OS X, whatever its merits to users, is irrelevant because absent.
Conclusion: it may be too late for desktop Linux or BSD, but not because of OS X.
I think the (possibly regrettable, I don't know) answer to this is that Linux users need to choose: they can have an easy-to-use distribution that is a near monopoly in the Linux world (which is WHY it will then solve problems like the one you describe), or they can have a hundred different distributions.
.tar.gz to compile from source (gets you right into th dependency hell you want to avoid)
Right now, so long as you pick one of the "big three" (Debian, Red Hat/Fedora, SuSE), you will have very little package/software install trouble.
Most companies that release Linux software offer the following downloads (as do most OSS software websites for individual products):
1.
2. RPM for Red Hat/Fedora
3. RPM for SuSE
3. DEB for Debian
I have been in the Red Hat family since Red Hat 5 or so and I can tell you that beginning with Red Hat 8 things started to get really easy, and by the time the Fedoras had come around, I spend nearly zero time compiling my own software or chasing package dependencies. Tools like yum/apt even make it so that you don't have to FIND a download site and double-click on and icon, you just type in a command that says "I WANT IT!"
But even for commercial software like Flash or Java, it's cake, I just install the package. The reason is because the package is DESIGNED FOR MY OPERATING SYSTEM.
Sorry, but most of the other Linux operating systems (Slackware, Mandrake, Yoper, Xandros, whatever) are too small for packagers to target them, and that's generally what results in package hell--you are trying to use a package that assumes the components installed by default in another operating system. So even if they are both RPMs, installing a Red Hat/Fedora RPM on Mandrake will cause you trouble. Even once you get the packages all installed, the configuration and support files are likely to be located in all the wrong places.
And yes, generally the packages ARE clearly labeled. So I guess my answer is the one people hate to hear, but if you're going to ask the question about "package hell" then you're going to get this answer: switch to a bigger distro (best case is probably Red Hat/Fedora) and the problem will generally go away.
STOP . AMERICA . NOW
And linux desperately needs LSB. At the very least. And in in my daily job I am clamouring for more!
Right now we are seeing binary packag installers check for stdint.h so they it can guess that the right version of some libraries is in a different place - just so it works across RHES, Fedora, Suse and Debian. Not nice.
Dw
I will probably get modded flamebait, but I agree.
I just went throught the process of adding Bugzilla to my installation of Fedora Core 3. I run Fedora because that is the default Linux installed by my provider and anything else would more than double my costs. I just checked the LSB Certified Distribution List, and sure enough Fedora is not on it. I tried upgrading my system using Yum, but the versions installed with Yum were not current enough for my purposes. Every piece of source I had to download to get Bugzilla installed had to be configured with a switch pointing to a non-standard install directory.
This really surprised me, because the LSB has been around for a long time. I thought all major distributions had become compliant several releases ago. I especially expected Fedora, which many people consider the standard for Linux, to be compliant.
Insert Generic Sig Here:
Let's forget once and for all about binary compatibility. Bury it. Because it does not really benefit most people. There is one very well-known operating system which implements as near full binary compatibility as you can get -- and it's generally regarded as a disaster.
..... but the new stuff probably will be more like the natural stuff so nothing will need to be changed}.
What matters is source compatibility. And right now GNU/Linux has that in spades. Not just GNU/Linux, but the BSDs, Mac OSX, Solaris and even Windows have it. If the source code is properly written, and properly packaged, then it will compile on any machine that is up to the job of running it. If you make any really drastic changes -- the standard C library for instance -- you might well have to recompile some applications. Is that a major hardship? I don't think so. Back when we changed from round-pin 5 and 15 amp plugs to rectangular-pin 13 amp plugs, people had to have their houses rewired. When we went from artificial gas to natural gas, people had to have their cookers and heaters modified. When Channel Five launched, many VCRs needed their RF output shifted. These were all necessary changes for the better {ironically enough, we probably will be going back to artificial gas in future
Binary compatibility was never more than a nasty hack, fudged in for the benefit of those who want to lock up the source code of their software. These people are pure evil. By not sharing their code with you, they are just one very tiny step removed from stealing from you. It had the beneficial {at least, it was beneficial when processors were slow and disk space small} side effect that you did not have to spend CPU time and disk space compiling applications locally; but now that disk space and processor power are cheap, the benefits of pre-compiled applications are diminished substantially.
There's even a good argument to be made in favour of deliberately introducing binary incompatibility. If programs compiled on my computer would only ever be able to run on my computer, and any program compiled on anyone else's computer would never be able to run on mine, then there would be no such thing as viruses or buffer overrun vulnerabilities. {Unfortunately, this raises the question of how to ever get any computer up and running}.
Je fume. Tu fumes. Nous fûmes!
Yabut, Glibc was providing binary compatability via. symbol versioning back before the LSB was concieved. The LSB relies on Glibc providing symbol versioning; not the other way around. What the LSB says actuallu impacts very little on Glibc, because with or without LSB, Glibc would be doing exactly what it's doing right now.
Ulrichs complaint is basically "LSB is poorly thought out, poorly tested and poorly communicated" which is a fair comment.
In the old days, UNIX splintered and it never really recovered until Linux came along. I think Linux is moving in the same direction in ways. With the popularity of the 31337 gentoo boyz that do all sorts of different things without understanding what they are really doing (just building and installing shit to play, mostly) to the different security policies implemented across the board to the different places that basic files are installed and the different ways that basic processes are started up. As a developer I'd have to say that it's still safest to static link your app and supporting "Linux" can be like supporting 5 different platforms.
It's not a monumental task but it's not a simple one either, the majority of the market doesn't want to "configure" stuff, they want to install it and go. LSB isn't about the geeks but that's really who it has been aimed at, the filesystem isn't supported by hardly anybody. What would be better, IMO, take a fairly open community distribution. Pick a base platform out of it, capture all the version numbers and locations and crap, and call that "Linux2005 Standard Base" do it again in a year or 2. Build GUI versions if you need. You want to be compatible with that.
This lib64 madness seems a little shortsighted as well. Why don't we assume that 64bit machines will strive to be native 64bit across the board and have a lib32 directory for compatibility and not rename every damn library? Or put it in the soname. I about shit when I saw that and it turns out, that's what LSB wants you to do.
If we're serious, we need to step this up. There is no "common" way to install and run java applications on Linux. If you do anything with a database you have to potentially ship it and it requires some amount of configuration by a user who may or may not know what they are doing, why can't we fix that? There are tons of other things too.
#1. Define the format of the package that LSB apps will be shipped in.
#2. Define the functionality needed by the package management system to install, update/upgrade, remove those packages.
#3. Let the various distributions add that functionality to their own systems IN ADDITION to the functionality they already have.
Never define a app as the "standard".
Always define the functionality so anyone can write an app to that standard.
And while he happens to be right in this case, I don't think very highly of him. He's clearly very bright, but the poster above who said that Ulrich had a bigger ego than Theo was spot on. Too often, he lets his ego and NIH syndrome get in the way.
For example glibc is the only major C library that doesn't support the new buffer proctected string functions originally written by OpenBSD (at least last time I checked). These fuctions are faster, safer, and easier to use then the POSIX ones and are supported not just on BSDs but almost every commercial UNIX. Source compatability alone would dictate including them.
Drepper however has repeatedly refused to include them because they work and they make it too easy to not code buffer overflows (no this is not a joke). According to Drepper programmers should be good/smart enough not to mess up something so simple as a string buffer so including a defacto standard that makes it easy to get it right is inappropriate. WTF?
The policy of the United States is worse than bad---it is insane. -- Ludwig von Mises, Economic Policy(1959)
autoconf? Hell no! That stuff is some of the most bloated stuff the FSF makes. If you really want to fix the code look at how SSH handles portability. That's a much better way to do it.
The policy of the United States is worse than bad---it is insane. -- Ludwig von Mises, Economic Policy(1959)
Citation and statistics obviously help. But they aren't the be-all and end-all.
Example 1: Interest Rate Prediction Markets
Traders make judgements based on lots of market data - consumer confidence surveys, growth predictions, oil/stock/property price trends, major company results, etc. When you have a particular outlook it is almost always easy to come up with LOTS of figures to support that position. These are all verifiable statistics! To objectively ensure you are taking into account a representative sample when making your decision is what separates the good traders from those who (often unknowingly!) rely only on luck.
Example 2: Microsoft vs GPL
Imagine the following argument from Microsoft to a PHB. Microsoft: "If a GPL component takes up even 0.1% of your (binary linked) codebase your entire codebase must be GPL licensed when you distribute. Therefore the GPL is a disproportionate license." This is verifiable and true, and seems on the surface to be a perfectly cogent (economic) argument against using GPL'd components. Asking yourself "why is this information coming from Microsoft" helps understand things far more clearly.
Example 3: Groklaw vs SCO
The magistrate judge granted SCO's motion for ridiculously wide discovery. PJ gave all sorts of plausible legal and technical arguments why this wouldn't happen. She backed this up meticulously with quotations, citations, etc. And yet, by not looking at the bigger picture - namely, that discovery is almost always broadly interpreted, many (myself included) were shocked at this decision which in retrospect seems obvious.
Example 4: Dihydrogen Monoxide
OK, this one is stupid, but it shows how easy it is to misattribute symptoms for the cause.
Now, back on-topic, I know next to nothing about the LSB beyond 'it seeks to make to distribute software cross-distro'. Ulrich's arguments seem excellent to me. But without any deep domain knowledge, it would be foolish not to ask questions like "Does RedHat have business reasons for disliking the LSB?", "Does RedHat seek to be a controlling influence on the direction of the LSB?", etc.
It's perfectly reasonable to suppose that there is no agenda here - after all, RedHat aren't known for widespread FUDing. But facts can easily be twisted/cleverly presented to suit an agenda, and so it is advisable to consider side-channels too.
If the test-suite is broken, then the LSB guaranties are worthless.
Riiight, you have apps. that are checking for an include file in a specific location, which is _also_ provided by dietlibc ... and I can guarantee they can't use that.
Instead of say just running /lib/libc.so.6 and comparing the version (which, of course, isn't ideal and it could even be done using the output of LD_DEBUG/ldd to work even if libc wasn't in /lib).
Hell given the "portability" requirements, they could just test the output of: "rpm -q glibc || dpkg -l libc6"
Your problem isn't going to be solved by LSB, even if LSB was actually amazingly well done.
ustr: Managed string API with ave. 44% overhead over strdup(), for 0-20B
First what I said above is true, at the time Ulrich said specifically that strlcat and strlcpy wern't nessessary because programers could just check their code for the common mistakes the strl* functions are intended to solve.
1) It is true that they are not in the POSIX, ANSI, ISO, or Single UNIX standards, but neither is a ton of the other stuff in glibc. However, they are supported on almost every non-GNU libc -- making it a defacto standard. Many open source apps use them and there is a BSD-licensed reference implementation.
Not implementing something because sun can't copy and paste correctly on the first try is about as silly as arguements come. If anything the arguement that roll-your-own tends to get messed up is an arguement FOR inclusion not against.
2) asprintf, a gnuism which is also non-standard and less widely implemented, is less used then the strl* functions in actual code. The strl* functions are admittedly not as general as a fixed printf but they are much faster and are near drop in replacements for old code. asprintf is simply not a workable replacement for strlcpy or strlcat. Why should you do a full printf just to cpy or cat a string? Truncation is not the problem you've made it out to be. The strl* API will tell you when it has occured allowing you to make whatever adjustments you need to make. Expanding the heap space to suck up whatever gets thrown at a program is just asking for DoS in many cases. Furthermore truncation is not something strl* was designed to fix. These are drop in replacements for the broken, slow strn* functions. They couldn't fix truncation in the way you and Ulrich want and be compleate replacements.
3) You already said above that roll-your-own was likely to get messed up. It's also stupid to have a few dozen open source apps carting around the code instead of putting it in a library. The GNU C library is one of the only C libraries without this functionality, because of that many programers continue to make the same mistakes that the strl* functions were intended to fix. Not including good, frequently used, defacto standardized code is a really dumb idea, especially when the excuse is that people didn't use the overkill GNU solution.
These functions demonstrably prevent common coding mistakes, prevent buffer overflow attacks, and improve code security. There is every reason to include them in glibc. Not including them was pure ego and NIH syndrome.
The policy of the United States is worse than bad---it is insane. -- Ludwig von Mises, Economic Policy(1959)