Ulrich Drepper On The LSB

Sam Lowry writes "In a recent post at his livejournal, Ulrich Drepper criticizes the LSB standard and urges the distributions to drop it." It's an interesting piece; Ulrich raises some good points.

Comments

who?

[mmkkbb]

Who is Ulrich Drepper, and why should I care about what he says on his LiveJournal?

Re:who?

[Anonymous Coward]

How he is considered qualified to talk about the LSB when it doesn't have much of anything to do with Glibc, I don't know.

I take that right back. I'd forgotten that LSB goes as far as defining the ABI, which is clearly the realm of Glibc and something which Ulrich is more than qualfied to comment on.

I've always thought that the biggest problem with LSB was that it didn't go nearly far enough, which means that distributors and users can't all use the same binary and we end up with these ABI issues that Ulrich complains about.

From what Ulrich says, the idea of the LSB is good but the implementation is deeply flawed. The standards board are seperated from the implementors who are seperated from the testers and communication and understanding between the groups is poor. Which is a shame, but LSB has always struck me as a bit of a lame duck.

Re:who?

[schon]

More to the point, why should you care about the opinions of someone who would write this:

My advise: but the losses. To some extend, I think, the claims a scaled back meanwhile, if I understood Art correctly.

To quote Lisa Simpson, "I know those words, but that sentence makes no sense to me!"

Can someone.... anyone convert that into English?

Re:who?

[Cyno]

Glibc is the GNU C Library. Being a library, or a file of any kind, it is subject to some of the LSB rules, such as the location of libraries within a filesystem.

Some of these rules are *nix standards and make sense in an old fassioned traditional way. Or they make sense in that we need a standard place to always find these files between different systems, in order to assume some sort of compatibility across platforms, way.

But they don't always offer the best solution. Sometimes they have unnecessary rules that should simply be optional, like /opt and /usr/local. And that's just talking about the filesystem heirarchy. I don't know all the details, but if they're anything like the LSB filesystem layout, there has to be a lot of unneccessary crap that makes it a chore for a distro to be LSB standards compliant.

I won't be certifying my distro against the LSB and I doubt the majority will.

But Ulrich Drepper is qualified, if anyone is, to talk about the LSB. He's maintaining glibc, the most important and most used library in the GNU system. He's not some kid who booted a Knoppix CD once. He's also done a lot of stuff.

But from my perspective he hasn't had the best track record with glibc maintenance. There was a few months of beta and CVS releases during a critical time for NPTL integration with a severe lack of documentation, roadmap or anything to help out the distros. That appears to have been resolved, but leaves us with some concern about the level of professionalism displayed by core projects.

Luckily most core system software is maintained by responsible professional organizations who have a better track record than some multi-billion-dollar corporations.

Re:who?

[nietsch]

How do you conclude that the implementation is poor? The implementation is the stuff the various distro's put in. What the LSB is, is a spec and a testsuite. His rant is about parts of the testsuite, which appear to be written by less skilled people. I agree that that is a bad decision, the test should at least be written by someone skilled in the software to test, preferably the software developers themselves. But testcode is not the deliverable code, it may contain bugs, just like any other piece of software.
Where Mr Ulrich has a point too, is that there appaers to be no process in place to adequately fix the bugs in the testsuite, and what is being done with those fixes.

But Mr Ulrich proposes to throw away the baby with the bathwater. The generally accepted idea behind the LSB (one common architecture for commercial ISV's) is still sound even though the organisation and processes are a bit shaky.

Ulrich Who?

[LithiumX]

Just curious as to who this guy is...

...and realizing that in today's net-driven society, all it can take is for people to quote you, and others automatically assume you're important. I have no idea who this guy is, and I'm already assuming he's someone since ./ quoted him in an article.

Re:Linux is too fragmented

[Anonymous Coward]

I do believe this is a troll. Note the author's name, when spelled backwards, is a slur.

I agree, but something needs to happen

[Anonymous Coward]

I've been using Linux for many years, and the problem of obtaining software packages drives me to the end of my nerves. Every single time I try to get a package that isn't something extremely common like Apache, I run into major, major problems. Honestly, I don't care how the problem gets fixed. Distribute a binary with everything compiled in for all I care. Distributions distribute every package known to man anyway. :)

Something needs to be done. Even with the source, half the time I have to make all sorts of include changes. What is so hard about providing a common build and install process? If you get Apache, OpenOffice, and Mozilla to adopt a convention, everything else will follow. Why not have something like Apache Ant that simply installs either to a user directory or to a common directory and links to every user directory? Then provide a nice GUI on top of it, where it will either compile if the source is there and then install, or just install otherwise? How hard could that be? Forget this ./configure nonsense. It sucks.

Regardless, this is a perfect example where sometimes it really does make sense to have "management" provide leadership by imposing structure. Ideally, they would be serving and representing the interests of users and helping to overcome the disinterest of joe programmer who doesn't do the psychologically difficult work of catering to someone other than themselves. The "scratch an itch" metaphor breaks down when other people don't know how to "scratch" themselves and need the help of a division of labor to serve their needs. Before you say that they should learn how to "scratch", think that as a community, society, and economy we all scratch eachother's itches in an incredibly diverse number of ways. This comes about because of intentionally trying to fulfill a demand. In the case of the Linux stack of Free/Open Source software, the developers have not taken responsibility for how their product is consumed.

Re:I agree, but something needs to happen

[Mr. Underbridge]

I've been using Linux for many years, and the problem of obtaining software packages drives me to the end of my nerves. Every single time I try to get a package that isn't something extremely common like Apache, I run into major, major problems.

No kidding. You'll find some decent looking project, and it's no big deal, the developers just require this neat toolkit that they consider standard, and all the 133! distros have it, just not the old ones like RedHat, Slackware, and SuSE. Of course, the most recent build is two years ago, because after a year of development all the kids got egos and couldn't stand each other.

Of course, then you find out that the neat toolkit they use depends on an old version of Python, and naturally it's built to do a hard-coded check for a specific version of python in the configure - not the current one of course. And naturally the references to the old version of python are strung throughout the config file. And as it turns out, if you fix all the references in the config, that will break the calls somehow. So you can either install yet another version of python, or forget about this neat little program.

I really prefer compiling from source, but it's getting to the point where it's just not worth the crap.

who cares?

[banana fiend]

RTFA

It could have been written by Bill Gates or my mom.

Why does the author have to be so important if the facts are laid out and verifiable. You don't have to agree with his analysis nor his conclusions, but the facts should stand or fall regardless of the author

Re:who cares?

[otisaardvark]

Unfortunately this is no longer true. I tried for many years to adopt a "judge by content, not by source" policy, but have realised it's just hopeless idealism.

There has always been spin and FUD, but these days it has developed a very organised, very slick phenomenon. This means that you need to give increasing weight to background motivations to pierce the veil.

Re:who cares?

[banana fiend]

it's just hopeless idealism.

ummmm... at some point someone has to produce content to gain credibility. You say that FUD has become slick? Just because someone produces a slick info shot doesn't mean you shouldn't STILL be checking the facts.

I think we're probably on the same side here, but you don't need anything to "pierce the veil" except verifiable references.

Which this guy has. You can go to the bugzilla database that he talks about and discover for yourself if most of the bugs submitted are indeed bugs that show the tests are broken

Re:Linux is too fragmented

[Jjeff1]

It's interesting because it's true.

The poster points out some of the same frustrations many non-linux people have when they try to use the OS. Keep in mind, that anyone switching to Linux still has to do work. This means any switching to Linux research is going to occupy spare time. That time better be spent getting Linux to do my work better, not me making Linux work at all.

Re:Three Key Issues

[aussersterne]

All of these problems were solved ~5 years ago as Linux really hit the desktop.

A) You should never have to recompile the kernel or break anything by upgrading the kernel in a recent distro. And in the Red Hat product family, since Red Hat 8 (i.e. 8, 9, FC1 through FC4) all you do to install software is double-click on the package icon.

B) In the same lineage of Linux systems just mentioned (i.e. RH8 and later), all administration, from firewall through Apache setup, can be done with graphical configuration tools in the start menu.

C) Both KDE and GNOME (and pretty much all distros use one or the other for a long time now), all applications and the system itself are documented in an extensive help system. The KDE system is somewhat deeper and more comprehensive, but both are much more accessible than MAN pages.

BUT as a final comment: the reason to use Linux never has been, and never will be, for a nicer "desktop" environment than Windows. Desktop environments are inherently limited and there will come a point at which not much more can be done to "improve" them.

The REAL REASON to use Unix/Linux that opens your computing world WIDE OPEN once you master it and lets you accomplish things that you never thought possible more quickly than you can possibly imagine IS the command line environment, documented by the man pages.

So while the Linux world has gone a long way to solving most of the desktop issues (including those you name) and is now providing a very plug-and-play Desktop environment, at least in the major distros like Fedora, I'm almost sorry about that, because it means that 90% of the people trying Linux will never touch the command line, never learn about it, never touch the Unix computing model or realize that computers in general are already much smarter and much more capable that most people in the Windows world realize.

It's like watching people starve to death right before your eyes.

crackrock

[SalsaDoom]

Hi there,

Reality check: OS X isn't unix, its just not so stop saying it is. Its a mach kernel with some bsd stuff tossed in. Its unixy, I'll give you, but not so unixy that its a unix. That a minor technical detail I suppose, it its annoying that mac posers are trying to call themselves unix guys now. Your not. :)

Now -- to the important issue of why your on crackrock. OS X is not free. Its not free in any sense of the word, or in any sense that matters.

Allow me to explain.

One, its not available freely under any circumstances, unless you purchase an Apple Mac computer. Well, I'd rather not, so it'll never be free for me in monitary terms.

Two, and more importantly, its not free as in freedom. Read the license its under, thats not really open source no matter what Bruce Perens and his gang at OSI says. Bruce is my homeboy from way back when, but he's just plain wrong on when it comes to Apple and it smells like a sell out. Its not free, its just not.

Therefore, Apple isn't offering us anything that is any good to the sort of people who care about Linux. I gather that you just use Linux because there are circumstances where you find it useful, thats fine. Enjoy. However, your not a real open source person because the issue of freedom doesn't matter to you, as you have made very clear with your fauning of Apple and its bastard OSX. I'm glad that you've found true happiness with OSX, but allow me to be clear on this: We don't care, because OSX doesn't make us (Open Source, Linux and freedom loving people) happy, because it doesn't suit any of our needs. So if you think linux missed the boat, thats fine, but its also entirely irrelevent to absolutely everything. Linux is progressing at a steady rate everywhere.

Even on the desktop linux is slowly but surely pushing forward. Thats fine with us. I gather its not fine with you, so, stick with OSX. But don't tell us that our solution is no good because we like ours better then we like yours.

--SD

Re:Linux is too fragmented

[$RANDOMLUSER]

OK, I've got the karma to take the OT hit to answer you.

You've been incrementally learning Windows for 10 years now. Every time you change versions you have to go through another learning-curve bump. "Where did they put "ODBC Drivers" now?". If you were suddenly presented with learning Windows on a tabula rasa, your learning curve and frustration level would be just as high as they are for a Windows user moving to Linux for the first time.

If you're a programmer, let me ask you this: How many text editors have you had to learn? Isn't it a pain in the ass learning a new one? "Hell, I already know 43 editors, I have no desire to learn another one". This does not make any of the editors you already know superior to the one you don't, nor does it make the new one inferior just because you don't. Different isn't a priori bad, it's just different.

The problem is the LSB does not PUSH LINUX FORWARD

[furry_wookie]

The idea of a common set of standards for lots of stuff obviously has many potential benifits for Linux.

The problem with the LSB is it does not do much. What is needed is not a standard for "thou shalt have this version of libc in this directory", but instead a standards body needs to come up with "this is the way you will perform your system initilzation", "this is how you will set and store your ip networking configuration" etc...this would make YOUR skills transferable from distro to distro, would allow the community to come up with BEST OF BREED solutions for things like system configuration tools etc.

Having 1000 different distros do this stuff in 1000 different ways is WORSE THAN not being able to run Oracle on a particular distro without a little tweaking.

Re:WE NEED STANDARDS

[An Onerous Coward]

Hey, if you want to create a "setup.exe" with a nice, friendly icon, that is actually just:

#!/usr/bin/bash
apt-get install openoffice

then knock yourself out.

Better yet, use Synaptic.

Even better, try not being a plagarizing troll. Go outside and get some fresh air, perhaps also try dating. You'll be happier.

Re:currently leads Glibc

[Tet]

I wish he'd put a XML parser into glic so that no-one has an excuse for not using XML for configuration files and for data export / import.

Were there one available, I would still be unlikely to use it. The fact remains that after you've seen through all the marketing hype, XML remains inappropriate for many tasks, and configuration files are right at the top of the list. You only have to look at Jabber or Tomcat to see some perfect examples of that.

Re:Thought-Out, or Whining?

[ArsenneLupin]

Errm, actually there is a single point in the piece: there are *huge* bugs in the test cases.

All other points raised are shown to be consequences of this.

The specific example he cited is a rather enormous bug (a thread which is detached can by definition not be joined. "Detaching" a thread means telling the system that you are not interested in its exit status... and join()ing is reading the exit status).

(This doesn't mean that other examples are as clear cut. It could still be that most tests do actually show genuine glibc bugs, and that he just picked up the right example to bolster his point.)

that the audit raises many bugs

... in the test cases ...

that other distros "somehow" achieve their compliance but he's not sure how

I'd say, if Ulrich is right about the test cases, the situation should be fixed by removing/rewriting the dodgy test cases althogher. Deliberately running distros with non-standard shared libraries or on dog-slow hardware to make them succeed the tests is pointless. If that is indeed how "somehow" some distros achieve to pass the tests, Ulrich is indeed right on the mark that it would make the test suite completely meaningless. You are not certifying a distribution, but you are certifying a distribution tweaked to run the tests...

Better fix the suite, and run the distro under "normal" conditions (i.e. the same as normal users would do).

Re:Linux is too fragmented

[PsychicX]

How many text editors have you had to learn?

Should I have to learn a text editor? Sure, I had to learn emacs and vim. What about nano? Nano was obvious. It listed the commands at the bottom. Sure it's not the most powerful editor around, but still. It's a freaking text editor. I should be able to open a file, type stuff in, save, and quit, without ever having seen the editor before, and without having to read the man page. None of the GUI editors really suffer from this problem, since they have menus and toolbars to fall back on. But for some reason, the DOS style alt menus in console never caught on in the Linux world, not even as an option.

Sorry, just had to get that one out.

Re:Thought-Out, or Whining?

[Rakshasa Taisab]

If you have broken tests you either fix or deprecate them. What you absolutely do *not* do is break the code being tested, just so it can pass the test.

Well, you can... But then they are a hinderance, not a benefit.

Re:Thought-Out, or Whining?

[iabervon]

He's not whining that it's hard. He's whining that it's impossible, because the tests don't match the either the standards or common practice. He's whining that distros must be somehow faking compliance, because they ship *his software* which doesn't "pass" the buggy tests.

His argument is: no set of Linux software could pass the LSB suite by actually consistantly giving the desired results, because there's no libc that consistantly gives those results (when run on sufficiently fast hardware to expose the bugs in the tests, for example); yet distros do claim to pass the suite; therefore, the LSB is not ensuring compatibility, because it certifies things that don't work by their rules.

Furthermore, he argues that programs that don't work tend not to work because they rely on undefined behavior. Certifying that the environment behaves in accordance with the standard doesn't help, because the software developer's environment and the user's environment may do different things in some cases, while both comply with the standard. Unless the programs are tested for doing non-standard things, they won't necessarily work. And the undefined behavior is undefined for a reason: you can't improve the system without changing it (especially when the thing not defined is which takes longer: executing a certain function or waiting .001 seconds). And the same cases are particularly hard to test programs' assumptions about.

The sections that you dismiss as whining are actually providing examples, which is important in engineering (or science). There are theoretical flaws in any process; it is always important to know whether those situations ever actually occur. If he didn't have an example of a program relying on undefined behavior which should vary between systems, one could say that nobody would actually write code like that and think that it worked; but it turns out that people actually do write such code, and these people happen to include the people writing LSB tests, which is why they're flawed tests.

Linux File System Standard

[Lodragandraoidh]

The LFSS (Linux File System Standard) is the main standard I am really concerned about; if developers and OS distributions would stick to that it would solve a great deal of the problems I see when installing applications.

The LSB is overrated imho.

Re:Let's forget binary compatibility

[therealking]

You sir are a Class A moron, who has no idea what he's talking about.

Binary compatibility is EXTREMELY important to Linux if you want acceptance on the same level as Windows or OSX.

If you make any really drastic changes -- the standard C library for instance -- you might well have to recompile some applications. Is that a major hardship? I don't think so.

This laughable. That I even have to compile an app is laughable.

End users do not want to compile and application. They do not want to debug it, figure out what version of a library is running, download nessesary components from 3rd party websites, or even think about the OS. No one has time for that.

They want to slide in the CD and click install. Answer as few YES/NO questions as possible, and start using the application. Thats it. Everything else is a road block to getting a the original task done.

I swear many /.ers don't acctually do any work, they just theorize on what work would be like if they could do it after they get thier distro installed.

Re:Linux needs a stable ABI *BADLY*

[An Ominous Cow Erred]

Yes, and it's very dogmatic. It presents good reasons for the internal API to change, but IMHO the reasons against not offering a stable ABI for external driver support amount more to calling the developers of these "leeches".

Guess what? Tons of products now offer "linux support" by ugly wrappers or, worse yet, per-distro builds, or even WORSE no drivers at all.

Hell yeah I want open source drivers in the kernel. It's nice when the drivers come built-in! Lots of companies agree with this concept in the Win32 environment as well and various builds of the NT kernel ship with drivers for all sorts of hardware built-in.

The problem is not everyone is going to do this... ...and as a result I'm stuck sitting here porting Highpoint's stupid wrapper to 2.6.13. They're NOT going to give me open-source drivers, and there was no other card in its price range that did what we needed it to do here.

To whoever modded me flamebait -- it wasn't flamebait! I'm not trying to piss anyone off. I'm just saying that while it's Highpoint's fault somewhat for not updating their driver, it's also the fault of Linux for not having a better-structured way for deprecating and removing old code. (i.e. don't remove deprecated code in minor revisions!)

Fortunately the needed changes aren't that big (I had to familiarize myself with the APIs, both old and new, which is why I'm taking so long), but I shouldn't have to be doing this at all. That's why I'm grumpy.

So right now, with a server that needs to be deployed, would I benefit from a stable ABI (or at least a stable API!) for device drivers? Hell yeah! I would be doing other, more productive things with my time.

THANKFULLY drivers for the chipset family that the 1820A uses are finally starting to trickle into patches to the kernel, but they're in a very very very alpha state at the moment. Maybe in half a year this won't be an issue anymore. =/

Trying to strongarm companies into releasing open source drivers by making closed-source ones a bitch to make work will NOT convince them to open their code (witness ATI and nVidia). We have to show them other merits to opening their code (being installed by default being a good one to start with -- assistance from the community in bugfixes being another).

In order to accomodate stuff that is still closed, we need solid ABIs for things like drivers, for things like standard libraries. Right now the only one we can count on is the basic executable environment ABI. The only way you can count on THAT working is to have everything statically linked into the executeable... ...and for device drivers you're just screwed. =/

If we had these my job would have a lot fewer headaches and I could focus on more important tasks.

Re:currently leads Glibc

[Clover_Kicker]

Here's something I wrote in 2001 about this topic. I expect I'll still be quoting it in 2011.

I've seen a lot of people floating this idea about "XML config files everywhere" for the last few months.

I'll come out and say that I don't think this would provide much benefit. For the sake of argument we'll just pretend that I like the idea.

The big practical problem here is that you've got a helluva lot of code to rewrite before we see any payoff.

Lots of folks will disagree with you about the value of rewriting $application's config files in XML. Are you prepared to fork (and maintain!) a version of cron-XML and named-XML and ssh-XML and sendmail-XML and qmail-XML and xntpd-XML and $deity knows what else?

That's just the big "server" stuff, what about all of my trivial "desktop" apps (slrn, xscreensaver, xmms, whatever) ?

I'm sure there are a ton of brain-dead apps/scripts that manually parse /etc/hosts or /etc/fstab or /etc/passwd instead of going through the appropriate system calls; you'll be breaking them too. Some of those brain-dead proggies are going to be big $$$ commercial apps whose vendors won't want to change, I almost guarantee it.

There is soooo much inertia here that it's going to be very hard to get the ball rolling. Certainly not impossible, but hard.

Hey, prove me wrong. Get in there and start coding. Next time I see you, I'll buy you a beer for every app/syscall that you XML-ize. :)

Re:currently leads Glibc

[Boronx]

What you need is to move all of the config files to xml, but have old line-record type files be auto-generated from them to placate obsolete scripts and apps. /etc/fstab.xml is the real fstab, but an /etc/fstab is kept up to date for the dinosaurs.