Glibc Steering Committee Dissolves; Switches To Co-Operative Development Model

First time accepted submitter bheading writes "Following years under controversial leadership which, among other things, led to a fork (which was in turn adopted by some of the major distributions) the glibc development process has been reinvented to follow a slightly more informal, community-based model. Here's hoping glibc benefits from a welcome dose of pragmatism."

Comments

Summary

[TheRaven64]

The pissing match between RMS and Drepper that resulted in the steering committee is no longer longer relevant now Drepper has gone to work at Goldman Sachs (something that makes me smile: I can't think of any other company more deserving of him).

Re:Summary

[TrekkieGod]

Ahahaha, really? That's hilarious. Damn, I remember Drepper. I was told "he's quite nice in person", but you'd never know it from how he acted over email.

Heh...I read through the summary and thought, "actually, we could do with less pragmatism and more with just doing what is right regardless of what the people keep requesting. Design by committee is the path to bloat. Then I clicked the links and saw this comment by Drepper in response to a bug report on arm:

It's working fine everywhere but this carp architectures. I'm not going to make the code perform worse just because of Arm. Providing your own copy of that file if you care.

That's not lack of pragmatism, that's lack of professionalism. In fact, this is a good time to point out that "nice in person" is code for "asshole, but too much of a coward to behave like one if the other person is within punching distance."

Re:Summary

[bheading]

When I wrote about pragmatism I was thinking of this problem where a modification to glibc's malloc() implementation broke the Adobe flash player. It is worth contrasting the attitude of Linus Torvalds in that thread with that of the glibc maintainers. I think most reasonable people would agree there is a trade off between supporting broken applications and ensuring things are done right. In this case, it would have cost glibc nothing to make a minor concession.

Re:Summary

[TheLink]

They can't provide very good reasons to copy stuff backwards, yet they insist on doing it and it breaks stuff.

For similar stupid problems (and responses) sometimes I get the impression that the Linux Desktop bunch are actually sabotaging Desktop Linux.

Re:Summary

[iluvcapra]

Linux Desktop bunch are actually sabotaging Desktop Linux

Is that like the People's Front of Judea and the Judean People's Front?

Re:Summary

[TrekkieGod]

When I wrote about pragmatism I was thinking of this problem where a modification to glibc's malloc() implementation broke the Adobe flash player. It is worth contrasting the attitude of Linus Torvalds in that thread with that of the glibc maintainers. I think most reasonable people would agree there is a trade off between supporting broken applications and ensuring things are done right. In this case, it would have cost glibc nothing to make a minor concession.

I understand your point, that it doesn't break anything else to make the concession. That said, this is exactly what I mean by "we need less pragmatism." The way I see it, overlapping memcpy is a bug, so that's what needed to be fixed. You code according to an API, and the moment that applications start depending on how you implement that API internally, you're asking for trouble. Pragmatically, you'd get flash working, but I'd actually be in favor of leaving the backwards copy in there even if it serves no other purpose than to expose bugs in people's code using your library.

I think doing things the right way is more important than getting things working for now, because it prevents things from breaking in the future. More long term planning and less short-term concessions is the way to go. That said, "this doesn't work at all on arm" problem was an actual bug on glibc's side. If he thinks the new way of doing things hurts performance in other architectures, then he can provide the different code paths with optimizations if he cares to do so.

Re:Summary

[Anonymous Coward]

I understand your point, that it doesn't break anything else to make the concession. That said, this is exactly what I mean by "we need less pragmatism." The way I see it, overlapping memcpy is a bug, so that's what needed to be fixed. You code according to an API, and the moment that applications start depending on how you implement that API internally, you're asking for trouble. Pragmatically, you'd get flash working, but I'd actually be in favor of leaving the backwards copy in there even if it serves no other purpose than to expose bugs in people's code using your library.

Sometimes there is a point to that type of thinking. This is not one of those times.

The original definition of memcpy() defined overlapping copies as "undefined" behavior not for any good API design reason which would stand the test of time (it's usually a bad idea for APIs to permit inputs which cause "undefined" behavior without returning an error code or throwing an exception or whatever), but rather because it made the implementation smaller.

That's because memcpy() is literally from the dawn of the C Standard Library, when the Bell Labs hackers were trying to squeeze as much functionality as they could into machines which look pathetically small by today's standards. Sometimes they went a little too far down the minimalism path. This was one of those times. Today, there is no legitimate reason to have that gotcha hiding in memcpy. It doesn't save enough space and it doesn't save enough time.

I think doing things the right way is more important than getting things working for now, because it prevents things from breaking in the future. More long term planning and less short-term concessions is the way to go.

At this point, the long term planning is in favor of just aliasing memcpy() to memmove(). The definition of memcpy() where it has "undefined" behavior for overlapping regions was one of those short term concessions to reality, ~40 years ago. We can move on. It won't hurt, promise. ;)

Re:Summary

[Guy Harris]

That's sound software development advice, I don't disagree with it. I'm not defending reverse memcpy, I'm merely saying that "flash doesn't work when we implement memcpy like this" isn't a good enough reason to change the implementation if the bug is on the flash side.

How about "Flash and some other programs in Fedora don't work correctly when we implement memcpy() like this, and some of them might "not work correctly" in ways that are security holes, and we currently have no tools to reliably analyze C code to catch incorrect use of memcpy(), and the failure modes are not always obvious, so it might be quite a long time before all the code out there is fixed?

No, people shouldn't use memcpy() in calls where the source and destination regions overlap, but if it's going to be a long time before those are all fixed, and if the failure modes can be insufficiently obvious that the bugs might not even be discovered before they do damage, this might be like {warning, car analogy ahead!} going through a traffic intersection with a 2-way stop, on the street without the stop signs, without slowing down and checking for oncoming traffic on the other street, because people shouldn't go through stop signs.

Here lies the body of Jonathan Day,
who died defending his right of way.
He was right, dead right, as he sped along,
but he's just as dead as if he'd been wrong.

I think the key here is that the punishment for violating the C standard isn't that your program fails in an obvious fashion, it fails in a somewhat random fashion. If memcpy() would check for an overlap and abort the process if it discovered it, fine, but, at that point, it could, instead, check for an overlap and punt to memmove() with no loss of performance from the version that aborts in the case where there is no overlap and with a gain of "programs don't just barf mysteriously to the end user" in the case where there is.

fork valley

[buchner.johannes]

There are a couple of forks apparently. Why does this have to be one monolithic library?

For this reason, several alternative C standard libraries have been created which emphasize a smaller footprint. Among them are Bionic (based mostly on libc from BSD and used in Android[14]), dietlibc, uClibc, Newlib, Klibc, and EGLIBC (used in Debian, Ubuntu and ArkLinux).

https://en.wikipedia.org/wiki/GNU_C_Library

Re:fork valley

[TheRaven64]

Those are not forks, they are different implementations. The Android libc is based on FreeBSD libc with some tweaks. It does not share code with glibc.

Excessive "modularity" can become hell.

[Anonymous Coward]

Monolithic libraries are the way to go. They make software development much easier.

If you don't believe me, just look at the GNOME project. The last time I tried to build it from scratch by hand, there must have been at least 50 libraries I had to build first. That was several years ago, so there are probably many more that are needed now. Those were just libraries from the GNOME project, too! That's not including glibc, the many X libraries, Gtk+, and so forth! Don't forget that you'll need to start making sure you're using the right versions, too. Some of these libraries are released yearly, while others have a new release every week.

To realistically build something like GNOME, where they went absolutely stupid with unnecessary modularity, you need to use one of the scripts that are out there that'll do it all for you. Those scripts end up being a solution to a problem that shouldn't exist in the first place! They're only needed because what should be one monolithic library was split out into 60 smaller libraries. You'll need all of the libraries to get even a basic GNOME installation up and running, so there's no point in separating them.

It's not the 1980s any longer. We don't statically link everything using dumb linkers that can't strip out unused executable code. Modern OSes using dynamic linking and delayed loading only ever use the parts of libraries that are actually used.

Re:Excessive "modularity" can become hell.

[Anonymous Coward]

Agree on that. Similarly, even though GTK+ purports to be cross platform, I never use it because of the many libraries it requires, GLib, ATK, Pango, WhatEver. It's just too much pain to get them all right and distribute the stuff to somebody not running GNU/Linux. GTK+/GNOME guys suffer badly from overgeneralization. They just don't realize that nobody is going to give a shit about their complex system and want to modify its tiny little details using their wonderful module system.

Re:Excessive "modularity" can become hell.

[slashbart]

Amen!
I like Qt's approach with only a couple of large libraries (QtCore, QtGui, QtXml, ...) where each has a very clear usage, and if you don't want graphics you don't use QtGui, but if you do, everything is in QtGui. Here's the list

Re:Excessive "modularity" can become hell.

[3dr]

+2 that.

The building complexity is compounded also by a project's over-specification of dependencies. That is, instead of just requiring FooLib x.y, they are requiring x.y.z. Part of the problem here is that projects use dotted-version notation inconsistently. I would never expect an API change with a .z change, and I would only expect an API change to add functionality with a .y change but never a break in current code. A change in major version means anything goes. Many projects follow this. I wish I had kept notes of those I've run into that do not.

Several years ago I was building a couple projects on Linux, and after downloading all the packages that weren't already on the system, I got things built. It was extremely time consuming, but it worked. Whatever happiness I had was soon vanquished because another program I wanted to use required one of the same libraries but with a different .z version, which, in some case, could cause a huge rippling of library do-overs. Ridiculous.

As a related aside, one thing I would like to see is a migration of projects away from autoconf tools. I've been using cmake for over a year now to build cross-platform tools and it does everything autoconf could clumsily do, and better. It is truly a better tool for this job.

Re:Excessive "modularity" can become hell.

[skids]

The building complexity is compounded also by a project's over-specification of dependencies. That is, instead of just requiring FooLib x.y, they are requiring x.y.z.

Really this is a matter of not having the assets to do proper testing. A good test-driven-development and automated test environment would allow developers who are just trying to be cautious to be more confident about allowing more versions of dependencies.

It's not a fork

[aglider]

It's clearly written in the fery first FAQ:

EGLIBC is not meant to be a fork of GLIBC.

The icon is wrong

[phoxix]

It should be GNU, not Debian. Glibc is very much a GNU project. How do people not know GlibC is a GNU project?

Re:The icon is wrong

[bheading]

The Debian tag is there because they promoted the eglibc fork.

I am not quite sure that glibc is a GNU project. I thought for that to be true, the copyrights would have to be transferred to the FSF. Drepper posted a long rant a while back about how the FSF would do that over his cold, dead body.

Re:Drepper and Theo are great men. Respect them.

[Anonymous Coward]

They are vilified as being "mean" or "intolerant" by some, merely because they point out the foolishness of others in public.

They are vilified as being "mean" or "intolerant" because they act like mean intolerant assholes.

A truly great programmer does not feel the need to belittle others' efforts. A truly great project leader does not feel the need to be rude and dismissive.

Compare Linus Torvalds, who is just as opinionated and can be just as abrupt, and yet somehow manages not to be widely perceived as a mean intolerant asshole. Consider the possibility that this is because he is also frequently observed being polite and saying nice things in public.

Re:Drepper and Theo are great men. Respect them.

[TheRaven64]

Theo and Drepper are very different. Theo is usually technically correct and has no time for people who can't work out why for themselves. Drepper is very often wrong, and is still an asshat in these cases.

Re:Drepper and Theo are great men. Respect them.

[TheRaven64]

I don't have exact numbers here, but Ulrich has had a massive role in producing what may very well be the most widely used C library ever

Is this 'may be' as in the Wikipedia definition, meaning 'isn't'? FreeBSD libc - not GNU libc - is the basis of both the Android and OS X libc implementations, and either one of these uses likely dwarfs glibc installs. And that's before you get to the Microsoft one (which is crap, but is installed on every Windows system).

Re:Drepper and Theo are great men. Respect them.

[rgbrenner]

When I first started using Linux (about '97).. I emailed Torvalds to say that I thought Linux needed to advertise because a lot of people didn't know it existed. He actually responded and politely explained how the project is put together/why that wasn't going to happen.

Re:Drepper and Theo are great men. Respect them.

[TheRaven64]

Not that have some kind of allegiance to the glibc, but what do all those Linux based GPSs, routers, TVs, etc, not to mention all those servers?

Most embedded users don't use glibc either, they use something like newlib or uclibc, depending on the resource constraints.

Re:Drepper and Theo are great men. Respect them.

[donscarletti]

Well, there are people with a lot of ability who are able to cooperate with others and there are those who cannot. Theo should pick a smaller project than an operating system to do by himself and I'm sure he could do brilliantly and make it famous and widely used (OpenBSD is known by most of slashdot, mostly for its phenomenal strokes of brilliance in the areas Theo cares about and its gaping lack of features in other areas). Drepper on the other hand, well, I'm not sure what's so great about glibc, sure, sprintf never has crashed on me but fucked if I'm handing out any credit to the guy who manages stlib.h just because he manages it.

For a good laugh

[BlackPignouf]

Read http://sourceware.org/bugzilla/show_bug.cgi?id=4980 or http://sourceware.org/bugzilla/show_bug.cgi?id=4403 for a good laugh. /. trolls pale in comparison. :D

Re:Drepper and Theo are great men. Respect them.

[shutdown -p now]

I dunno; GP has actually signed his post, unlike you, and anyone can go check his Slashdot comment history. In fact, I don't even need to do that, because I see his upmodded posts often enough - and, invariably, they are informative and correct when technical matters are involved (and he doesn't post about stuff he doesn't know, either).

Drepper, on the other hand... his stubbornness when pointed out that his code is plain wrong and non-portable is enough to judge him on his merits alone, regardless of the attitude problem that he has.