Slashdot Mirror
Linux Applications And "glibc Hell"?
cybrthng asks: "How do you stop glibc hell on Linux? I thought I'd long lost the ever familiar DLL hell on Windows, but with Linux it breaks the applications so bad its not funny. Will Linux only be able to truely survive with applications available in source form? For instance take Oracle Applications, it is nearly impossible to install it on RedHat 7.0 or any glibc 2.2 based distro since the applications were built against 2.1.x. When you install this software it tries to relink itself with the correct libraries and fails miserably. You can however force it to use glibc-compat, but that isn't a solution for a production system. Do vendors have to recompile there applications for every kernel, every library and every distro? How can I use Linux when the core libraries don't seem to be forwards or backwards compatible across different distributions?"
I work for a company building a network montioring system available for FreeBSD, NT (and 2K), and both RedHat and Debian Linux. We're adding platforms as people request them.
:)
Really, RedHat 7.0 includes the libraries that shipped with 6.2, so while we only support RedHat 6.2 we still work out-of-the-box on RedHat 7.0. Why not use the compatibility libraries ? That's what they're there for - they're not performing worse or anything, they are just older versions of the library.
On UNIX-like systems you actually have VERSIONING on your system libraries. So you can have a perfectly running system with ten different versions of the C library, and each application will use the version it requires.
You're welcome to check out our beta-versions available from sysorb.com, if you don't believe me
It is not usually an option for a vendor to link statically because of license restrictions.
However, a vendor is allowed to ship a specific version of glibc and libstdc++ with the software, as long as they provide some reasonable access to the source code as well.
As posted somewhere else, that is what we ended up doing for the RedHat 6.2 port of our network monitoring software. We ship a version of libstdc++ that matches our binary, it is installed without interfering with the other versions of libstdc++ that may be installed on the system, and everyone's happy.
Really, I am surprised how well this stuff works, and I cannot understand why so many people keep complaining about how horrible the system is. I think it's brilliant. And programs can still share the shared libraries, it's not like the Win2K way of doing things, where each app ships it's own set of so-called "shared libraries".
First off, If you get anything out of this post it is this: DO NOT RUN A REDHAT X.0 RELEASE IF YOU DON'T UNDERSTAND LIBRARY VERSIONING
NO SUCH THING AS "GLIBC HELL"
There is _no_such_thing_ as "glibc Hell". UNIX (including Linux) has versioning on libraries -- right down tot he filename. _Unlike_ Windows, you can have _multiple_library_versions_ installed. Even Microsoft still has NOT addressed this (and I run into it daily) simply by versioning filenames of libraries. This is a _farce_ and the result of people not understanding the OS in front of them.
UNIX v. WINDOWS ON SYSTEM LIBRARIES
So, in a nutshell, you can have your library issues two ways:
- Recompile for new libraries (if you have source)
Otherwise, no option if you don't have source -- total SOL!Especially on Win2K which makes some libraries untouchable (and quite incompatible with a lot of existing software).
Microsoft does this for "stability", but it is a library- ignorant way of NOT addressing the _real_ issue, lib versioning.
LIBRARY VERSIONING AND SYMBOLIC LINKS
The main reason Windows cannot have UNIX-like library versioning and versioning on filenames is because it lacks symbolic links. With symbolic links, you can have multiple subrevision of a library, with one subrevision the "default" revision, with that (or another) the default "main" version. E.g.,:
libmy.so -> libmy.so.1
libmy.so.1 -> libmy.so.1.1
libmy.so.1.0 -> libmy.so.1.0.7
libmy.so.1.0.7
libmy.so.1.1 -> libmy.so.1.1.3
libmy.so.1.1.2
libmy.so.1.1.3
libmy.so.3 -> libmy.so.3.0
libmy.so.3.0 -> libmy.so.3.0.1
libmy.so.3.0.1
In the preceding example, there are actually only 4 library versions: 1.0.7, 1.1.2, 1.1.3 and 3.0.1. We could easily introduce more versions if programs required them. Most libraries are "parent revisioned" (I don't know what the "official term" is, but that's what I'll call it), so the latest "x.y.z" "version.revision.subrevision" is synlinked as "x.y" "version.revision" as well as "x" "version". As far as compatility between versions, anything goes (and is a per-library consideration), but the "general rule" is as follows:
Most OSS projects, including GLibC, have good versioning schemes that change subrevisions (the "Z" in x.y.z when updates, bugfixes, or non-structural changes are made -- meaning 1.1.2 and 1.1.3 are most likely header/function compatible. So, depending on the library, most programs are fine when linking against X.Y instead of x.y.z-- and do so to keep from requiring the user to have numerous differnet libraries installed. A simple x.y symlink to the latest x.y.z (latest being the max(Z)) is usually all it takes. Again, a "parent revision" symlink does the job.
Now different revisions (the "Y" in x.y.z) usually involve some header/function changes that _may_ be INcompatible. As such vendors usually do not link against just the major version (the "X" in x.y.z) for that reason. E.g., some programs work fine on any GLibC 2.y.z system (BTW, GLibC 2 is aka LibC 6), but most are tied to GLibC 2.0.z (RedHat 5.x), GLibC 2.1.z (RedHat 6.x) or GLibC 2.2.z (RedHat 7.x). Major version changes (again, the "X" in x.y.z) are left for radical, completely incompatible changes -- like LibC 4 (RedHat 3.x), LibC 5 (RedHat 4.x) and GLibC 2 (RedHat 5.x+ -- which caused a bigger stir than 7.0 awhile back ;-PPP).
DON'T RUN A REDHAT X.0 RELEASE UNLESS YOU KNOW WHAT YOU ARE DOING!
98% of the "bitching and moaning" about RedHat 7.0 comes from user naivity on library versioning. Yes, I _do_agree_ that RedHat did release 7.0 too earlier with unfinished components, but since patching GLibC and GCC through December, RedHat 7.0 is a _solid_ release. Never, never adopt a RedHat X.0 release unless you are a seasoned Linux user! Please get the word out on that (although the RedHat 7.0 README on CD 1 *DOES* stress that point too!!!)
[ Side note: The kernel is another matter though -- but understand RedHat cannot "time" the release of GLibC, GCC and the kernel since they are all independent development teams. ]
RedHat gives you _full_warning_ of the all-important GLibC change in a new release. It's always been RedHat's model -- introduce a new, and possibly INcompatible GLibC on a X.0 release. All revisions in a major release have the same GLibC and GCC, and are quite interchangable. I've said it before and I'll say it again, only RedHat seems to do this (although I haven't checked out Debian yet). So I know that going from 6.2 to 7.0 means issues, just like 4.2 to 5.0 did for me almost a half-decade ago. NOTHING NEW!
So, again, if you do NOT know what you are doing, stick with RedHat X.2 releases (or at least X.1)!!! At X.0 releases, most of the Linux world has NOT yet adopted the new GLibC versions -- hence the wealth of binary library incompatibilities. So if you are not familiar with how to deal with them, do NOT try to deal with a RedHat X.0 release!
REDHAT MAINTAINS LIBRARY COMPATIBILITY
Today's RedHat 7.0 (GLibC 2.2) release comes with full RedHat 6.x (GLibC 2.1) compat libs, even a compat devel libs and a compat compiler/linker. And you can also install compat libs from the RedHat 6.2 release for 4.x (LibC 5) and 5.x (GLibC 2.0) compatibility.
[ Side note: I *DO* have a "complaint" with RedHat for not including LibC5 and GLibC2.0 compatibility libraries with RedHat 7.0. And I've let Alan Cox know about them. They should _at_least_ be available on the Powertools CD. I know LibC4 shouldn't be included for security reasons, but LibC5 and, especially, GLibC2.0 should be! ]
ANY LIBRARY ISSUES ARE ALMOST ALWAYS THE PACKAGERS FAULT!
Now that we have the user out of the way, as long as a vendor/packager dynamically links against the specific library version, the binary will use that version. Too many vendors are used to just linking against what they have, especially if it is an older library on their development system. So when a user has a new version of the library, possible with function name, parameter and other structrual changes, core dumps will occur.
If a developer is really worried, there is always the option of statically linking -- i.e. putting the library in the binary itself (so no external library references/dependencies). Of course, there could be licensing issues with GPL or other OSS code to/from commerical and vice-versa. If people think this is limited to Linux, they are _gravely_mistaken_ as most commercial IDE and development tools introduce their own issues. Under Windows, where only one system library can be installed (and that library may be "fixed" in Win2K), you're in for a world of hurt trying to sort them out.
END-USER LIBRARY VERSION ADMINISTRATION
As previously discussed, symlinks are often used to "parent revision" a full "x.y.z" "version.revision.subrevision". Most of the time, RPMS and tarballs/makes installs do this for you. But sometimes, you'll have to administer and create them yourself. Again, doing this is usually easy (it's just a symlink -- "ln -s ") and the first thing to try when a program cannot find a library (e.g., libmy-2.0) where a library exists with a more complete version (e.g., libmy-2.0.1).
-- Bryan "TheBS" Smith
-- Bryan "TheBS" Smith
Independent Author, Consultant and Trainer
You have some interesting viewpoints, but I think you're avoiding the question rather than dealing with it.
In the programming language research community, the feeling is that dynamic languages are very good for things like scripting and prototyping, but are not as good an idea for large software systems.
The problem is twofold - first, as you mention, dynamic languages always get a performance hit. But the second reason - which you miss - might be more important - fewer errors can be detected at compile time... they would only turn up at runtime, or worse, end up as hard to detect bugs. Moreover, the runtime may fail in someplace other than where the error occured. For example, let's say you have a bunch of "polygon" objects in a linked list, and you mistakenly put a "circle" object in that list as well. Much later, you're traversing the list and expect to find a polygons, but instead you find a circle. Type error! But the real error was where you put the circle in the linked list. In a dynamically typed language, you'd have to look to see where the circle was inserted - and the bigger the software system, the harder that becomes. However, in a statically typed language, the compiler tells you right away "hey buddy, you're putting a circle in to polygon list. Fix that, or you don't get object code!".
I don't think that statically typed languages are going to go away. As it often is with issues with software development, the real problem is psychological rather than technological. If backwards compatibility across ".x" releases was a priority for the glibc team, perhaps we wouldn't have this problem. As it is, they are probably more driven to adding new features or fixing really bad old problems in ways which break compatibility... if there are people willing to work on the project who have different goals perhaps it may be time to fork libc again?
However, they offer clear conditions. First, they don't guarentee upwards-compatibility, that is code compiled against glibc 2.2 working with 2.1. Second, C++ is currently off limits (which will change with gcc 3.0). Third, it applies only to shared versions of the library. Fourth, private internal interfaces are off limits.
The Oracle problem is simple: they're using static libraries (ie, ar archives of object files). This doesn't work because symbol versioning (the magic that enables compatibility in shared libraries) isn't implemented for object files. HJ Lu has a page on this issue and possible resolutions.
90% of other compatibility problems result from using private interfaces. This happened to Star Office a while back.
The evaluation of an action as 'practical' . . . depends on what it is that one wishes to practice.
Bzzzzzzt! Wrong answer, thanks for playing.
Here's a clue for you - Win95 and NT4 are two TOTALLY SEPERATE PRODUCTS from seperate code bases, whereas glibc is glibc - the same (ha ha!) library, just different versions.
Of course, what you OUGHT to have written was try to search for 100% binary compatibility between say Windows 95 and Windows 98 or try to search for 100% binary compatibility between say Windows 2000 and Windows NT 4.0 which is extremely easy to do. But then why let trivial things like facts get in the way of a good troll, eh? :(
--
People should not be afraid of their governments - Governments should be afraid of their people.
Are there alternatives? Plenty, actually:
You could probably invent a new calling convention for C together with some changes to the dynamic linker that would address this problem for C libraries. While you are at it, you should probably also define a new ABI for C++, something that avoids "vtbl hell" using an approach to method invocation similar to Objective-C. These new calling conventions would be optional, so that you can pick one or the other, depending on whether you are calling within the same module or between different modules. Perhaps that's worth it given how much C/C++ code is out there, but it sure would be a lot of work to try and retrofit those languages. Why not use one of the dozens of languages that fix not just this problem but others as well?
A related approach is to still write a lot of stuff in C/C++ but wrap it up in a dynamic language and handle most of the library interactions through that. That was the dream of Tcl/Tk (too bad that the language itself had some limitations).
Altogether, I think the time to fix this in C/C++ has passed, and COM-like approaches didn't work out. My recommendation would be: write your code in a language that suffers less from these problems, Python and Java are my preference, and add C/C++ code to those when needed for efficiency or system calls.