Slashdot Mirror
Linux Applications And "glibc Hell"?
cybrthng asks: "How do you stop glibc hell on Linux? I thought I'd long lost the ever familiar DLL hell on Windows, but with Linux it breaks the applications so bad its not funny. Will Linux only be able to truely survive with applications available in source form? For instance take Oracle Applications, it is nearly impossible to install it on RedHat 7.0 or any glibc 2.2 based distro since the applications were built against 2.1.x. When you install this software it tries to relink itself with the correct libraries and fails miserably. You can however force it to use glibc-compat, but that isn't a solution for a production system. Do vendors have to recompile there applications for every kernel, every library and every distro? How can I use Linux when the core libraries don't seem to be forwards or backwards compatible across different distributions?"
...or rather, it is only relevant for C++ libraries, the C ABI has been stable for a long time.
So has the glibc ABI actually, except that it is not 100% bug compatible. I.e., applications that relies on bugs in the library in order to work, may break when new the library is updated.
GLibC doesn't do this. Everything's crammed in. And that is bound to make for problems.
IMHO, what GLibC needs to be is a skeleton library with a well-defined API but no innards. The innards would be in seperate, self-contained libraries, elsewhere on the system.
This would mean that upgrading the innards should not impact any application, because the application just sees the exoskeleton, and the API would be well-defined. The addition of new innard components would then -extend- the API, but all existing code would be guaranteed to still work, without relinking against some compat library.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Read the whole message that you responded to and he explains the problem *and* the fix that is possible on Unix type systems. I can have five apps of various vintages that each require a different set of libraries that they linked against.
/usr/$sysname-glibc20-linux/lib/ld-linux.so.2 \ --library-path /usr/$sysname-glibc20-linux/lib \ $netscape $defs $cl_opt "$@" ( real world example )
Like so:
exec
With names like:
libORBit.a
libORBit.la*
libORBit.so@
libORBit.so.0@
libORBit.so.0.5.1*
libORBit.so.0.5.6*
We can keep different versions of files for use, not like the different versions of mfc42.dll that all have the same name. If another version of a library is completely backwards compatible, then a simple symbolic link gives the complete name that the run-time linker is looking for.
Kindness is the language which the deaf can hear and the blind can see. - Mark Twain
http://sources.redhat.com/glibc/glibc-faq.html
tell your vendor to link it static (using .a libraries instead of .so).
also remind them that a "Linux" version is
meaningless, they should say "Linux/x86" or
"Linux/Alpha" or whatever.
I hate it when a vendor supplies a "Linux" version
that won't work on my hardware, and I can't tell until *after* I've downloaded it.
Getting tired of getting a copy of Oracle for Solaris 2.3, iPlanet for SunOS 4.1.3 and Veritas for Solaris 7 and finding that none of them support my Solaris 8 system. Dammit, what is Sun doing wrong!?
;-) for those who did not guess....
You'd think that you would actually have to pick an OS revision based on the least-common denominator of the supported platforms for your application needs!
Someone needs to go write a Python-based OS and then never change anything. That'll solve it.
...but nobody reads the documentation anymore, so they bring problems on themselves.
Using private interfaces, using static libraries, not using versioning even when shared libraries are in use... I'm not surprised Oracle had problems.
You cannot apply a technological solution to a sociological problem. (Edwards' Law)
For instance take Oracle Applications, it is nearly impossible to install it on RedHat 7.0 or any glibc 2.2 based distro since the applications were built against 2.1.x. When you install this software it tries to relink itself with the correct libraries and fails miserably.
If there are substantial glibc 2.1-> 2.2 problems it is really poor coding on the part of the vendors. The use of private (but available) glibc functions was made impossible in the changeover.
There are a few models that will work in this case. First, the older version of glibc can be included with Oracle, and set LD_LIBRARY_PATH or LD_PRELOAD to load those libraries first. Then there is no problem.
Talk to your vendor. Ultimately, if you want to pay to use their software, they have a responsibility to ensure you can use it with some ease.
Nonsense. The size of a data structure is an implementation issue, and should never be exposed to the user anyway. Access to such data structures should only be granted through opaque pointers and accessor functions. The user should not be allowed to allocate such a structure manually; they should be forced to call a library routine which does the work for them. Any library which exposes the size of any significant structure as part of its interface is poorly designed.
Unfortunately, C does not do much to encourage data insulation. Object oriented programming, and especially abstract classes, would be a great help in alleviating these sorts of problems. Read "Large Scale C++ Software Design" by John Lakos for some extensive discussion of insulation (the process of making an interface binary compatible without hindering implementation extensions).
NT and 95 have much better binary compatibility than different Linux distributions. Windows 2000 has almost perfect binary compatibility with NT and 9x. I find that I can write a program once in Windows and have it work on all my friends' computers without trouble. With Linux, on the other hand, I would never even try to distribute a compiled binary. Source code only. Of course, that is not a problem for me, since all of my code is GPL or LGPL.
The thing is, the Windows API passes everything around as handles. Handles are opaque pointers, meaning that the caller has no idea what sort of structure they point to. I suspect that a thread handle in 98 points to a data structure that bears no relation to a thread handle in NT, but does that cause binary compatibility problems? No, because Microsoft correctly used opaque pointers.
Now, most of POSIX and ANSI-C do similar things (FILE*, DIR*, etc.). I am not saying that I particularly like the Windows API (I don't). All I wanted to point out is that it is possible to make a library that can be extended without breaking binary compatibility.
One last point: Binary compatibility is useful for 100% open source systems! What if a critical bug is found in an older version of glibc, forcing you to upgrade? From the sounds of it, you would have to re-compile every program on your system to make it work! I don't want to do that! With a properly written library, a new version could be dropped in without disturbing anything. Hell, you might not even have to reboot.
------
When using real software like Oracle under linux, you find out what the requirements are for the application you're going to run, and install a compatible setup. You don't just run out to the ftp site, burn a copy of the latest distro of Mandrake, and expect every application you install onto the new system to work flawlessly. Maybe you can run something like apache on every machine everywhere, but Big Important things like Oracle generally have pretty specific system requirements, even under other unicies.
Java, for example, couples libraries and user code much less tightly, yet uses statically type checked interfaces. Java's type checking is actually unnecessarily strict: classes are considered incompatible on dynamic linking even though only some aspects of their implementation changed. ML implementations could easily do the same thing.
Also, the fact that languages like C++ and Java tie inheritance hierarchies to static type checking is an unnecessary and idiosyncratic restriction. You can have perfectly statically type-safe systems that do not have these kinds of inheritance constraints: as long as the compiler and/or linker determines that the aspects of the interfaces you are relying on are type-compatible, it can make the two ends fit together safely, no matter what other changes or additions have happened to the classes. The "signature" extension for GNU C++ did this at compile time, and something similar could be done by the dynamic linker when libraries are loaded.
The efficiency issue is not significant. Even for a completely dynamic object system like Objective-C, a good runtime will have hardly more overhead for a dynamic method call than a regular function call. Any of the systems based on static type checking I mentioned above would do even better. And Java, of course, can actually do better than C/C++ when it comes to libraries because the Java runtime can (and does) inline library code as native code at load/execution time.
Of course, sometimes, things just have to change incompatibly. But as far as I can tell, almost none of the changes in glibc (or most other C/C++ libraries I use regularly) should affect any user code. Almost any kind of library interface would be less problematic than what exists right now.
So, I agree: statically typed languages will not go away. But "DLL hell" is avoidable whether you use statically or dynamically typed languages. In fact, as I mentioned, you could even make it go away in C/C++ by introducing a special library calling convention that has a bit more information available at load time. However, why beat a dead horse?
I work for a company building a network montioring system available for FreeBSD, NT (and 2K), and both RedHat and Debian Linux. We're adding platforms as people request them.
:)
Really, RedHat 7.0 includes the libraries that shipped with 6.2, so while we only support RedHat 6.2 we still work out-of-the-box on RedHat 7.0. Why not use the compatibility libraries ? That's what they're there for - they're not performing worse or anything, they are just older versions of the library.
On UNIX-like systems you actually have VERSIONING on your system libraries. So you can have a perfectly running system with ten different versions of the C library, and each application will use the version it requires.
You're welcome to check out our beta-versions available from sysorb.com, if you don't believe me
First off, If you get anything out of this post it is this: DO NOT RUN A REDHAT X.0 RELEASE IF YOU DON'T UNDERSTAND LIBRARY VERSIONING
NO SUCH THING AS "GLIBC HELL"
There is _no_such_thing_ as "glibc Hell". UNIX (including Linux) has versioning on libraries -- right down tot he filename. _Unlike_ Windows, you can have _multiple_library_versions_ installed. Even Microsoft still has NOT addressed this (and I run into it daily) simply by versioning filenames of libraries. This is a _farce_ and the result of people not understanding the OS in front of them.
UNIX v. WINDOWS ON SYSTEM LIBRARIES
So, in a nutshell, you can have your library issues two ways:
- Recompile for new libraries (if you have source)
Otherwise, no option if you don't have source -- total SOL!Especially on Win2K which makes some libraries untouchable (and quite incompatible with a lot of existing software).
Microsoft does this for "stability", but it is a library- ignorant way of NOT addressing the _real_ issue, lib versioning.
LIBRARY VERSIONING AND SYMBOLIC LINKS
The main reason Windows cannot have UNIX-like library versioning and versioning on filenames is because it lacks symbolic links. With symbolic links, you can have multiple subrevision of a library, with one subrevision the "default" revision, with that (or another) the default "main" version. E.g.,:
libmy.so -> libmy.so.1
libmy.so.1 -> libmy.so.1.1
libmy.so.1.0 -> libmy.so.1.0.7
libmy.so.1.0.7
libmy.so.1.1 -> libmy.so.1.1.3
libmy.so.1.1.2
libmy.so.1.1.3
libmy.so.3 -> libmy.so.3.0
libmy.so.3.0 -> libmy.so.3.0.1
libmy.so.3.0.1
In the preceding example, there are actually only 4 library versions: 1.0.7, 1.1.2, 1.1.3 and 3.0.1. We could easily introduce more versions if programs required them. Most libraries are "parent revisioned" (I don't know what the "official term" is, but that's what I'll call it), so the latest "x.y.z" "version.revision.subrevision" is synlinked as "x.y" "version.revision" as well as "x" "version". As far as compatility between versions, anything goes (and is a per-library consideration), but the "general rule" is as follows:
Most OSS projects, including GLibC, have good versioning schemes that change subrevisions (the "Z" in x.y.z when updates, bugfixes, or non-structural changes are made -- meaning 1.1.2 and 1.1.3 are most likely header/function compatible. So, depending on the library, most programs are fine when linking against X.Y instead of x.y.z-- and do so to keep from requiring the user to have numerous differnet libraries installed. A simple x.y symlink to the latest x.y.z (latest being the max(Z)) is usually all it takes. Again, a "parent revision" symlink does the job.
Now different revisions (the "Y" in x.y.z) usually involve some header/function changes that _may_ be INcompatible. As such vendors usually do not link against just the major version (the "X" in x.y.z) for that reason. E.g., some programs work fine on any GLibC 2.y.z system (BTW, GLibC 2 is aka LibC 6), but most are tied to GLibC 2.0.z (RedHat 5.x), GLibC 2.1.z (RedHat 6.x) or GLibC 2.2.z (RedHat 7.x). Major version changes (again, the "X" in x.y.z) are left for radical, completely incompatible changes -- like LibC 4 (RedHat 3.x), LibC 5 (RedHat 4.x) and GLibC 2 (RedHat 5.x+ -- which caused a bigger stir than 7.0 awhile back ;-PPP).
DON'T RUN A REDHAT X.0 RELEASE UNLESS YOU KNOW WHAT YOU ARE DOING!
98% of the "bitching and moaning" about RedHat 7.0 comes from user naivity on library versioning. Yes, I _do_agree_ that RedHat did release 7.0 too earlier with unfinished components, but since patching GLibC and GCC through December, RedHat 7.0 is a _solid_ release. Never, never adopt a RedHat X.0 release unless you are a seasoned Linux user! Please get the word out on that (although the RedHat 7.0 README on CD 1 *DOES* stress that point too!!!)
[ Side note: The kernel is another matter though -- but understand RedHat cannot "time" the release of GLibC, GCC and the kernel since they are all independent development teams. ]
RedHat gives you _full_warning_ of the all-important GLibC change in a new release. It's always been RedHat's model -- introduce a new, and possibly INcompatible GLibC on a X.0 release. All revisions in a major release have the same GLibC and GCC, and are quite interchangable. I've said it before and I'll say it again, only RedHat seems to do this (although I haven't checked out Debian yet). So I know that going from 6.2 to 7.0 means issues, just like 4.2 to 5.0 did for me almost a half-decade ago. NOTHING NEW!
So, again, if you do NOT know what you are doing, stick with RedHat X.2 releases (or at least X.1)!!! At X.0 releases, most of the Linux world has NOT yet adopted the new GLibC versions -- hence the wealth of binary library incompatibilities. So if you are not familiar with how to deal with them, do NOT try to deal with a RedHat X.0 release!
REDHAT MAINTAINS LIBRARY COMPATIBILITY
Today's RedHat 7.0 (GLibC 2.2) release comes with full RedHat 6.x (GLibC 2.1) compat libs, even a compat devel libs and a compat compiler/linker. And you can also install compat libs from the RedHat 6.2 release for 4.x (LibC 5) and 5.x (GLibC 2.0) compatibility.
[ Side note: I *DO* have a "complaint" with RedHat for not including LibC5 and GLibC2.0 compatibility libraries with RedHat 7.0. And I've let Alan Cox know about them. They should _at_least_ be available on the Powertools CD. I know LibC4 shouldn't be included for security reasons, but LibC5 and, especially, GLibC2.0 should be! ]
ANY LIBRARY ISSUES ARE ALMOST ALWAYS THE PACKAGERS FAULT!
Now that we have the user out of the way, as long as a vendor/packager dynamically links against the specific library version, the binary will use that version. Too many vendors are used to just linking against what they have, especially if it is an older library on their development system. So when a user has a new version of the library, possible with function name, parameter and other structrual changes, core dumps will occur.
If a developer is really worried, there is always the option of statically linking -- i.e. putting the library in the binary itself (so no external library references/dependencies). Of course, there could be licensing issues with GPL or other OSS code to/from commerical and vice-versa. If people think this is limited to Linux, they are _gravely_mistaken_ as most commercial IDE and development tools introduce their own issues. Under Windows, where only one system library can be installed (and that library may be "fixed" in Win2K), you're in for a world of hurt trying to sort them out.
END-USER LIBRARY VERSION ADMINISTRATION
As previously discussed, symlinks are often used to "parent revision" a full "x.y.z" "version.revision.subrevision". Most of the time, RPMS and tarballs/makes installs do this for you. But sometimes, you'll have to administer and create them yourself. Again, doing this is usually easy (it's just a symlink -- "ln -s ") and the first thing to try when a program cannot find a library (e.g., libmy-2.0) where a library exists with a more complete version (e.g., libmy-2.0.1).
-- Bryan "TheBS" Smith
-- Bryan "TheBS" Smith
Independent Author, Consultant and Trainer
You have some interesting viewpoints, but I think you're avoiding the question rather than dealing with it.
In the programming language research community, the feeling is that dynamic languages are very good for things like scripting and prototyping, but are not as good an idea for large software systems.
The problem is twofold - first, as you mention, dynamic languages always get a performance hit. But the second reason - which you miss - might be more important - fewer errors can be detected at compile time... they would only turn up at runtime, or worse, end up as hard to detect bugs. Moreover, the runtime may fail in someplace other than where the error occured. For example, let's say you have a bunch of "polygon" objects in a linked list, and you mistakenly put a "circle" object in that list as well. Much later, you're traversing the list and expect to find a polygons, but instead you find a circle. Type error! But the real error was where you put the circle in the linked list. In a dynamically typed language, you'd have to look to see where the circle was inserted - and the bigger the software system, the harder that becomes. However, in a statically typed language, the compiler tells you right away "hey buddy, you're putting a circle in to polygon list. Fix that, or you don't get object code!".
I don't think that statically typed languages are going to go away. As it often is with issues with software development, the real problem is psychological rather than technological. If backwards compatibility across ".x" releases was a priority for the glibc team, perhaps we wouldn't have this problem. As it is, they are probably more driven to adding new features or fixing really bad old problems in ways which break compatibility... if there are people willing to work on the project who have different goals perhaps it may be time to fork libc again?
However, they offer clear conditions. First, they don't guarentee upwards-compatibility, that is code compiled against glibc 2.2 working with 2.1. Second, C++ is currently off limits (which will change with gcc 3.0). Third, it applies only to shared versions of the library. Fourth, private internal interfaces are off limits.
The Oracle problem is simple: they're using static libraries (ie, ar archives of object files). This doesn't work because symbol versioning (the magic that enables compatibility in shared libraries) isn't implemented for object files. HJ Lu has a page on this issue and possible resolutions.
90% of other compatibility problems result from using private interfaces. This happened to Star Office a while back.
The evaluation of an action as 'practical' . . . depends on what it is that one wishes to practice.
Bzzzzzzt! Wrong answer, thanks for playing.
Here's a clue for you - Win95 and NT4 are two TOTALLY SEPERATE PRODUCTS from seperate code bases, whereas glibc is glibc - the same (ha ha!) library, just different versions.
Of course, what you OUGHT to have written was try to search for 100% binary compatibility between say Windows 95 and Windows 98 or try to search for 100% binary compatibility between say Windows 2000 and Windows NT 4.0 which is extremely easy to do. But then why let trivial things like facts get in the way of a good troll, eh? :(
--
People should not be afraid of their governments - Governments should be afraid of their people.
Are there alternatives? Plenty, actually:
You could probably invent a new calling convention for C together with some changes to the dynamic linker that would address this problem for C libraries. While you are at it, you should probably also define a new ABI for C++, something that avoids "vtbl hell" using an approach to method invocation similar to Objective-C. These new calling conventions would be optional, so that you can pick one or the other, depending on whether you are calling within the same module or between different modules. Perhaps that's worth it given how much C/C++ code is out there, but it sure would be a lot of work to try and retrofit those languages. Why not use one of the dozens of languages that fix not just this problem but others as well?
A related approach is to still write a lot of stuff in C/C++ but wrap it up in a dynamic language and handle most of the library interactions through that. That was the dream of Tcl/Tk (too bad that the language itself had some limitations).
Altogether, I think the time to fix this in C/C++ has passed, and COM-like approaches didn't work out. My recommendation would be: write your code in a language that suffers less from these problems, Python and Java are my preference, and add C/C++ code to those when needed for efficiency or system calls.