Slashdot Mirror
Microsoft to End DLL Confusion
MankyD writes "ZDNet is reporting that Microsoft is attempting to do away with DLL version conflicts in its next version of Windows with a technology it calls 'Strong Binding'. When new programs attempt to overwrite old versions of DLL's, the Strong Binding will index the DLL's and allow the programs to reference them by a unique ID, rather than by file name. Hopefully it will prevent a new program from breaking an old one. I would think this might add to DLL clutter however."
Sounds like a great idea. While there will be more DLLs in the registry, at least each and every program will have it's "own" DLL and can't be broken. Although I wonder if the software will default to the newest DLL and then go back if it doesn't function correctly.
Slashdot: Playing Favorites Since 1997
I never really understood the advantages of a DLL over a static lib in modern times.
In the old days, when diskspace & memory were precious goods, they made sense to share code. But today, what's the burden of 4MB extra app size compared to the DLL misery ?
Except for plugins, I see no reason why developers would need DLLs. Can anyone shed some light here ?
When will I end this grieving ? When will my future begin ?
I always find it interesting that Microsoft gets to announce and shake up the world with "a new feature" that they caused and of which at least one other major OS had long since solved. Versioning the library API? "Who would've thunk it?!?"
In other news, Microsoft invents a journaling file system to prevent data loss.. oh, wait..
Bel, the mostly sane.. "Of course I can't see anything! I'm standing on the shoulders of idiots." -- Me
I wonder how they deal with upgrades to DLLs where the the upgrade represents a security fix. In such circumstances one would definetly not want an application to use an old version of a particular DLL...
I could see some possible security problems with this. When a DLL needs to be patched for a security issue it will only fix programs witht the correct version of the DLL. and the old version of the DLL wont be fixed. This is bassicly defeting the porpose of DLLs in the fact most applications will be using simular but slitght different versions of the DLL. At this point why not just compile everything staticly with DLL that way it is no longer an issue. As well as quicker and easer to install and uninstall.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
Microsoft re-invents static linking.
Christ, if you are going to do this, why not create a recompiler that bundles the executable and all of its referenced DLLs into one EXE and be done with it.
As a nice side effect it'd make it a hell of a lot easier to move programs around your disk and between machines... Oh, ok, now I get it.
-josh
This is a capability of the .NET framework. It has nothing to do with the next version of Windows.
.NET assemblies. .NET assemblies are the equivalent of DLLs for the .NET system, but this won't solve the problem of "DLL hell" unless all applications are re-written in .NET.
.NET, but this article is just the Microsoft PR/marketing machine, and is nothing new technically.
It isn't really DLLs either, but rather
This is a great feature of
Librarys have version numbers /lib you will see you have several links to various version of a c library e.g. libc.so.6, libc.so.5.
look in
These point to the actual library.
As long as a libraries API doesn't change between major versions (as it should) there is no problem.
Jeroen
Secure messaging: http://quickmsg.vreeken.net/
In addition, the OS should be intelligent enough to know when an EXE's been manually deleted (thrown in the Recycle Bin). The current practice of placing all unstallation responsibilities on the vendor tends to result in DLL buildup when the uninstaller doesn't work right or isn't provided (not uncommon.) There should be a unified process for adding a DLL that links it to the executable file that requires it.
I thought Microsoft already claimed to have fixed "DLL Hell" once or twice before with Windows 2000 and Windows XP. How many times are they going to "fix" the same problem?
One of the really annoying things about Microsoft is they always promise to fix something in the next version. It's always "next time" with them, but the problems never seem to go away.
Sounds a lot like what Apple did with OS X's Frameworks.
Read more about that here. Be sure to read through the section on Framework Versioning.
Also note that MacOS has long done a great job at packaging applications together so that the installer is unecessary.
I instruct people in Linux, and my biggest complaint with RPM is that the user must solve his dependencies himself.
E.g. we had made an installation, but left out the development tools. When you try to install gcc, it says which packages are missing, but not where you can find them. You have to dig them up yourself from the CD-ROM's, and sometimes you have to look on all of them.
I do not have any problems with the RPM system itself, but why has Red Hat still no system implemented like Debian apt ? After the installation it asks for the CD-ROM's, scans them and builds a database about what packages reside where.
So, in the case of gcc, it would say what packages are missing, select them automatically, load the needed packages onto the disk and asking for the appropriate CD-ROM whenever necessary.
This is much more friendly than the stock Red Hat approach. Oh, I know there are tools to do that with Red Hat, but you still have to install them yourself. It should come out of the box.
I don't get it. The sole idea about DLL is, that they are dynamically linked libraries (hence the name ;) shared by applications. This new idea suggest that, in theory, every program can have their own unique DLL. What's wrong with this picture? It's a workaround because today, programmers create DLL's that are not backwards compatible thus breaking older programs once the DLL has been overwritten. Yes, the workaround will work, but at the same time it undermines the idea of sharing libraries and it doesn't exactly urge programmers to write nice code that doesn't break existant functionality.
/Christian
Well this makes one start to wonder, if each program is going to have a unique DLL (or at least one shared with VERY few others) why bother to have DLLs at all? Why not just roll the library up into the application binary?!
It'll be interesting to see how this adds to the bloat, I imagine it won't take long for the average user to amass quite a number of these things, mostly doing the same job!
There must be a better solution than this!
According to Microsoft's Ivo Salmre, quoted in the article: "When a .Net component is installed onto the machine, the Global Assembly Cache looks at its version, its public key, its language information and creates a strong name for the component."
In a few cases in the past, backwards compatibility has been (slightly) broken by service packs and security fixes. How will they deal with that? Presumably, the public key of a library can be affected by a patch. If an application uses the strong binding to request a specific version of a DLL, does that mean that it will keep its own copy of the DLL without the patches? Or will it have to deal with potential incompatibilities introduced by the patches? How "strong" will this binding be?
And by the way, this idea looks rather similar to the usual UNIX shared libraries that allow an application to bind to libpng.so (version doesn't matter), libpng.so.1 (version 1 required) or libpng.so.1.0.89 (specific version and patchlevel required). The proposed system for DLLs does not seem to be very different from that.
-Raphaël
SlashDot regurgitates, period. NYT, Wired, whatever. The vast majority of the stories mentioned on SlashDot I've seen elsewhere, sometimes days earlier.
What's the problem?
This is not a "Breaking News" site, it's a community discussion board. One doesn't come here for "news," per se, but to read what like-minded people in the "geek community" think about that news.
You're getting upset because your dog doesn't 'meow.'
Correct me if I am wrong, but wouldnt that slow windows down even more. You computer will now have to search through your registry to find the correct DLL for your application. And what happens if you registry gets corrupted (not like that ever happens), then that you put you in an even worse position. Would it be easier to make the DLLs backwards compatible?
In order to solve "DLL confusion" in Windows, Microsoft are going to increase of the number of DLLs on the system, potentially by as much as there are applications installed, and give each DLL a unqiue, but symbolic only (xyz123:pdq098 versus msvcrt.dll for example) identifier.
One result is that from one machine to the next, not only will you not be sure what applications are using which DLLs, you will also have applications that use radically different identifiers for accessing their libraries.
This eliminates library confusion... how? I can't wait to have to troubleshoot it. Here's another solution Microsoft: document your standard libraries so that idiot application developers don't feel they need to re-invent the wheel and dump custom libraries all over the place.
Of course, the rest of us will continue using open source software.
Why bother.
It's funny reading through the different replies to this story-
.
Reply 1- "This is a horrible idea! Look at all the RAM/disk space this is going to use. M$ programmers are idiots!"
Reply 2- "VMS/Apple/*enter slanted fav OS here* already does this! This was a good idea when it was done 10 years ago by ____
Reply 3- An idea like this is so stupid, it will NEVER work right.
So, its a stupid idea that will never work EXCEPT it has already been done BUT will take up too many resources UNLESS it is done by our fav OS AND then thats okay
*grin*
---"What did I say that sounded like 'Tell me about your day?'"---
Well, it is one thing to say that application can now obtain the version of the DLL they want if they *explicitly ask for it*. It is another thing to say that, they always and forever, until the end of time get the DLL they were "compiled against" or "packaged with". It is not clear from the article which of these two situations is the case.
The point of shared libraries is that you CAN upgrade one single library and have many applications "automatically" inherit the changes. This is how you can update a file dialog for instance, without recompiling every single one of your GUI applications. This is a Good Thing. The question then becomes "why is this shit breaking so much". The right solution is a proper combination of carefully-followed deprecation and backwards compatibility rules (preferentially married with some sort of standard version naming convention), and the ability for applications to explicitly choose the library version they want (or even better yet, runtime configuration directive that can be set by the user or administrator) in the cases that *it is known that the new shared library is not backwards compatible*.
It's 10 PM. Do you know if you're un-American?
This has nothing to do with DLLs, but with .NET components. Read the article carefully! .NET application can bind to a specific version of a component, but that's not even news - it already exists in .NET 1.0 and some aspects of it were already available in COM years ago. .NET components are usually exported from DLLs, but this won't solve any of the problems with 'normal' DLLs.
A
The only aspect of it that has to do with DLLs is that
Indeed most libraries have subversions, but most apps just link to the major version. When an app insists it needs version 6.3.2.4.33 it gets nasty..
Stop spreading FUD. You can access any library you want with LD_PRELOAD. So if libfoo is at 6.3.4 and you have a 6.3.2.4.33 on the system that your app absolutely requires, a simple
will do the trick. In fact, I do this specifically for StarOffice so I can use my local copy of freetype2 with the bytecode hinter turned on instead of the version which comes with StarOffice.
As long as a libraries API doesn't change between major versions (as it should) there is no problem.
Unless the semantics of an API change subtly from one version of the DLL to the next. This is sometimes done to fix bugs, security holes, etc. in one version of the DLL. You wouldn't believe how many proprietary programs in practice rely on undocumented behaviors of specific versions of libraries.
Will I retire or break 10K?
OS/2 - as an example only - had a much better scheme where o/s stuff lived in its own space and the stuff you built/bought lived in its own space (and never the twain shall meet). On top of that, they implemented the idea of a LIBPATH env variable so that you could set the path OS/2 would take when looking for DLLs. Consequently, screwups were minimized, versioning was not an issue, built/bought software could be maintained easily, and (wait for it...) you could upgrade the o/s without blowing away all your apps!
Can't wait to hear what MS 'discovers' next!
CrazyLegs
"Pork!!" said the Fish, and we all laughed.
The artical doesn't make it very clear, but I think it only applies to applications developed using the .NET framework. So, for all those other applications out there it's dll hell as usual ;)
First, what are the advantages of DLLs?
- Less Memory Footprint
- Less Disk Footprint
- Global Security Fixes
- Use of third party binaries
- Plug ins
Less Memory FootprintIn Unix, when you have two instances of an application running, say, vi, the executable code between the two is automatically shared. The shared library gains you nothing. To gain memory footprint, you need to use the same shared library from two applications at the same time. For example, libc might be used by vi and cc.
However, if you compile statically, you bind in only the routines that are needed. For shared libraries, you need to have all routines available, since you don't know which of them are used. Now, your virtual memory system may notice that a shared libary page isn't used, and page it out. Yes, this requires additional run time execution time. The upshot is that you save memory only when you have enough different programs use the same shared library to overcome the overhead. I claim that this happens with libc, libX11, and not a whole lot else.
Less Disk Footprint
If you have 50 programs that use the same shared library, you can save some disk space becasue that libary code does not need to be duplicated that many times. However, shared libraries need to have the symbol information requried to perform the dynamic binding. The savings isn't that much.
In the old days, when an entire Unix distribution fit on a 150 MB tape, the libc shared library savings amounted to about 30%. You could get more reduction in size by using compression.
In fact, programs could be compressed on disk. The loader could decompress the image as it reads it into RAM. For slow disks, this may be faster than loading the uncompressed version into RAM. The down side here is that you then may not be able to use the original file on disk for virtual memory paging.
In any case, it's getting hard to get a disk drive under 20 GB. 30% overhead reduction for the most common shared library doesn't amount to much.
Global Security Fixes
So, your libzlib.so.5 has a bug. You whip up a quick fix, create a new libzlib.so.5, and drop it into your system. You've just fixed all of your libzlib dependent programs, right? In fact, you fixed programs you didn't even know used libzlib. You may also have broken programs that you didn't know use libzlib. And, short of testing every program on your system, you don't know. The more complex the patch, the more likely you are to have broken many things.
Quick. What is a utility which will tell you all the shared libraries that an application uses?
Use of third party binaries
Third party binaries can just as easily be distributed in source form or in a library that is statically bindable. Static binding is preferable, since you are unlikely to use a large fraction of a kitchen sink shared library - where the authors have no idea how the programmers will use it. Source is preferable, since the documentation rarely specifies enough semantic detail to allow proper use.
Plug ins
OK. Your application is Apache, and you want some real flexibility. If Apache is compiled so that modules can be loaded at run time, then the administrator can add the new module and turn on it's use in the configuration. This doesn't save any RAM or disk, but it may allow the admin to change a line of config, restart the web server, and start using some new feature.
For Apache, the admin can also recompile with the new module compiled statically. I've done it both ways. My measurements show a small run time advantage to static compilation.
Granted, if you can't recompile IIS, then DLLs will give you the same flexibility in exchange for a small performance penalty.
The Dark Side of Shared Libraries
If you compile your application statically, then upgrade your OS, you can copy the old application to the new OS, and it just runs.
If your app has shared libraries, you have to track them down on your old OS, and copy them to your new OS. If you make a mistake, and copy your old libc.so over your new one, you run the risk of trashing every program on your new system. Brilliant.
Take netscape as an example. It comes installed in it's own /usr/local/lib directory
tree. In /usr/local/bin, netscape is a
script which sets up the shared library search
path to include the libraries that netscape needs,
then runs the binary. This introduces script
overhead
and shell dependencies on a complicated package.
And, when you upgrade your OS, you still need
to find the old libc.so and copy it forward.
RPMs
Many seem to think that RPMs solve all these problems. However, many packages have bugs in their dependencies, etc. Many RPMs use different versions of the same shared libraries. I find that I have to override the dependencies to get stuff to install. Often, the requried package IS installed. Not just once in awhile. Much of the time. The difference between theory and practice is that, in theory, they are the same.
Conclusions
Shared libaries seldom save RAM or disk space. The problem with using them to fix bugs globally is that you don't know what you fixed, or even if you broke some things. Third party binaries should invariably be statically linked. In an open source environment, plug ins are not strictly needed. Shared libaries make OS upgrades more painful.
So, what I'd like is a Linux distribution with no shared libaries. The compiler, gcc, would be configured to compile statically by default. Then, after some years of running the system in production, and after adding hundreds of applications to it, I'd be able to upgrade to a new distribution without having to recompile or do the shared library search.
-- Stephen.
Unix has had library versioning from the very beginning. Shared library filenames specify what version of the shared library the file contains, and when programs load they can request a specific version thru the file name.
And here comes M$ taking the same idea, and adding a point of failure in the form of some binary index of dlls. Jeezz this is just another thing I'm gonna have to fix when my windows friends start having trouble with thier computer. Really unnecessary. Couldn't they have just outright copied the Unix method? At least then they would have done it right.
I code in assembly and a few other languages. I can understand that it can be a very good thing to reuse one piece of code in several different places. I understand also that it can save space to reuse. No news here...
As for the idea of "Strong Binding", I wonder what Billy G. expects to acomplish by adding yet more poorly designed, poorly documented LIBs to the programming mess that Windows has evolved into. On top of that, I wonder why I would need to save EVERY SINGLE VERSION of a DLL that makes it to release...
Version tracking will become a nightmare.
Consider:
+User installs program COOL_PROGRAM.EXE
-COOL_PROGRAM uses MS_COOLNESS.DLL
+User gets an update to MS_COOLNESS.DLL:
MS_COOLNESS_V2.DLL
-The fix in V2 repairs a buffer overflow
in a function that COOL_PROGRAM used from
COOLNESS.DLL.
Question : Does the installer for V2 know that COOL_PROGRAM is dependent on it? If this is the case, Billy G. is gonna have his hands full trying to keep track of what goes where with third party devs.
If not, perhaps COOL_PROGRAM will go by default to the newest version of COOLNESS.DLL. Ok, now Billy will likely contend with tracking and modifying functions that have previously been used in highly specialized ways for security/system critical functionality that Windows does not provide either by accident or by intention. So NOW third party devs developing well organized and functional code/programs are forced to keep up with the madness of Windows development to save space. Hmmm... Guess it got the better of the buffer overflow this time. Or maybe they introduced a new bug into the system {par for course with MS}...
Better yet, how about people developing security/system critical environments use their own code to avoid this whole mess? Ok, now you dont need DLLs do you? How about 3-5 times as many? Wait for the next Windows release? So the effort YOU made during XP to keep up with DLLs and other updates is pointless right? Or XP+1? The style of MS defines itself....
Security/system critical programs?...
Thats only one side?...
Ok. Try this:
Graphics, network comunication, encryption, file editing, database editing? Or maybe drivers, file converters, scripts, inter-app comunication, diagnostics?
The list goes on. The problems generated by and complications arising from this framework are not worth the hassle.
Instead of building a system where things get more complicated I would recomend a redesign of the system itself. Current and past states of instability/insecurity are more than I care to witness again. Billy has enough money to sit around daydreaming for the rest of his days while still paying his programers for doing nothing but daydreaming themselves for the rest of theirs... Perhaps they could get up off their butts and design a system from the ground up that is easy to use, safe, fast, and reliable for users old and new... Logical?
I love C programming. C++ and Java are lots of fun. But IF you want something done right the first time, assembly and careful thought is the only answer...
S-()-u-|-s-!-|)-E
Well, I was going to say this earlier, but the network hung. All I can do now is expound on this...
The largest EXE on my box is a little over 3 megs (it's AbiWord, by the way). The largest DLL on my box is a little over 5 megs (it's the bulk of the image loader/editor that came with a cheopo digicam I bought). Let's be really, really conservative and say that AbiWord decides to load that DLL. Yeah, I know it would never happen but this is just a worst-case scenario. That's 8 megs resident in memory. Now, how many windows do I typicly have open? 5 or 6, and many times it's the same app like IE or MSVC. Even under a worst-case scenario like 8 separate huge apps open, that's 64megs. Now of course this worst-case scenario is an extreme. I wager a more typical scenario with everything self-contained would result in less than 32 megs of code resident in memory. What's 32 megs cost? They don't even sell 32 meg modules most places. A lot of boxes are coming with half a gig, and if you want more you just grab for some loose change and snap it in.
Of course, apps aren't the only thing on the box. The System Information in Windows shows a lot of DLLs loaded by Windows, many of them legacy support. On a box with 128 megs of ram, I sometimes break over 50% resource utilization, but there's no noticeable impact on performance so who cares?
Now, weigh the cost of RAM against all the hours spent putzing with different dynamic library versions.
Plainly, dynamic libraries are a holdover from the days when memory costs and address-space limits were something to think about.
Now, I'm not saying that there aren't circumstances where dynamics are a good idea. For example, it would have been nice if Microsoft had installed MFC DLLs with earlier versions of Windows. I shudder to think of all the bandwidth wasted downloading those.
The "solution" of maintaining different versions of DLLs and giving them unique IDs is almost an admission of defeat that dynamics don't work. It's probably better to think of it as a way to ween people of dynamics, and of providing those who still want to use them with the option.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
You underestimate the effects of that. Sure, a few promiment programs can do that without issue. However, if every single binary on your system did this, the effect would be horrible.
/bin would be that size. 110 binaries in /bin. Currently du shows about 4.6M in there. This would grow to 187 MB, and that is using the conservative ls as a comparison, some would be significantly larger. In files that would be in /usr/bin and /usr/X11R6/bin, things get worse as they have more complex linking requirements, libraries that are a lot larger and do a lot more. If every miniscule GUI required to be the size of itself plus system plus whatever toolkit they chose to use, drive space would suck.
Quick example. ls is 68k by itseIf you add the size of all libraries it links to, it becomes about 1.7 MB on a typical system. I would say ls is pretty consevative in terms of linking, so I'll pretend everything in
Not only is drive space not a moot point, but this has implications in terms of consistency and interoporability. If applications all used internal versions of GUI libraries, there would be absolutely nothing enforcing any sort of consistency and complex inter-process communication becomes really difficult due to version mismatches.
XML is like violence. If it doesn't solve the problem, use more.
Given the current cost of massive hard disks, why are programs still putting DLLs anywhere outside of ...\Program Files\Foo\dlls? I think most people would be happier losing some extra space per program due to DLL redundancy than to keep dealing with shared libraries!
Let me explain this because many people seem not to understand.
.NET way of doing things is different though, and Microsoft won't logo or certify apps unless they follow these practices:
When a program installs a "shared" DLL, the assembly manager looks at the DLL version. One of three things will happen:
1. The DLL does not exist in the assembly cache - it is added.
2. The DLL exists, but all other instances of it are a different major/minor revision. (X.Y.0.0) In this case, the DLL is added to the assembly cache as a separate version.
3. The DLL exists in the cache, and the major/minor versions are the same. In this case, if the installing DLL has a newer revision (0.0.X.Y), then it will overwrite the old DLL. Otherwise, it is thrown away.
When a program executes, it's manifest specifys what major/minor version of the DLL it needs, and the assembly cache will fetch it. HOWEVER, bug fixes, etc are supposed to be changes to the revision numbers only, so if a bug fixed version of the DLL is installed, the app will use that version.
The assembly cache also keeps track of what set of DLLs go together. If version 1.2.7.X of FOO.DLL needs to also be run with 1.2.7.X of BAR.DLL, then the assembly cache can make sure a program never uses a mismatch, which has been a
MAJOR cause of difficult to track instability over the years.
The "new"
1. If you have a DLL only used by your application, install it in your application's folder.
2. If you have a suite or many apps that work together and use the same DLL, install it into program files\common\yourname.
3. ONLY install DLLs into the System folder if they are very very widely used, or are actual system objects or libraries. (I.E. your app needs a newer version of the microsoft common dialog runtime. In that case, you ship the MSM which has the latest version of common dialog and related libs that are all known to work together, for EACH version of windows. The Windows Installer knows how to read the MSM and pick the appropriate set of files for the current OS/service pack level you are on. That way, developers running Windows 2000 don't b0rk a Win9x user by shipping the w2k libraries.)
3a. An even easier way of handling things is to write your app for a specific service pack level on each OS (or possibly hotfix if a bug was fixed that is affecting your app.) In this way, you just tell your users "you need service pack X on OS Y, or service pack Z on OS A" to run the app.
Natural != (nontoxic || beneficial)
This is not clutter, apart from the fact that your list includes static links and directories, about two thirds of the dll's you listed are symbolic links to versioned filenames to avoid clutter.
He saw some dirty arabs and fired. Too bad it was just some friendly kurds, BBC reporters and his fellow cowboys.