Microsoft to End DLL Confusion

MankyD writes "ZDNet is reporting that Microsoft is attempting to do away with DLL version conflicts in its next version of Windows with a technology it calls 'Strong Binding'. When new programs attempt to overwrite old versions of DLL's, the Strong Binding will index the DLL's and allow the programs to reference them by a unique ID, rather than by file name. Hopefully it will prevent a new program from breaking an old one. I would think this might add to DLL clutter however."

Comments

DLL vs static libs

[selderrr]

I never really understood the advantages of a DLL over a static lib in modern times.

In the old days, when diskspace & memory were precious goods, they made sense to share code. But today, what's the burden of 4MB extra app size compared to the DLL misery ?

Except for plugins, I see no reason why developers would need DLLs. Can anyone shed some light here ?

Re:DLL vs static libs

[IWannaBeAnAC]

Well, on a typical system there might be a hundred or so processes. Add 4MB to all of them and its suddenly not insignificant anymore.

Plus, if two running processes are sharing a shared library, then a task swap doesn't completely blow away the cache.

Finally, I think with Windoze I think there is a mode in which DLL's can have global data, shared among all programs using it. Not sure though, its years since I did any windoze programming.

Re:DLL vs static libs

[infront314]

Bugs.

If there is a bug or a security problem in a DLL you only have to replace that DLL instead of all statically linked programs.

Remember the problems with zlib a year ago? Would you like to replace 500 applications or one library?

Re:DLL vs static libs

[Anonymous Coward]

There are plenty of reasons to want dynamic linking. Library code is code you didn't write. Therefore, you have no control over its quality or security. If there was a buffer overflow exploit in a system library, and all programs statically linked that library, you would need to update your ENTIRE SYSTEM to remove the vulnerability. If the code is saved in a shared library (DLL), you just need to update the single shared library.

It's a classic case of the elimination of duplication in computer code. By duplicating (statically linking) the library code into every app, you only increase the burden when you want to update the library.

Furthermore, on Debian GNU/Linux I never have "DLL misery" because my operating system is not brain-dead. Multiple versions of shared libraries can coexist, there is a consistent versioning scheme, and a competent team of people who check to make sure that apps use the correct versions of the shared libraries.

Re:DLL vs static libs

[Arethan]

Plenty.

Each static library you use in your code adds to the overall program size, as you've stated. Thus increasing load time when you run the app. A 4MB exe loads a lot faster than a 400MB exe. (At least when the bulk of the difference is actual code/runtime-data, rather than generic data tacked on to the end of the file.) When you have DLLs, the OS will reuse already loaded DLLs for your app, and will keep newly loaded DLLs in memory for a while even after the app shuts down. This tremendously speeds up application loading.

Plus it allows for version upgrades in the DLLs without requiring recompiles of the applications. Of course, this feature really isn't used in Windows, since MS can't seem to make an application run according to specs, most developers work around and sometimes even depend on the bugs that are in MS's code. Thus creating DLL version dependant apps. If MS goes back and fixes the broken DLLs, they will break a bunch of third party apps in the process. Probably some of their own as well.

Plus their is security. Applications do not have the same level of access as DLLs. In fact, if you want to do some low level hardware access in Windows, you MUST use a DLL as a thunking layer, since you can't access the hardware from code within a regular process.

There are other reasons, but I'm getting lazy, so I'll let someone else finish if they feel like it is needed.

In short, there are plenty of reasons to use dynamically loaded libraries. But the generally poor coding practices that occur on the Windows platform pretty much ruins almost all of them. It's actually pretty sad. Windows had a lot of potential to become a great OS. Too many fuck ups and shortcuts have wrecked that chance. Fixing it would require an entirely new code base. One that doesn't have native ABI support for old apps. Which, of course, MS will never do. At least there's open source software to do things right. :)

Welcome to VMS

[beldraen]

I always find it interesting that Microsoft gets to announce and shake up the world with "a new feature" that they caused and of which at least one other major OS had long since solved. Versioning the library API? "Who would've thunk it?!?"

In other news, Microsoft invents a journaling file system to prevent data loss.. oh, wait..

Re:Welcome to VMS

[Metaldsa]

When 3% of desktop computers change a feature it isn't news. When 90% of desktop computers change their dll system it can become a worthy newspost. I think its not about the feature but how it will affect the mom and dads who don't follow techie news. No longer will they become confused (or irate) when they install an app and another breaks.

Though this could bring slower performance I think it will be negligable to most windows users.

Re:Welcome to VMS

[vocaro]

Microsoft Windows has had versioning of DLLs for as long as I can remember, way back in the days of 3.x. The article says, "Windows and Windows applications have no notion of DLL version numbers," but this is false. You can see the version numbers for DLLs on your system by starting Windows Explorer and navigating to the c:\windows\system directory. Right-click on a DLL, then click Properties, then click the Version tab. You'll see the version numbers there, including the company name, copyright info, and other important details. Windows applications can retrieve this data by calling the GetFileVersionInfo function.

Security Issues vs. Api Versions

[psychofox]

I wonder how they deal with upgrades to DLLs where the the upgrade represents a security fix. In such circumstances one would definetly not want an application to use an old version of a particular DLL...

Re:Security Issues vs. Api Versions

[sward]

Apparently, if you have 10 applications that each make use of the same DLL, not only will you now have 10 copies of that DLL, but in order to upgrade that DLL (e.g. for a security fix), you'll have to wait for each application's vendor to release its own upgrade package. You would then have a variety of security holes in your set of applications, despite their supposed common use of the same DLL.

And the way most companies work, this upgrade package may also include changes to the application itself instead of being more focused as most here on Slashdot would prefer. The DLL fix may be the "main course", but you'll get a side of new bugs in the app, and possibly an upgrade fee as a cover charge.

Umm Security.

[jellomizer]

I could see some possible security problems with this. When a DLL needs to be patched for a security issue it will only fix programs witht the correct version of the DLL. and the old version of the DLL wont be fixed. This is bassicly defeting the porpose of DLLs in the fact most applications will be using simular but slitght different versions of the DLL. At this point why not just compile everything staticly with DLL that way it is no longer an issue. As well as quicker and easer to install and uninstall.

Again?

[Darth Daver]

I thought Microsoft already claimed to have fixed "DLL Hell" once or twice before with Windows 2000 and Windows XP. How many times are they going to "fix" the same problem?

One of the really annoying things about Microsoft is they always promise to fix something in the next version. It's always "next time" with them, but the problems never seem to go away.

Re:Slashdot is getting slow, lazy?

[RobotRunAmok]

SlashDot regurgitates, period. NYT, Wired, whatever. The vast majority of the stories mentioned on SlashDot I've seen elsewhere, sometimes days earlier.

What's the problem?

This is not a "Breaking News" site, it's a community discussion board. One doesn't come here for "news," per se, but to read what like-minded people in the "geek community" think about that news.

You're getting upset because your dog doesn't 'meow.'

So let me get this straight...

[Lethyos]

In order to solve "DLL confusion" in Windows, Microsoft are going to increase of the number of DLLs on the system, potentially by as much as there are applications installed, and give each DLL a unqiue, but symbolic only (xyz123:pdq098 versus msvcrt.dll for example) identifier.

One result is that from one machine to the next, not only will you not be sure what applications are using which DLLs, you will also have applications that use radically different identifiers for accessing their libraries.

This eliminates library confusion... how? I can't wait to have to troubleshoot it. Here's another solution Microsoft: document your standard libraries so that idiot application developers don't feel they need to re-invent the wheel and dump custom libraries all over the place.

Of course, the rest of us will continue using open source software.

Re:Auto-DLL Managment?

[bourne]

Sounds like a great idea.... each and every program will have it's "own" DLL

Call me old-fashioned, but I thought the point of dynamic libraries was to reduce program size and duplication of effort by allowing multiple programs to load common functions out of libraries.

So, it's a great idea, insofar as it completely negates the advantage of having DLLs in the first place. Why don't they just compile statically instead?

Re:Auto-DLL Managment?

[AltControlsDelete]

I have not read the article, but I think this simply works by allowing different versions of the same DLL. If you have two programs that require the same version of the same DLL, they'll both use that DLL. However, if you install a new program that wants to install a different version of an already-installed DLL, that program will use its version of the DLL while allowing other programs to continue using the pre-existing DLL.

Re:In other news

[yerricde]

As long as a libraries API doesn't change between major versions (as it should) there is no problem.

Unless the semantics of an API change subtly from one version of the DLL to the next. This is sometimes done to fix bugs, security holes, etc. in one version of the DLL. You wouldn't believe how many proprietary programs in practice rely on undocumented behaviors of specific versions of libraries.

Re:Part of .NET, not Windows

[wadetemp]

Thank you for pointing this out. There've been other good comments around the board about DLLs and how sharing should work, but you're evidentally the only one who's read the article and understands the technology.

The feature was present in .NET 1.0 as well.

I've always wondered, though... why did MS choose to keep the extension ".DLL" for .NET assemblies? That choice alone has caused more issues than I have ever seen (name conflicts between a COM DLL and a corresponding .NET interop assembly DLL, people trying to register a .NET assembly DLL with regsvr, VB6 allowing you to attempt to use a .NET DLL as a component when it doesn't work, etc. etc. etc.) While they may be "dynamic link libraries" in a traditional sense, they're not ".DLLs." Chalk the reporter up as one more of the confused.

Re:Auto-DLL Managment?

[gilesjuk]

They're probably doing it to break WINE as well. DLL problems aren't that bad with Win2k and XP, system DLLs are protected and with XP you can go back to a restore point.

If an app installer is so badly written that it messes up your installation then the software can't be much good either.

Static is the way to go

[suitti]

The physician takes an oath "to do no harm". I'm considering putting together a Linux distribution with everything compiled statically.

First, what are the advantages of DLLs?

• Less Memory Footprint
• Less Disk Footprint
• Global Security Fixes
• Use of third party binaries
• Plug ins

Less Memory Footprint

In Unix, when you have two instances of an application running, say, vi, the executable code between the two is automatically shared. The shared library gains you nothing. To gain memory footprint, you need to use the same shared library from two applications at the same time. For example, libc might be used by vi and cc.

However, if you compile statically, you bind in only the routines that are needed. For shared libraries, you need to have all routines available, since you don't know which of them are used. Now, your virtual memory system may notice that a shared libary page isn't used, and page it out. Yes, this requires additional run time execution time. The upshot is that you save memory only when you have enough different programs use the same shared library to overcome the overhead. I claim that this happens with libc, libX11, and not a whole lot else.

Less Disk Footprint

If you have 50 programs that use the same shared library, you can save some disk space becasue that libary code does not need to be duplicated that many times. However, shared libraries need to have the symbol information requried to perform the dynamic binding. The savings isn't that much.

In the old days, when an entire Unix distribution fit on a 150 MB tape, the libc shared library savings amounted to about 30%. You could get more reduction in size by using compression.

In fact, programs could be compressed on disk. The loader could decompress the image as it reads it into RAM. For slow disks, this may be faster than loading the uncompressed version into RAM. The down side here is that you then may not be able to use the original file on disk for virtual memory paging.

In any case, it's getting hard to get a disk drive under 20 GB. 30% overhead reduction for the most common shared library doesn't amount to much.

Global Security Fixes

So, your libzlib.so.5 has a bug. You whip up a quick fix, create a new libzlib.so.5, and drop it into your system. You've just fixed all of your libzlib dependent programs, right? In fact, you fixed programs you didn't even know used libzlib. You may also have broken programs that you didn't know use libzlib. And, short of testing every program on your system, you don't know. The more complex the patch, the more likely you are to have broken many things.

Quick. What is a utility which will tell you all the shared libraries that an application uses?

Use of third party binaries

Third party binaries can just as easily be distributed in source form or in a library that is statically bindable. Static binding is preferable, since you are unlikely to use a large fraction of a kitchen sink shared library - where the authors have no idea how the programmers will use it. Source is preferable, since the documentation rarely specifies enough semantic detail to allow proper use.

Plug ins

OK. Your application is Apache, and you want some real flexibility. If Apache is compiled so that modules can be loaded at run time, then the administrator can add the new module and turn on it's use in the configuration. This doesn't save any RAM or disk, but it may allow the admin to change a line of config, restart the web server, and start using some new feature.

For Apache, the admin can also recompile with the new module compiled statically. I've done it both ways. My measurements show a small run time advantage to static compilation.

Granted, if you can't recompile IIS, then DLLs will give you the same flexibility in exchange for a small performance penalty.

The Dark Side of Shared Libraries

If you compile your application statically, then upgrade your OS, you can copy the old application to the new OS, and it just runs.

If your app has shared libraries, you have to track them down on your old OS, and copy them to your new OS. If you make a mistake, and copy your old libc.so over your new one, you run the risk of trashing every program on your new system. Brilliant.

Take netscape as an example. It comes installed in it's own /usr/local/lib directory tree. In /usr/local/bin, netscape is a script which sets up the shared library search path to include the libraries that netscape needs, then runs the binary. This introduces script overhead and shell dependencies on a complicated package. And, when you upgrade your OS, you still need to find the old libc.so and copy it forward.

RPMs

Many seem to think that RPMs solve all these problems. However, many packages have bugs in their dependencies, etc. Many RPMs use different versions of the same shared libraries. I find that I have to override the dependencies to get stuff to install. Often, the requried package IS installed. Not just once in awhile. Much of the time. The difference between theory and practice is that, in theory, they are the same.

Conclusions

Shared libaries seldom save RAM or disk space. The problem with using them to fix bugs globally is that you don't know what you fixed, or even if you broke some things. Third party binaries should invariably be statically linked. In an open source environment, plug ins are not strictly needed. Shared libaries make OS upgrades more painful.

So, what I'd like is a Linux distribution with no shared libaries. The compiler, gcc, would be configured to compile statically by default. Then, after some years of running the system in production, and after adding hundreds of applications to it, I'd be able to upgrade to a new distribution without having to recompile or do the shared library search.

Hmmm, this sounds like fun...

[Anonymous Coward]

I code in assembly and a few other languages. I can understand that it can be a very good thing to reuse one piece of code in several different places. I understand also that it can save space to reuse. No news here...

As for the idea of "Strong Binding", I wonder what Billy G. expects to acomplish by adding yet more poorly designed, poorly documented LIBs to the programming mess that Windows has evolved into. On top of that, I wonder why I would need to save EVERY SINGLE VERSION of a DLL that makes it to release...

Version tracking will become a nightmare.

Consider:
+User installs program COOL_PROGRAM.EXE
-COOL_PROGRAM uses MS_COOLNESS.DLL
+User gets an update to MS_COOLNESS.DLL:
MS_COOLNESS_V2.DLL
-The fix in V2 repairs a buffer overflow
in a function that COOL_PROGRAM used from
COOLNESS.DLL.
Question : Does the installer for V2 know that COOL_PROGRAM is dependent on it? If this is the case, Billy G. is gonna have his hands full trying to keep track of what goes where with third party devs.

If not, perhaps COOL_PROGRAM will go by default to the newest version of COOLNESS.DLL. Ok, now Billy will likely contend with tracking and modifying functions that have previously been used in highly specialized ways for security/system critical functionality that Windows does not provide either by accident or by intention. So NOW third party devs developing well organized and functional code/programs are forced to keep up with the madness of Windows development to save space. Hmmm... Guess it got the better of the buffer overflow this time. Or maybe they introduced a new bug into the system {par for course with MS}...

Better yet, how about people developing security/system critical environments use their own code to avoid this whole mess? Ok, now you dont need DLLs do you? How about 3-5 times as many? Wait for the next Windows release? So the effort YOU made during XP to keep up with DLLs and other updates is pointless right? Or XP+1? The style of MS defines itself....

Security/system critical programs?...
Thats only one side?...
Ok. Try this:

Graphics, network comunication, encryption, file editing, database editing? Or maybe drivers, file converters, scripts, inter-app comunication, diagnostics?

The list goes on. The problems generated by and complications arising from this framework are not worth the hassle.

Instead of building a system where things get more complicated I would recomend a redesign of the system itself. Current and past states of instability/insecurity are more than I care to witness again. Billy has enough money to sit around daydreaming for the rest of his days while still paying his programers for doing nothing but daydreaming themselves for the rest of theirs... Perhaps they could get up off their butts and design a system from the ground up that is easy to use, safe, fast, and reliable for users old and new... Logical?

I love C programming. C++ and Java are lots of fun. But IF you want something done right the first time, assembly and careful thought is the only answer...

S-()-u-|-s-!-|)-E

Re:All Programs Should be Self-Contained

[Junta]

You underestimate the effects of that. Sure, a few promiment programs can do that without issue. However, if every single binary on your system did this, the effect would be horrible.

Quick example. ls is 68k by itseIf you add the size of all libraries it links to, it becomes about 1.7 MB on a typical system. I would say ls is pretty consevative in terms of linking, so I'll pretend everything in /bin would be that size. 110 binaries in /bin. Currently du shows about 4.6M in there. This would grow to 187 MB, and that is using the conservative ls as a comparison, some would be significantly larger. In files that would be in /usr/bin and /usr/X11R6/bin, things get worse as they have more complex linking requirements, libraries that are a lot larger and do a lot more. If every miniscule GUI required to be the size of itself plus system plus whatever toolkit they chose to use, drive space would suck.

Not only is drive space not a moot point, but this has implications in terms of consistency and interoporability. If applications all used internal versions of GUI libraries, there would be absolutely nothing enforcing any sort of consistency and complex inter-process communication becomes really difficult due to version mismatches.

Re:Auto-DLL Managment?

[molarmass192]

I really don't think WINE was high on their list of priorities here. I think their idea had several desired outcomes:

1 - force all non-longhorn users to upgrade
2 - force all software vendors to code to the new .NET API, and
3 - integrate SQL*Server into the OS

Also, imagine what a nice kick in the teeth to Java (which I'm sure is a bigger radar blip than WINE) this will be. I think this will backfire on them, lack of full backwards compatibility is *one* of the reasons why XP never took off. This one lacks any backwards compatibility so you can just extrapolate the barriers to adoption.

Re:Auto-DLL Managment?

[e2d2]

They're probably doing it to break WINE as well. DLL problems aren't that bad with Win2k and XP, system DLLs are protected and with XP you can go back to a restore point.

Gimme a break. DLL Hell has been a problem for a long long time so when they actually try and fix it they are now only trying to fix it to break WINE? That's a strech and I'm sure you know it. Yes you can go back to a restore point but that does not solve the problem. Now one app works but the other doesn't because they are both calling the same dll expecting different versions. Restore points are a temporary solution to a legit problem.

If an app installer is so badly written that it messes up your installation then the software can't be much good either.

So using the current system how do you propose that app developers deal with this? Say when I compile against a dll I am expecting version 1 and it works fine. Three years go by and the dll has gone through 2 new versions released with the latest service pack. Someone installs my application and it copies the old dll over the new one. The system is now screwed. My other option would be to link dynamically and since the new version is on the machine my application simply wouldn't work. BUT using the new system both my application and other applications would work fine, using their appropriate dlls. So how is this a bad thing again?

Can anyone propose a better solution? If so I'm all ears.

Re:Auto-DLL Managment?

[bratmobile]

It's a LOT more complicated than that. Microsoft is trying to solve a very real problem that has plagued developers on EVERY platform that supports DLLs, including Linux. Using symlinks is just one approach. And while it does solve some of the problems, it is not a complete solution.

Here's one of the main situations that Microsoft is trying to address: Microsoft ships FOO.DLL with Windows, or as part of some SDK (like DirectX). Company 1 develops an app, ships it, and on the app CD, it has a copy of FOO.DLL. Company 2 does the same thing.

Now. A bug is discovered in FOO.DLL, and it must be fixed. Unfortunately, fixing it one way causes app 1 to crash, and fixing it the other way causes app 2 to crash. And both apps link to the same version.

So what do you do? In the past, you just crossed your fingers and looked the other way, and tried to write code that behaved as best as possible in whatever circumstances you could think of. But inevitably bug fixes cause other bugs (regressions).

So, Microsoft is trying to solve this, by changing DLL binding, in two important ways:

1) DLL binding will use much more than just a name. Microsoft has developed a very powerful & flexible way to do DLL binding. This is what the .Net framework uses, but it is NOT limited to .Net DLLs -- traditional "unmanaged" code can use this, too. I won't go into details here, because it's very well-documented elsewhere, but suffice to say that developers will have a LOT more control over how apps and DLLs bind to DLLs. This is purely a Good Thing, and it IS fully backward-compatible -- it's an opt-in.

2) You'll have total control over redirecting DLLs, on a per-app basis. Some docs here. This means that you can override DLL binding -- if app 1 MUST have version 4.3.5.1.34 of FOO.DLL, and app 2 MUST have version 4.3.5.1.35b, then you can write a simple XML file for each one, that controls exactly which version they get.

Anyone who reads some kind of evil into this is just plain stupid, and has never done any serious development. Any programmer worth their salt knows that DLL binding is ridiculously crude -- and that goes for every modern platform. Microsoft suffers from this more than most, and has therefore decided to do something about it, and has designed a pretty good system. If you have half a brain, you should check it out, and try to keep that knee-jerk reaction under control. (I'm not directing that comment at Grishnakh, but at all of the slobbering idiots who just flame Microsoft whenever they see the name in a headline.)