Zlib Security Flaw Could Cause Widespread Trouble

BlueSharpieOfDoom writes "Whitedust has an interesting article posted about the new zlib buffer overflow. It affects countless software applications, even on Microsoft Windows. Some of the most affected application are those that are able to use the PNG graphic format, as zlib is wildely used in compression of PNG images. Zlib was also in the news in 2002 because of a flaw found in the way it handled memory allocation. The new hole could allow remote attackers to crash the vulnerable program or even the possiblity of executing arbitrary code."

Comments

Modularised code will always have this problem.

[Ckwop]

Why are we still having buffer overflows? There's a compile option in Visual C++ that allows automatic buffer overflow protection. Does GCC have this switch? If so, why not? And why are people not using this? We have enough processing power on a typical PC to spend on these security such as this. Performance is not an excuse.

Looking further, this is an interesting example of the problems with monoculture. The BSD TCP/IP stack was copied for Windows and Mac OSX - this is great, it saves a tonne of time but you also means you inherit the exact same bugs as the BSD stack. This gives you an impression of how difficult it is to design secure operating system. If you borrow code such as this, you have to make sure it's secure. You can't really do that without line by line analysis which is unrealistic. In libraries the problem is especially accute. If you make a mistake in a well used library it could effect hundreds of pieces of software, as we've seen here.

We can't modularise security either, like we can modularise functionality, because you can take two secure components and put them together and get insecurity. Despite the grand claims people make about formal verification, even this isn't enough. The problem with formal verification is that the abstraction of the language you're using to obtain your proof may not adequately represent the way the compiler actually compiles the program. Besides, it's possible to engineer a compiler that deliberately miscompiles itself such that it compiles programs with security flaws in it.

What i'm trying to say is that despite what the zealots say, achieving security in software is impossible. The best we can do migitate the risk the best we can. The lesson to learn from security flaws such as this is that while code-reuse is good for maintainability and productivity, for security it's not great. As always, security is a trade-off and the trade-off here is whether we want to develop easy to maintain software quickly or whether we want to run the risk of these exploits being exploited. Personally, I fall in the code-reuse camp.

Simon.

Re:Modularised code will always have this problem.

[mistersooreams]

There's a compile option in Visual C++ that allows automatic buffer overflow protection

Is there? I haven't seen it. Even if there is (and I'm inclined to trust you), the reason that no one uses it is because it slows programs down so much. The pointer semantics of languages like C and C++ are fundamentally dangerous and the only way you can make them safe (checking every dereference individually) is painfully slow. I think a factor of three or four was the general consensus on /. last time this debate came up.

I guess it's about time for the Slashdot trolls to start calling for the end of C and C++. Strangely, I think I'm starting to agree with them, at least above the kernel level. Is speed really so critical in zlib?

Re:Modularised code will always have this problem.

[Da Fokka]

For some reason your comment is moderated 'troll', probably because you had the filthy guts of uttering the Forbidden Word 'Visual C++'.

However, your question is prefectly valid. Automatic buffer overflow protection only covers the straightforward buffer overflow problems, i.e. array index overflows. In the case of more complex pointer arithmetic, where most of these problems occur, automatic protection is not possible (at least not without losing the option of pointer arithmetic).

Re:Modularised code will always have this problem.

[Da Fokka]

For some reason your comment is moderated 'troll', probably because you had the filthy guts of uttering the Forbidden Word 'Visual C++'.

Actually, 'forbidden term' would be more appropriate. My bad.

Re:Modularised code will always have this problem.

[Ckwop]

See here

On the broad issue on whether we should be using other languages, I think that saying "the programmer should carefully" is a bit misguided. Humans make mistakes and this is something that computers can do very well. Besides, if coding in such languages is slow, we can use a profiler to find the hot-spots and optimise the slow section using a lower level language.

For that reason, I don't really buy the "but it's too slow argument" - I think it's a good trade-off to use a language that doesn't allow buffer-overflows.

Simon.

Re:Modularised code will always have this problem.

[Anonymous Coward]

Speed is and will always be a feature. Let's say you have a program that takes 1 second to open a zip/gzip file and another program that takes 8 seconds to open a zip/gzip file.
In the first case most users will just open the file to check the contents, which is a feature, fast access means you'll use the files in another way. An 8 second access time slows you down, so you'll be less inclined to check the contents.

This goes for everything in computing. Previewing a filter change in your image program before applying it? If it takes 10 secs, why bother..., if it takes 2 secs, sure.

Re:Modularised code will always have this problem.

[grahammm]

Also hardware support would help. Even 25 years ago the ICL 2900 series systems had a native hardware 'pointer' type (they called it a descriptor). This included in it the size of the object pointed to, and the hardware would check that any dereferences were not out of bounds.

Re:Modularised code will always have this problem.

[CaptainFork]

Why are we still having buffer overflows? There's a compile option in Visual C++ that allows automatic buffer overflow protection. Does GCC have this switch? If so, why not?

If so why not? - and if not, why so?

Why why not but not if not? Why not not?

Re:Modularised code will always have this problem.

[bersl2]

There also exist modifications to gcc that perform the same function. A little checking on your part was all that was necessary to not fall into the publicization trap.

However, all such methods introduce a very noticable performance penalty.

Furthermore, there are documented ways of bypassing all such stack protection mechanisms.

Stop bitching. Audit your goddamn code already. Or would you rather all the bugs be found by the bad guys (this one was found by the Gentoo security team)?

Re:Modularised code will always have this problem.

[Da Fokka]

So get over it, "Da Fokka", and all you other insecure OSS fanboys, too.

Actually, I agree with the first part of your post, I use VS.NET 2005 and I love it.

Here is a pointer that will help you understand the first part of my post.

Re:Modularised code will always have this problem.

[Grey Ninja]

Okay, you are just stupid. Not a MS fanboy. I mean, seriously, you are using some of the shittiest desktop apps, and your server is running absolutely ANCIENT software.

Re:Modularised code will always have this problem.

[Anonymous Coward]

In Delphi, you can switch on {$R+}range checking{$R-} around part of the code you want to protect. (While not slowing down the rest of it.) It's still possible to do Dangerous Pointer Tricks that won't be protected, but you really have to want to.

Re:Modularised code will always have this problem.

[Detritus]

I like programming in C, but I recognize that it is completely inappropriate for many of the applications that it is used for. When modern computers are thousands of times faster than those used for the development of C, we can afford to spend some CPU cycles on reliability and security.

Re:Modularised code will always have this problem.

[Anonymous Coward]

Java is safe from such buffer overflows as well. And there are no optional switches to worry about!

And why are people not using this? We have enough processing power on a typical PC to spend on these security such as this. Performance is not an excuse.

Re:Modularised code will always have this problem.

[Stiletto]

Your argument assumes that buffer overflows are a natural and unavoidable aspect of C programming. I can show you plenty of examples of C modules without buffer overflows. Writing a complex system without buffer overflows is only a matter of using these modules together with a carefully constructed interface.

I also doubt your argument that achieving security in software is impossible. People have been doing it for years and years. Unfortuately we are seeing more and more security breaks because the percentage of careless programmers out there has been steadily rising.

You can write 100% bug-free code if you take your time, are careful and methodical, and do thorough unit and system tests. Those with the "Hey, all warnings but no errors--Ship It!" mentality give the software writing skill a bad name.

Re:Modularised code will always have this problem.

[ookaze]

You don't buy the "but it's too slow argument", OK, that's your right, and it is not surprising when your vision of programming is so narrow.
We are talking about a low level LIBRARY here. Which means reentrant, placement independant code, efficiency, API accessible easily to any language and compiler.
I think all of this is still very difficult to do right in other languages than C. It is already very difficult to do in C++.
Anyway, your language without buffer overflows would not use pointer arithmetic, so would create a zlib a lot slower than the one we now, even if you optimise your high level language to the max.
When you see that what takes time are basic lines of your language, you are toast.

Re:Modularised code will always have this problem.

[mindstormpt]

Or would you rather all the bugs be found by the bad guys (this one was found by the Gentoo security team)?

I wonder how long it took them to emerge their minds after finding the bug...

Re:Modularised code will always have this problem.

[menkhaura]

That would be neat indeed; but how could such a hardware pointer be included in the modern, convoluted, ubiquous x86 instruction set? Yeah, not all the world is a x86, but most of it is.

Could some similar feature be supported by the operating system, or even the libc, by keeping track of every malloc, calloc, realloc, and whatever, reserving some memory space to store information about malloc'ed objects and their sizes?

Re:Modularised code will always have this problem.

[Richard W.M. Jones]

Automatic buffer overflow protection only covers the straightforward buffer overflow problems, i.e. array index overflows. In the case of more complex pointer arithmetic, where most of these problems occur, automatic protection is not possible (at least not without losing the option of pointer arithmetic).

Actually, automatic checking is very much possible, and has been for years. For example, Bounds checking gcc (that website is down right now, so try my page on the subject). That was written in 1994, and there are newer systems available now which don't have such a serious performance penalty.

The real solution is to stop writing critical code in C. Other languages provide bounds checking, and are faster and safer than C: for example OCaml which I prefer nowadays.

Rich.

Re:Modularised code will always have this problem.

[sqlrob]

ummm, most C libraries do that (how do you expect it to deallocate if it doesn't know the size?)

The problem is the dereference beyond the range of the block. Either all dereferences have to go through the library (slow) or you need hardware support.

And even hardware support doesn't completely solve the problem. Nothing prevents you from hitting another block that was allocated, just not the one you're looking at.

Re:Modularised code will always have this problem.

[springbox]

Is there? I haven't seen it.

The updated complier is included in the latest version of the Platform SDK or the free command line compiler that Microsoft is giving away. I haven't actually used it, but I would suspect that most people would only activate it while they were in the process of testing.

Re:Modularised code will always have this problem.

[CaymanIslandCarpedie]

publicization trap

What does this mean? I tried to lookup "publicization" in a few dictionaries, but it doesn't seem to be a word. A little checking on your part was all that was necessary to not fall into the using nonsense words trap ;-) Are you a Bush speech writer? ;-)

Re:Modularised code will always have this problem.

[Bodysurf]

You can't really do that without line by line analysis which is unrealistic

Why not?

OpenBSD does this for everything.

And why not for something as crucial as your example of a TCP/IP stack. It is NOT unreasonable for someone/a group of people to go through that line by line.

Re:Modularised code will always have this problem.

[Anonymous Coward]

The lesson to learn from security flaws such as this is that while code-reuse is good for maintainability and productivity, for security it's not great.

So, you want to say that if you will not reuse somebody else's code that was written years ago and went through a lot of bug-squashing since then, but instead will write your own for example BSD TCP/IP stack - it will have fewer bugs and will be more secure? I bet you will have a lot of bug-squashing to go through just to get close to that reused and cleaned up (and as a result - more secure) code.

Re:Modularised code will always have this problem.

[doctormetal]

I also doubt your argument that achieving security in software is impossible. People have been doing it for years and years. Unfortuately we are seeing more and more security breaks because the percentage of careless programmers out there has been steadily rising.

This is a problem of education. When I was at school you learned to program C and assembler. If you made a stupid programming error you would notice it real soon. Now they mostly teach languages like java which hides most of the defensive programming from the programmer. If you don't know about such things, you should not be programming in languages where you need to these things.

You can write 100% bug-free code if you take your time, are careful and methodical, and do thorough unit and system tests. Those with the "Hey, all warnings but no errors--Ship It!" mentality give the software writing skill a bad name.

Consider all compiler warnings as errors is what I always do. Every warning can be a potential runtime error.

Thinking before you code can also help a lot.

Re:Modularised code will always have this problem.

[aws4y]

Why are we still having buffer overflows? There's a compile option in Visual C++ that allows automatic buffer overflow protection. Does GCC have this switch? If so, why not? And why are people not using this? We have enough processing power on a typical PC to spend on these security such as this. Performance is not an excuse.

The problem I have with this statement is that any checks that Visual C++ may have are at best a fig leaf. Buffer Overflow protection is something that has dogged not just programers but hardware manufactures for decades now. If security is of such great consern why not make the assembler do buffer checks?, why not the operating system? why not the processor?, why not create a ram infrasturcture called SDDR in which the RAM itself does not allow anything to be accessed without a secure hash? the answer to all of these questions is that for every solution, event the stupid one at the bottom, the buffer overflow might take on a new form or the security measures themselves may backfire.

Ultimatly the parent is IMHO over reacting, we are always going to have buffer overflows. This is not necissarily a problem so long as people are willing to disclose the vulnerability and work hard to get it patched before an exploit is out in the wild. This is the main argument as to why Microsoft software is insecure because often known vulnerabilites go months without being patched. They are getting better but they are nowhere near the transparancy displayed here. They made a mistake in coding, they are attempting to fix it but until all the vulnerable aplications are patched we need to be on guard for signs of malicious behavior from programs relying on zlib. In other words this is just a part of life in the world of computing.

Re:Modularised code will always have this problem.

[Anonymous Coward]

Actually, x86 already does, but nobody uses these features. When they fixed segmentation with the 386, segments were now accessed through selectors and offsets. The selectors pointed to one of two tables (GDT - global descriptor table or LDT - local descriptor table). Whenever a memory access was made using a selector, the CPU would look up the descriptor corresponding to the selector. It would check whether the current program had necessary access rights and privilege. If not, then a GPF would be thrown. Segments can be marked as read-only, read-write, executable and maybe a few more combos. Although the GDT and LDT each have only room for 8192 entries, that's still probably more than most programs would need. Each segment could correspond to a single object or array of primitive objects. There would be no buffer overflows because the CPU catches attempts to go beyond the limit of a segment. Stack data couldn't be executed inadvertently because the stack segment would properly be marked as non-executable.

There are a few reasons, though, why we don't use this system. One is that loading descriptors is slow because it was never optimized in the CPU with the equivalent of a TLB as for paging. The other is that using segmentation requires 48-bit pointers rather than 32-bit pointers, or it requires loading segmentation registers and doing a dance with those. I suppose using longer pointers was a problem back in the days when memory was scarce, but it's hardly a problem now (check out 64-bit). Intel *could have* made segment descriptor access checks and loading fast, but I guess there wasn't a demand for it once paging was available.

Re:Modularised code will always have this problem.

[Anonymous Coward]

OpenBSD does this for everything

I can't resist:

Apperently they *don't* do this for everything, or you've just shown that code inspection won't trigger everything at all. And so does Visual C++'s rumoured "magical buffer overflow detection". Quite frankly, it's crap. It'll only trigger on *big* hefty overruns, and almost never on an off-by-one. And even when it triggers, all it does it put up the mightily annoying and terribly useless "DAMAGE detected after NORMAL block. Press Cancel to continue, Retry to abort, and abort to debug". And no, it won't even tell you exactly what variables were stored around the so-called "NORMAL" block. Bah.

Re:Modularised code will always have this problem.

[Tyler Durden]

Why have hardware support that simply helps prevent buffer overflows when we can use hardware features that solve it? I believe that can be done with the NX bit in many modern processors. For more information, look in the Wikipedia entry for "buffer overflow". Getting all new machines to run with chips with this feature and operating systems to take advantage of it is the key to stopping the overflows, not new languages to generate low-level code.

The problem I have with the argument, "Sure the software checks in higher-level languages will slow things down significiantly, but computers are so much faster now," is simple. Ever notice how even as memory/video card frame-rates/hard-drive space increases exponentially it seems that the newest applications tend to still max them out to compete? Well the same thing applies to speed. It's tough to explain to your manager that you are going to purposefully use a language that cripples the efficiency of your newest application to anticiplate your own carelessness. (I'm not saying I'm any better than anyone else on this point. I've had my share of careless programming moments myself).

Does anyone know of any disadvantages to the NX bit that I don't know about? (Like significant slow-down worse than software checks or possible overflows that it would miss).

Re:Modularised code will always have this problem.

[timeOday]

Stop bitching. Audit your goddamn code already.

Oh please. When are we going to get past, "I know! Let's just write perfect software all the time!"

It's well past time to start using typesafe languages for most software - at least in theory. Unfortunately I don't see how this will come about in the OSS world in the forseeable future, because no such virtual runtime exists.

The OSS JVMs are closest I guess, but without an OSS CLASSPATH they don't really count. Besides, there's no mainstream support for targeting the JVM from other languages. On the bright side, I see more significant OSS projects being written in script, such as Python, but modules from different scripting languages will never integrate seamlessly without a common runtime. Maybe Perl's Parrot will get us there one day.

I hate to say it but Microsoft has the right idea with CLR, which is language independent. They are positioned to become the first to use widespread managed code for the userspace. (I realize they're falling short of their original plans for building Longhorn on .net, but they seem way ahead of anybody else).

If Microsoft does manage to solve the problem of buffer overruns in practice, the continued presense of these bugs in OSS will look really bad.

Re:Modularised code will always have this problem.

[shish]

Does GCC have this switch?

Not officially, but back when I used gentoo there was -fstack-protector and IIRC one other switch which implemented stack smash protection and bounds checking. Despite it being a lot slower in theory, I never noticed a difference in practice (except when my programs crashed it'd be "Pointer error detected, aborting" rather than "Segmentation fault")

Re:Modularised code will always have this problem.

[Coryoth]

Despite the grand claims people make about formal verification, even this isn't enough. The problem with formal verification is that the abstraction of the language you're using to obtain your proof may not adequately represent the way the compiler actually compiles the program. Besides, it's possible to engineer a compiler that deliberately miscompiles itself such that it compiles programs with security flaws in it.

Does formal specification and verification eliminate all bugs and security issues? No. Does it make a vast and significant improvement over not doing so? Hell yes. There are reasons why these techniques are used when designing and writing safety or security critical software.

You may as well say "And even locking your doors isn't enough, burglars can still break a window and get in, and you don't know exactly how the lock works - maybe it's really easy to pick."

I think most people who live in areas with any significant amount of burgularies will still bother to lock their doors.

Jedidiah.

Re:Modularised code will always have this problem.

[Saint Stephen]

Foam at the mouth and fall over backwards. Is he foaming at the mouth to fall over backwards or falling over backwards to foam at the mouth. Tonight 'Spectrum' examines the whole question of frothing and falling, coughing and calling, screaming and bawling, walling and stalling, galling and mauling, palling and hauling, trawling and squalling and zalling. Zalling? Is there a word zalling? If there is what does it mean...if there isn't what does it mean? Perhaps both. Maybe neither. What do I mean by the word mean? What do I mean by the word word, what do I mean by what do I mean, what do I mean by do, and what do I do by mean? What do I do by do by do and what do I do by wasting your time like this? Goodnight

-- Monty Python

Re:Modularised code will always have this problem.

[bersl2]

I was going for whatever the substantive form of the verb "publicize" is.

So maybe such a form doesn't officially exist. Check Google; I'm not the only one.

Re:Modularised code will always have this problem.

[Waffle Iron]

Pervasive low-level performance-critical libraries like zlib will be the last thing (other than the kernel) that Microsoft ports to managed code. They'll probably continue use a C implementation of zlib and similar libraries for many years to come, if not indefinitely.

Re:Modularised code will always have this problem.

[bill_mcgonigle]

Stop bitching. Audit your goddamn code already. Or would you rather all the bugs be found by the bad guys (this one was found by the Gentoo security team)?

While we're at it we should stop doing automobile crash tests too, and just design goddamn safe cars already.

Re:Modularised code will always have this problem.

[jared42]

This post seems to have caused a buffer overflow in the part of my brain responsible for syntax parsing. I swear I read it three times, decided that it was correct, and then read it another three times just to enjoy the mental gymnastics.

Thank you!

Re:Modularised code will always have this problem.

[Anonymous Coward]

Visual C++ 7.x utilizes a "canary method" of detecting if a buffer overflow has occurred. What that means is that in the prologue of every function a snippet of code is added to take the return address and save it in a separate area of the process space, often called a "cookie". Then, in the epilogue, another snippet of code is added to compare the return code with that "cookie" to determine if it has changed. If so, the program will either terminate immediately or can jump to a function which can be used to clean up the process more gracefully.

This "canary method" isn't absolute, but at the same time incurs very little performance hit. It will only detect the most basic of buffer overflow situations which are also the most common mistakes and account for approximately 90% of wild buffer overflow vulnerabilities. Other software methods can be used to increase the range of detection, but they do incur further loss of performance, and no method could ever be 100% without the aid of hardware.

That said, particularly in this case, a performance hit to provide a measure of security is more than worth it. It's blatantly obvious by the sheer number of buffer overflow vulnerabilities identified in production code that programmers, regardless of how careful they are, will make mistakes. 39 buffer overflows have been identified in various components of Debian just this year alone. Nobody writes perfect code, and it is the arrogance of the people who claim that they do which often leads to these problems.

Re:Modularised code will always have this problem.

[bersl2]

Whereas if there's a previously undetected bug in the runtime, all of a sudden everything's vulnerable. And now you again have to patch every single machine in existence. How is that any different than in this case?

Re:Modularised code will always have this problem.

[Rylz]

Looking further, this is an interesting example of the problems with monoculture. The BSD TCP/IP stack was copied for Windows and Mac OSX - this is great, it saves a tonne of time but you also means you inherit the exact same bugs as the BSD stack.

What you're missing here is the fact that FreeBSD sent out a security advisory with a patch for this problem 4 days ago. I'm sure the other BSDs sent out patches around the same time, but I don't subscribe to their mailing lists. Anyway, my point is that while inheriting code from BSD may give you the same vulnerabilities, it also gives you frequent security advisories with patches that problem only have to change a little bit to work on your software. Basically what I'm saying is that if you use substantial code from BSD, watching the mailing lists and implementing the patches posted there may be the majority of your security department's work.

Re:Modularised code will always have this problem.

[Clovert Agent]

> For example, Bounds checking gcc (that website is down right now

Archive.org has a mirror:

http://web.archive.org/web/20040611220045/http://w ww-ala.doc.ic.ac.uk/~phjk/BoundsChecking.html

Re:Modularised code will always have this problem.

[bersl2]

Um... are you implying that I'm implying that we should stop testing our programs? Because I don't think that's what I'm saying.

Shit gets through initial testing; the recalls and lawsuits over defects in cars attest to that, and so do these frantic patchfests to defects in software. That's why there exist security teams to go through previous code and find the critical bugs that slipped through.

Re:Modularised code will always have this problem.

[Knightmare]

I am wondering if somehow you have slept through slammer, blaster, and all the ones in-between, there is no magic flag in ANY compiler that will protect you from security flaws. Microsoft has always had these little feel good buttons in their products, just like the "safe for scripting" option that you can set when writing active x components... Well it must be safe, the guy that wrote it clicked the box...

The only answer to these problems is developer education, always always always vet input from outside sources, don't trust "client" input.

Another thing that will help is heavier use of either loosly types languages where you don't set your buffer length or languages that have very flexible array and string memory management that happens behind the scene, no more malloc.

And as for your comment on code reuse is not good for security, that is a very narrow and IMHO ignorant view. I understand the concept of defense in depth but I don't think your infinite monkey's approach to software bugs is it. You would rather have hundreds of independant implementations of an algorithm floating around your enterprise which will all have their own unknown (to you) flaws which could be exploited by any teenager with some spare time (summer break) and a book on smashing the stack.

My preference would be a handful of implementations (not everyone is going to use the norm) so that I can better understand the threat posture posed by the released flaw and formulate a sensible remediation plan.

Re:Modularised code will always have this problem.

[kybred]

In libraries the problem is especially accute. If you make a mistake in a well used library it could effect hundreds of pieces of software, as we've seen here.

Of course the flip side of this is, when you fix the library you fix all the apps (that aren't statically linked) as well.

kybred

Re:Modularised code will always have this problem.

[erroneus]

I'm not a programmer. I've done some light coding here and there, but as far as writing anything big or awesome or important, those days are well behind me. (I got scared as I was developing my career path and had these visions of 1000 people stuck in a cubicle farm and I didn't want to be one of those people.) But recalling from what I knew not more than 10 years ago, I recall all sorts of programming fundamentals such as watching out for buffer overflows and things like that. I did a majority of my programming in assembler and C (not C++) and so maybe I'm missing something about the types of controls and protections are available to more modern programmers, but somehow I doubt it.

I've said it before and I'll say it again: If the problems are fixable, then they were preventable.

Stop programming the lazy way and plan for the unthinkable at every turn. Even when I was learning to code using BASIC, I did everything I could to manage user input in order to prevent the user from making my program look like crap. If that meant writing my own input routine instead of using 'INPUT "Enter your name:";a$' then that's what I did. Maybe I shouldn't have given up on being a programmer so early in life and I'd have a better argument to present. But even if I knew nothing of programming at all, to see that something is fixable proves that it's preventable.

Re:Modularised code will always have this problem.

[kasperd]

Is speed really so critical in zlib?

Yes.

It is used for compression and decompression. Those are tasks that typically takes loads of CPU time. If CPU time was never an issue, we could achieve much better compression ratios. Payloads where the bottleneck is the CPU time used for compression are not rare. And slowing down zlib would certainly make that worse.

But an interesting point however is that typically compression is a lot slower than decompression. Most the security problems are however found in the decompression code. There is a good reason for this. For compression any content on your input stream, so it is clearly impossible to make an attack based on invalid input. For decompression there may exist a few possibilities for invalid compressed input (but not many as those would clearly lead to a poor compression ratio). And even worse, people rarely test decompression of invalid input streams. Valid input streams OTOH are decompressed all the time.

This leads to the question could we sacrifice a bit of performance in the decompression code for better security? And could we do so without hurting compression performance?

Re:Modularised code will always have this problem.

[Anonymous Coward]

Say what!?

Re:Modularised code will always have this problem.

[Anonymous Coward]

Stop writing code in C?
This is the common lament of functional programmers
with an attitude, and has been forever.
I'm seriously tired of it. There are plenty of
non-language specific ways to limit programming
error(s) damage.
These and more rigorous and knowledgeable testing methods and testers, are the answer and not
specific languages.

Re:Modularised code will always have this problem.

[Cash Mitchell]

If so why not? - and if not, why so?

Why why not but not if not? Why not not?

Exactly What?

Exactly!

Re:Modularised code will always have this problem.

[amliebsch]

And now for something completely different.

Re:Modularised code will always have this problem.

[ScrewMaster]

Yes, and how many software managers who are concerned primarily with release dates will ever allow their developers the time to think before they code? Sure, there are poor programmers out there: but there also programmers that would do the right thing if the product weren't shipped right out from under them. Poor software security is just as much a matter of economics as it is programming. One could argue that part of why many major open source projects have a better track record on that score is that the economics of software development are less of an issue, if they are an issue at all. That, ultimately, is probably more important to security than the fact that the source is open. Code that is properly designed and well-implemented is going to have fewer flaws that code that is rushed: even if you have "many eyes" poring over your code, if the initial design is flawed it may not be fixable from a security standpoint.

Every programming position I've had (other than when I was contracting) has had this problem, to varying degrees. I mean, how often have you been in a situation where you knew the right way to code something, but also knew that you'd never be given the time to do that way so you had to find a cheesier (but quicker) way to do it instead? That happens a lot in the real world. It doesn't result in the best code, but it does get things to market.

Re:Modularised code will always have this problem.

[Tharkban]

I would use one of the following:

publication

or maybe one of the alternatives.

publizing
publicatizing
publicication
pubishi cation
publishizing
publicizing

yeah, lameness filter, I realize that's an awful long string of letters. How does the filter work anyway? I mean, those are obviously not words and it doesn't like their length.

Re:Modularised code will always have this problem.

[Anonymous Coward]

The main problem with Visual C++ buffer overflow protection is that is pretty much useless.

First, it is designed to protect against stack based buffer overflows, but can do nothing against heap based ones. I don't know if zlib's is a heap or a stack buffer overflow but stack overflows are becoming rare and I suspect the heap in this case.

Second, Microsoft's stack protection, which by the way is largely inspired by Stackguard, can be evaded without much fuzz. The protection relies on placing a so called "canary" value before RET address (address of the instruction to be executed after the function returns) and checking this memory location for modifications to detect eventual overflow attacks. The problem with Microsoft's implementation is that the user has can define a handler function that is called if such an overflow is detected. In some cases it is possible to modify the location to the function ( a variable called user_handler) so that it points to the buffer controlled by the attacker. Thus, overflow, detection, user function being called and attacker's code is executed. Ironic, isn't' it?

Here's a security advisory on this topic dating from 2002;
http://www.linuxsecurity.com/content/view/111652/1 51/

The problem with overflows is that the attacker is able to overwrite memory locations. Overwriting the RET address is the easiest way to go, but by no mean the only one. Any protection can and will be evaded by jumping between pointers and playing with pointer arithmetic. In the case of Heap overflows, I don't think that this kind of protection even exists.

Re:Modularised code will always have this problem.

[NicklessXed]

But if not then why, and if then why not? And why so if it is not, why not if it isn't so, and why isn't it so if it is, but is not not?

Re:Modularised code will always have this problem.

[TheSpoom]

Because.

Re:Modularised code will always have this problem.

[dodobh]

This gives you an impression of how difficult it is to design secure operating system.

Designing secure operating systems is easy. Designing usable secure systems is hard.
Implementing secure systems is even harder.
Implementing usable secure systems is even harder than that.

The right solution to the design problem has always been small pieces of code doing their work correctly, and integrating these via loose coupling so that your system is secure.

Code review helps.

There is no defense against a bad programmer. Your overflow moves up the stack. Witness the large number of XSS and SQL injection vulnerabilities found in applications developed in "safe" languages like PHP and Java.

The best solution to bad programming practices has been to train programmers to write good code. But good programmers are not cheap.

The lesson to learn from security flaws such as this is that while code-reuse is good for maintainability and productivity, for security it's not great.

Reusing good, peer reviewed code is always good for maintainability and security. The important thing is that the code needs to be reviewed for security holes and bugs continuously.

Re:Modularised code will always have this problem.

[Anonymous Coward]

> your language without buffer overflows would not use pointer arithmetic, so would create a zlib a lot slower than the one we now

Pointer arithmetic and language speed are unrelated.

Re:Modularised code will always have this problem.

[drsquare]

The BSD TCP/IP stack was copied for Windows and Mac OSX - this is great, it saves a tonne of time but you also means you inherit the exact same bugs as the BSD stack.

The other side to this is, it means you've only got one place to fix the bug, and it's fixed on all systems that use it. The more TCP/IP stacks you have, the more the developers are spread out, rather than concentrating on making one perfect, secure stack.

Re:Modularised code will always have this problem.

[Dun Malg]

Stop bitching. Audit your goddamn code already.

Oh please. When are we going to get past, "I know! Let's just write perfect software all the time!"

There will always be some subset of people who refuse to accept the impossibility of absolute perfection. I believe their thinking goes like this:

"Anyone can easily write a single line of bug-free code. If you can write twenty of those lines, you can write a bug free function. Write a dozen such bug-free functions and you've got a bug-free class. Write a half dozen or so bug-free classes and you have a bug free library. Using a collection of such bug free libraries you can write a few more bug-free classes held together by some bug-free lines of code and you've got a bug-free application. You're not so stupid that you can't write a single line of bug-free code, are you? There's no excuse for bugs. Just don't make mistakes. It's a choice, really."

(I never had to work for anyone who said the above, but my brother in law, a coder for a large trucking company, had to put up with a "quality consultant" whose entire theory was essentially the above, punctuated with shouts of "attention to detail, people!" in between such lectures. A similar consultant is documented in an email in "The Dilbert Principle". Sadly, it's probably not the same guy.)

Re:Modularised code will always have this problem.

[ringm000]

Anyway, your language without buffer overflows would not use pointer arithmetic, so would create a zlib a lot slower than the one we now, even if you optimise your high level language to the max.

Really? I don't see any reasonable way for a compiler to implement arrays internally without pointer arithmetic, be it C or any high-level language.

As for mandatory array boundary checks, they can really be optimized away whenever they are not needed. Most trivial example: in case the complier has verified that the loop counter is within the boundaries of the array, no check on access is necessary. Provided the optimizer is good, there won't be any significant difference between the optimized code with mandatory checks and the code where all necessary checks are added manually.

It can even be possible to allow real pointers and pointer arithmetic, have certain source-level compatibility with C, and still have the code which has all necessary checks and is not prone to buffer overflows. E.g. check out Cyclone, which was discussed on Slashdot some time ago.

Re:Modularised code will always have this problem.

[perrin]

> The real solution is to stop writing critical code
> in C.

Yeah. Right. It pisses me off that whenever security gets spoken of here, someone comes up with this pathetic magic fix.

Look, for commonly used libraries like zlib, you can't code it in anything but C. That is because only a C library can be called from every other language out there. Code it in, say, Python, and suddenly you lost all but Python programmers. Same goes for almost everything else. Yay for reuse of code!

Point in case - you write "for example OCaml which I prefer nowadays". Keyword 'nowadays'. What happens when you move on to the next big thing that comes along? Can you call libraries written in OCaml from that language? Extremely unlikely. But I bet it supports calling C libraries, because no serious language can avoid that.

Lots can be done to make C code safer. For example using safe versions of all non-safe functions, and integrating with a proper security model in the OS to drop non-needed permissions. But dropping the only language that can share code with everything else out there is just silly.

Re:Modularised code will always have this problem.

[Com2Kid]

• Consider all compiler warnings as errors is what I always do. Every warning can be a potential runtime error.

I finished up a course with an old Unix Wizard this spring (Author of bc, etc), and he made us compile everything with -Wall

Single warning, zero on the assignment.

Re:Modularised code will always have this problem.

[kurtmckee]

this is an interesting example of the problems with monoculture

Yes, but when left to their own devices, many developers never seem bother implementing the full spec completely or properly. For instance, Xanga doesn't output correct RSS feeds, Microsoft doesn't support the CSS specs completely (nor correctly for those parts that are supported), and many Jabber servers offer incomplete pieces of the full protocol.

There are drawbacks to monocultures, yes, but I think that in some cases it is useful to have strong communities built around a single core. For instance, the Gaim library is used by Gaim (Linux and Win32), GaimQPE (QTopia), and AdiumX (OSX). I'm sure there are others, but the point is that, since not all features in the protocols are implemented, it is far better to standardize on a single core and receive the benefits as that core improves.

I think the benefits of ever-increasing protocol/spec support far outweigh the drawbacks of security flaws. Maybe not in all cases, maybe not in the zlib case, but in some cases.

Re:Modularised code will always have this problem.

[Anonymous Coward]

Uh, I take it you don't actually write software anyone uses. O'Caml doesn't even have shared libraries, it's definitely not faster than C, and interfacing it from other languages in a pain in the ass. Nevermind that it isn't even a safe language when you consider its standard library that deserializes arbitrary bytestreams into types without any verification.

Re:Modularised code will always have this problem.

[Anonymous Coward]

Of course, the fact that djb writes secure software using exactly that technique means *you* probably need to rethink your assumptions.

Personally, my goal is always perfection. I know it's difficult and I don't always achieve it, but I gotta wonder about people like you who "aim low".

Unfortunately, giving people lectures doesn't teach them how to do this. In an industry with about zero barrier to entry, most software is going to be crap. Most programmers simply don't know what they are doing. But don't let that cloud your thinking. It *is* possible to write secure software exactly like you describe.

And it's not just watching for buffer overflows, it's also partitioning parts of your programs from one another, setting resource limits, filtering input, all the layers of security.

If you think "writing secure software is impossible", you've already lost, please get out of the industry, or at least don't write software that deals with network data.

Re:Modularised code will always have this problem.

[jadavis]

we can afford to spend some CPU cycles on reliability and security.

How often do we have to pay those cycles? How many cycles can we afford? How many cycles does it cost to solve the buffer-overflow problem?

Without answering these questions with real numbers, your statement is a meaningless generality.

Some things to consider:
(1) zlib is a library used by many, many applications without explicitly being invoked. If you're running a webserver, it may matter a lot to you if each zlib call takes 1/10th of a second.
(2) although average speed of many languages is close to C, you have to consider the details. Some types of in-memory manipulations (the kind that zlib may be doing) might be significantly faster in C.

Re:Modularised code will always have this problem.

[CharlesEGrant]

Your argument assumes that buffer overflows are a natural and unavoidable aspect of C programming. I can show you plenty of examples of C modules without buffer overflows. Writing a complex system without buffer overflows is only a matter of using these modules together with a carefully constructed interface.

Can you walk across a freeway at rush hour without getting hit by a car? Sure, but I wouldn't want to make a practice of it. Admonishing developers to be careful is not engineering. Even careful, skilled developers (and my understanding is that the authors of zlib are careful and skilled) live in the real world which means they have to work around phone calls, meetings, flu, fire alarms, and earthquakes, making lapses in concentration inevitable.

Unfortunately we are seeing more and more security breaks because the percentage of careless programmers out there has been steadily rising.

This is certainly not my experience. Certainly by 1988 and the Morris Internet worm the UNIX and VAX infrastructure was riddled with buffer overflow bugs. Can you provide some examples of major software projects that have not suffered from buffer-overflow issues?

Those with the "Hey, all warnings but no errors--Ship It!" mentality give the software writing skill a bad name.

The warnings remark is a straw-man. Ignoring compiler warnings is foolish, but a program can be 100% syntactically correct and still be full of buffer overflows and other problems.

Re:Modularised code will always have this problem.

[Scott Wood]

Non-executable stack/data does not solve buffer overflows; it just makes them harder. You can still overwrite the return address on the stack to point to a library function that you want to run, with arguments you specify on the stack (perhaps a library function that disables the no-exec bit, or a call to system()?) It's a bit trickier since, depending on the type of overflow, you may have to avoid embedded 0 bytes in the attack string, but it's still possible unless all library and program function addresses have embedded zero bytes (and that doesn't help if the overflow is from memcpy() or some other non-zero-terminated buffer).

And even if the NX bit does stop the buffer overflow from running code on the target, it can still be used to crash the target as a denial-of-service attack.

Re:Modularised code will always have this problem.

[Anonymous Coward]

Why, or why not. There is no if.

Re:Modularised code will always have this problem.

[bill_mcgonigle]

It appeared that your comment implied that the security responsibility rested squarely on the developer to audit his code. That's like saying that the automotive design engineers are solely responsible for the safety of the automobile. We know they try their hardest, but we also know that it's not good enough. So we have crash test teams and security audit teams.

The Gentoo audit team is earning their (metaphorical) money and the system is working as expected.

Open Source allows for the distribution of this effort beyond the initial development team, which is a Good Thing.

It may be that I interpreted your comment in a way other than was intended and that we're on the same page here.

Re:Modularised code will always have this problem.

[hoka]

Yeah, full blown bounds checking on everything. No matter what you try to do it is still a huge performance hit, and you are far better off using something which is just as potent like SSP. The bigger problem is that getting Bounds Checking to work completely on C++ is a bit harder than it sounds, and afaik there is no fully implemented patch available for any version of gcc, though there are many partial and/or attempted implementation. When I looked around for more information on active bounds checking I discovered that a lot of people talk about it, but nobody implements. There are plenty of papers on the issue but I've yet to see working and tested code.

Re:Modularised code will always have this problem.

[Bert64]

Actually, DEC's C Compiler for Digital UNIX (also ported to linux/alpha and i'm sure for VMS too) has had compiletime and runtime bounds checking for many years, and there are patches for gcc (tho i dont think it's in the mainline code yet)
microsoft's compilers were actually one of the last to get bounds checking support, and it doesn't work as well as DEC's did either.

Re:Modularised code will always have this problem.

[Bert64]

GCC doesnt have it by default, you need to patch it in.. i agree it should be integrated into the mainline..
Most people are simply too lazy to add the patch or even to use it if their os includes it..
gentoo has a useflag for bounds checking, but it's not turned on by default..

Re:Modularised code will always have this problem.

[Anonymous Coward]

Are you sure? I just had a look and it looks like the Sun Java VM on my Linux machine uses the zlib library itself (inside libzip.so).

They probably used zlib because Java is too slow to do decent zip decompression. :-)

Re:Modularised code will always have this problem.

[Bert64]

A denial of service is the last thing someone wants if they're trying to break in to your system.. if the service goes down, it's likely that the people responsible for the system will notice and have their attention drawn to it..
If it's a service theyre not using (and shouldnt have been running anyway) then it will just go down, and the attacker wont be able to attack it again.
The more diversity we have, the more likely that an attacker's first attack won't succeed, and if the service dies he cant try brute-forcing the correct addresses.. But this relies on different systems from different vendors, not one dominant os which always comes from the same place compiled in the same way and with the same memory layout..

Re:Modularised code will always have this problem.

[Geoffreyerffoeg]

That is because only a C library can be called from every other language out there.

This is wrong. What you meant to say is that only a library using the C calling convention can be called from every other language out there. Heck, I can write libraries in Visual Basic of all possible languages, and still have them compile against a C program (on Windows, and I'm sure it'll work on other platforms if someone ports vbc). If a language can call C functions, then it is likely (with a little effort) to have functions called by C - or any language that can call "C".

I haven't used Python or OCaml, but if either of those languages can produce C-style .lib's, .dll's, .so's, .obj's, whatever, then they'll work as well from C. I've seen libraries fully usable in C coded in Delphi, because Delphi designed its object file format to interface with C.

What's more, you say that C can work with (at least as a library) every other language out there. So what's the problem with a small C-language interface that just calls the Python function and returns the result?

Re:Modularised code will always have this problem.

[Dun Malg]

Of course, the fact that djb writes secure software using exactly that technique means *you* probably need to rethink your assumptions.

Get a clue, you anonymous turd. DJ Bernstein isn't working under the whip of a corporate master demanding working apps in murderous short timeframes. Furthermore, the existence of one brilliant man who can manage to keep enormous amounts of state in his head and turn out perfect code is not prima facie evidence that all who cannot are somehow lacking.

Personally, my goal is always perfection. I know it's difficult and I don't always achieve it, but I gotta wonder about people like you who "aim low".

Who the fuck said anything about "aiming low"? My point is that errors happen and that you can't just browbeat people into not making mistakes. You say it yourself: "I don't always achieve [perfection]".

Unfortunately, giving people lectures doesn't teach them how to do this.

THAT WAS EXACTLY MY POINT, JACKASS!

In an industry with about zero barrier to entry, most software is going to be crap. Most programmers simply don't know what they are doing. But don't let that cloud your thinking. It *is* possible to write secure software exactly like you describe.

It is possible, but the chaotic way most programming shops are run make it highly unlikely. Overwork, lack of sleep, poor communication, and shifting goals make errors inevitable. One brilliant man sitting down and writing qmail on his own time and at his own pace does not scale to ten overworked regular code grinders trying to crank out two months work in two weeks.

If you think "writing secure software is impossible", you've already lost

If you think that's what I said, go back and read it the fuck again. Your reading comprehension is as bad as your superiority complex.

please get out of the industry, or at least don't write software that deals with network data.

After you, you arrogant prick.

Re:Modularised code will always have this problem.

[Scott Wood]

A denial of service is the last thing someone wants if they're trying to break in to your system..

Of course. However, there's more than just one kind of attacker out there, and making things harder for the break-in sort doesn't mean that buffer overflows aren't a problem for the DoS sort. I wouldn't particularly like my browser to crash every time it loads an image that someone linked from a server whose admin got pissed, and switched it to a malicious PNG. Sure, it's less serious than a break-in, but still enough of a problem that buffer overruns shouldn't be called "solved" by the NX bit.

As for failed break-ins knocking out the server and preventing future break-ins, odds are if they overflow a buffer using the wrong exploit for the particular target, it'd crash anyway, so the NX bit doesn't really improve things much there either, other than causing existing exploits to become the wrong ones. New exploits that are targeted at NX systems will still work on non-NX systems, so in the long run it doesn't do much to increase diversity, either.

That's not to say that NX isn't useful as a short-term band-aid; it just shouldn't be treated as anything more than that.

Re:Modularised code will always have this problem.

[m50d]

For zlib it would be too slow. This is an incredibly low level library. It's used in networking, so it has to run on routers. It's used in png, so anything that displays graphics has to be able to run it. There's really no alternative to C/C++, it needs to be crossplatform, complieable, and run on embedded systems. Maybe OCaml since that's compilable, but even that might have too much overhead, plus not so many coders know it.

Re:Modularised code will always have this problem.

[Anonymous Coward]

... making lapses in concentration inevitable.

jesus. i am glad the people who build bridges, dams, etc... don't have your attitude.

Re:Modularised code will always have this problem.

[m50d]

I haven't used Python or OCaml, but if either of those languages can produce C-style .lib's, .dll's, .so's, .obj's, whatever, then they'll work as well from C.

Python can't, at least without embedding the python interpreter in the dll.

What's more, you say that C can work with (at least as a library) every other language out there. So what's the problem with a small C-language interface that just calls the Python function and returns the result?

Any language supports calling C libraries from that language. You have to, because for many things the only library available is in C, and there are millions of C libraries. Not so many support calling that language libraries from C - why would they? A big part of their attractiveness is their extensive standard library.

Re:Modularised code will always have this problem.

[spongman]

the reason that no one uses it is because it slows programs down so much

I was under the impression that most of XP SP2 was compiled with the /RTC (stack checking) flags enabled.

I don't see any performance difference over previous versions.

Re:Modularised code will always have this problem.

[CharlesEGrant]

jesus. i am glad the people who build bridges, dams, etc... don't have your attitude.

The people who build dams, bridges, etc., don't simply tell the construction crew to be be careful. They know that mistakes will be made, so they create protocols to minimize the likelihood of mistakes, and to maximize the likelihood of finding the mistakes that are made. For example, an in-law who works in avionics tells me that in any cable run that carries multiple cables, the cables are required to have physically incompatible connectors. They could just tell the manufacturing crew to be careful, but they know from long experience that if two cables can be confused, they will be, so they do their best to make it physically impossible to do so.

Personally, I work down a suite of tools to help ensure reliability:

1. Use a language and run-time that enforces bounds checking and managed memory. This is not always practial because of performance issues.

2. If I can't do that, use libraries that hide the complexity of managing memory (smart pointers, smart arrays, and the like). This is not entirely reliable because it depends on the discipline of the programmer.

3. If I can't do that, I use tools like valgrind in conjunctgion with unit testing, so that l can find memory errors after the fact. This can't be completely trusted because it depends on the completeness of the test suite.

4. Code carefully and cross my fingers.

Re:Modularised code will always have this problem.

[Anonymous Coward]

Read the article. The problem is not a buffer overflow, it is a memory corruption due to use of dynamic memory and mismanagement of pointers. Buffer overflow protection does _nothing_ for you here.

My question to the zlib authors is why don't they have a stress program that randomly corrupts bits in a compressed image, then try running it through the decompression module. If you right a simple program like this yourself, you will discover that it is (perhaps now fixed) quite easy to generate compressed files that cause the decompressor to segfault.

This problem belongs to the general class of input validation problems, where a system assumes the input X to f(X) is properly formed. Compressing an image with f^-1(X) generates only properly formed input for f(X), so these problems lurk around forever because no one ever tests the badly formed cases. Compress/decompress/verify is not a test of f(X), it is a test that f(f^-1(X)) == X, which only tests the full input parameters of f^-1(X), leaving edge cases in f(X) that may be exploitable.

Yes, I agree that buffer overflow protection is a great idea and almost trivial to implement in the compiler as well as the rest of your points.

I'm a fan of compile and run-time assertions, nightly tests, input randomization, and unit stress / verification code for validation in pre-release products, and lighter checks in releases. Use 1000 monkeys for randomization!

Re:Modularised code will always have this problem.

[Citizen of Earth]

Is speed really so critical in zlib?

Um, yes. The Deflate algorithm is really quite slow and is normally the bottleneck of operations like TAR+GZIP.

Re:Modularised code will always have this problem.

[Citizen of Earth]

When modern computers are thousands of times faster than those used for the development of C, we can afford to spend some CPU cycles on reliability and security.

Strangely, the problem sets that these programs are executed on are also thousands of times faster. Moore's Law really doesn't result in faster processing; it just means you can run bigger jobs.

Re:Modularised code will always have this problem.

[Anonymous Coward]

Well this "ANCIENT" software just doesn't happen to massive security flaws like we've recently seen with Apache 2.0 and to some extent, the Linux 2.6 kernel.

Re:Modularised code will always have this problem.

[Anonymous Coward]

As said elsewhere, having programmer-level access to pointers and a high-performance language implementation are two separate questions.

In fact, contrary to the popular urban myth that "C is fast", C's essentially unconstrained pointers (which show up as "programmer-accessible pointer arithmetic") inhibit a great many compiler optimizations. This has been a known situation in the compiler-optimization community for over twenty years.

Re:Modularised code will always have this problem.

[dkf]

So we have a choice between "correct" and "fast"? I'd much rather go with both (who wouldn't?) but if I had to pick, I'd go with "correct".

The problem is that correctness is a system property; the code is correct if and only if it has no mistakes anywhere in it, and there's no bound on the number of levels you can make a mistake at (starting from typos that make the compiler crap out, and going through to and beyond the problems you get when systems of libraries are joined together in an unfortunate way - a fault that is actually distressingly common, BTW). Given that correctness is really hard to achieve, lots of people prefer speed. I'm not quite sure why; maybe they subconciously hope that by making the code go by really quickly, they'll be out of harm's way before some random bolt from the blue makes them hit a bug?? (Anyone really thinking that should turn their software license right now; they're not fit to be let loose on a programming environment.)

Looking more specifically at zlib, it's one of those curious things that has to be written in C (alas, still the only language that really has meaningful portability across a truly wide range of systems) and yet the code is complex enough that almost anyone could stare at it for ages and not be sure whether any buffer overflows exist or not. OK, it's doing something complex, but even so the code is not written for maximum clarity; it needs more simplification so that it becomes easier to determine the overall "smell" of the code. (You all know about "code smell", don't you?) Like that it will be easier for more people to audit the code, and one of the clear key strengths of OSS will kick in.

Re:Modularised code will always have this problem.

[scum-e-bag]

Does GCC have this switch?

I believe this is the mudflap option.

Re:Modularised code will always have this problem.

[Anonymous Coward]

DJB's technique is to produce (imho) underfunctional software and deny security issues when they arise. (Witness the recent overrun in qmail's pop3 daemon, because the thing uses certain high bits for rather strange purposes.)

ISC mentioning it:
http://isc.sans.org/diary.php?date=2005-05-31

Wikipedia mentioning it:
http://en.wikipedia.org/wiki/Daniel_Bernstein

And his reply

http://cr.yp.to/qmail/guarantee.html " Nobody gives gigabytes of memory to each qmail-smtpd process, so there is no problem with qmail's assumption that allocated array lengths fit comfortably into 32 bits"

I don't know about you, but that a program's author would make that kind of assumption makes me somewhat uncomfortable.

Anyway, the REAL flaw in this logic is that it's a probability thing - a competent programmer can produce an error free line of code (say) 95% of the time. But when you have ten lines, it's a matter of all ten being error free - (0.95) ^ (10) - which is only 60%. Heck, at 99% chance of being good each line, it's a 90% chance those ten lines are good.

Now what if there's 10,000 lines of code?

Bugs happen.

Re:Modularised code will always have this problem.

[russotto]

The major problem with formal specification and verification is that the process is _at least_ as complex and error prone as the program you're verifying. And unlike the program (which can be tested), there's no real-world validity checking for your verification, so you could be completely off-base and never notice. So you end up going through a grindingly tedious process for no real gain.

Slashdot guilt-by-association subnet blocking sucks. I'm running out of places to post, and I have a cursed STATIC IP.

Re:Modularised code will always have this problem.

[Anonymous Coward]

funny. i think we are in agreement. what you describe is exactly what people who write software need to do. know mistakes will be made and write the protocols, APIs, etc. to ensure when something goes wrong (e.g. buffer overflows) it is not a problem. for some reason, though, it sounded like you think it's unavoidable where your latest reply says you understand the concepts needed to prevent problems like that from happeneing.

Re:Modularised code will always have this problem.

[Anonymous Coward]

Hey insightful fool. Would you mind explaining how the zlib buffer vulnerability being discussed could be exploited? Or even why it is a buffer overflow at all? Here's the patch: ftp://ftp.freebsd.org/pub/FreeBSD/CERT/patches/SA- 05:16/zlib.patch and here's thee source: http://www.zlib.net/ And btw, there's no way in hell to prevent oversights like this. It's not a typical, duh I used gets() type buffer overflow.

Re:Modularised code will always have this problem.

[Grey Ninja]

There's something really wrong when your computer makes Debian look modern.

Re:Modularised code will always have this problem.

[Anonymous Coward]

The input buffer in this case is a let's say a png file. Now please explain how you're going to segregate acceptable vs. unacceptable inputs....

Re:Modularised code will always have this problem.

[Anonymous Coward]

"We have enough processing power on a typical PC to spend on these security such as this."

Sorry, but I have an Athlon XP 1800+ with 640MB PC-133 RAM and my machine has to chug for a while to get the JVM running, and even then it's ridiculously slow.

Re:Modularised code will always have this problem.

[chefren]

The real problem is that for any large piece of software it will take years if not more to test the program thoroghly so you'll just never know. Also, you need to formally verify that all the algorithms you use are correct as well. Then you have to verify and test the APIs between all those beautiful lines of code/functions you have too.

Re:Modularised code will always have this problem.

[TheLink]

"I don't see any performance difference over previous versions"

Of course, that's because your machine is no longer running hundreds of trojans and spyware. Naturally it's just as fast as before or faster ;).

Re:Modularised code will always have this problem.

[julesh]

Actually, x86 already does, but nobody uses these features. When they fixed segmentation with the 386, segments were now accessed through selectors and offsets.

There are a few reasons, though, why we don't use this system. One is that loading descriptors is slow because it was never optimized in the CPU with the equivalent of a TLB as for paging. The other is that using segmentation requires 48-bit pointers rather than 32-bit pointers, or it requires loading segmentation registers and doing a dance with those. I suppose using longer pointers was a problem back in the days when memory was scarce, but it's hardly a problem now (check out 64-bit).

There are good reasons this facility isn't used for this purpose. Primarily, it was never designed for it. The descriptor tables are really supposed to be modified only by the OS -- allowing the application to modify them can cause severe problems. They're also a very scarce resource (you point out that there are only 8K of each type), and allocation constraints could be a problem. You certainly can't allocate every array in a large program like this, so you need to flag those where a buffer overflow might be a problem... at which point, it would probably be easier just to use something like a C++ object that emulates an array and performs bound checking. It would (in most circumstances) be as efficient, wouldn't require OS support, wouldn't be platform dependent, and would be just as easy for the programmer to do.

The problem is that most programmers believe that their buffers are safe, because they've thought about possible overflow scenarios, and none look like they can work. So nothing would end up being written to use the protection.

Re:Modularised code will always have this problem.

[Anonymous Coward]

He's on third!

Re:Modularised code will always have this problem.

[rbarreira]

So we have a choice between "correct" and "fast"? I'd much rather go with both (who wouldn't?) but if I had to pick, I'd go with "correct".

Would you be surprised if I told you that C code can be correct too?

Re:Modularised code will always have this problem.

[Anonymous Coward]

I wouldn't be surprised if you told us a C program can be correct throughout. I would be astonished if it ever actually happened.

The patch, and the E-Week article and quote

[alanw]

Here's the patch to inftrees.c (found on Debian.org):

$ diff -Naur inftrees.c ../zlib-1.2.2.orig/
--- inftrees.c 2005-07-10 13:38:37.000000000 +0100
+++ ../zlib-1.2.2.orig/inftrees.c 2004-09-15 15:30:06.000000000 +0100
@@ -134,7 +134,7 @@
left -= count[len];
if (left < 0) return -1; /* over-subscribed */
}
- if (left > 0 && (type == CODES || max != 1))
+ if (left > 0 && (type == CODES || (codes - count[0] != 1)))
return -1; /* incomplete set */

/* generate offsets into symbol table for each length for sorting */

And here's the E-Week article with the quote

However, Ormandy said, "Zlib is very mature and stable, so development is sporadic, but it's certainly not dead. Mark Adler [a Zlib co-author] responded to my report with a patch and an in-depth investigation and explanation within 24 hours, and I believe he expects to release a new version of Zlib very soon."

Re:The patch, and the E-Week article and quote

[inflex]

I wonder if it'd be possible to create a binary patch for prebuilt binaries ?

Anyone got some suggestions?

Re:The patch, and the E-Week article and quote

[Haeleth]

I wonder if it'd be possible to create a binary patch for prebuilt binaries ?

For specific builds of individual programs, trivial. There are dozens of good, fast, and robust binary patching systems available - xdelta, bsdiff, and jojodiff are three F/OSS options. Of course, bandwidth is cheap enough these days that most people who use binaries can just download the new version in its entirety.

A general-purpose fix that could be applied to any application using a statically-linked zlib would be much harder, possibly even impossible. This is one of the major advantages of dynamic linking - that a security update to the library in question can automatically benefit any application that uses the library.

Re:The patch, and the E-Week article and quote

[inflex]

Absolutely agreed on your points - there is one situation where a small binary patch does become useful, where it's a legacy system which simply cannot be swapped out for the 'latest version', non of us like those situations but they do exist :-(

Thanks for the pointers - amazing, /. being useful after all :-)

Re:The patch, and the E-Week article and quote

[Anonymous Coward]

Am I the only one that read that subject "Sex, Lies and Compressed tapes?"

Re:The patch, and the E-Week article and quote

[shurdeek]

Dude, your patch is reversed. Besides, you don't have to call diff yourself, just download the debian patch and extract it directly by copy&paste.

Yours sincerely,
Peter

Re:The patch, and the E-Week article and quote

[ThisIsFred]

So how about a patched version then, Adler? It's been four days.

Re:The patch, and the E-Week article and quote

[awkScooby]

That is NOT the correct patch! Instead it would unpatch a system. The correct patch is:

--- inftrees.c 2005-07-10 13:38:37.000000000 +0100
+++ ../zlib-1.2.2.orig/inftrees.c 2004-09-15 15:30:06.000000000 +0100
@@ -134,7 +134,7 @@
left -= count[len];
if (left }
- if (left > 0 && (type == CODES || (codes - count[0] != 1)))
+ if (left > 0 && (type == CODES || max != 1))
return -1; /* incomplete set */

/* generate offsets into symbol table for each length for sorting */

which is the same as what's in zlib-1.2.1.2-1.1.src.rpm from RedHat (except RedHat is using 1.2.1 instead of 1.2.2).

Re:The patch, and the E-Week article and quote

[Al Dimond]

For the benefit of all Gentoo users out there, Gentoo has had this patch up since Wednesday (July 6). If you're running zlib 1.2.2-r1, you have the patch.

Re:The patch, and the E-Week article and quote

[seebs]

I have a CPAN build directory containing an old copy of zlib (1.2.2), dated September 2004.

It has this patch already applied.

WTF?

Re:The patch, and the E-Week article and quote

[dradler]

I responded to the problem with a patch in about a day while on vacation in Greece (paying for telephone line use to get a whopping 24 kbps). I just got back from Greece yesterday. I do this for free, yet I plan to get a new version out within about a week for this vulnerability with no extant exploit, of course after adequate testing by our development group which itself will take a day or two. I hope this satisfies our paying customers like yourself.

I would venture to guess that this response time exceeds that of most commercial ventures.

Mark Adler

Re:The patch, and the E-Week article and quote

[alanw]

Sorry - as a couple of others have already
pointed out, I messed up and got the patch reversed.

Re:The patch, and the E-Week article and quote

[Anonymous Coward]

Care to elaborate on the vulnerability? It's not explained well anywhere online, nor it is obvious from looking at the code.

Re:The patch, and the E-Week article and quote

[ThisIsFred]

My apologies, then, because you've met and exceeded any reasonable expectations anyone could have for a free service, which are "none."

However, the lack of information about the severity or the status of the patch confused a lot of us, when if fact, just about every distribution had already included the patch by the 8th.

Thanks for the personal response, though, and sorry for being an asshole.

Important: Use a safe browser

[aussie_a]

Because Firefox renders PNG completely, it is prone to these sort of errors. However there is one browser that won't need a patch issued to be safe from this bug, which is Internet Explorer. While IE can render PNG a little, it hasn't implemented the full technology. By using IE, you ensure that you will be safe from any bugs that arise from new technologies, such as PNG.

So next time someone recommends a browser. Stop and wonder about what technology the latest browser has implemented properly without regard to any security issues, and remember that it will be decades before IE implements the technology (if it ever does) so it will be safe for quite some time, by being a stable browser that rarely changes.


Mods: This is not an attempt at troll, but a parody of the typical "This is why you should switch to Firefox" posts whenever a vulnerability involving IE. It should be painfully obvious, but then again most of you are on crack.

Re:Important: Use a safe browser

[Anonymous Coward]

excellent. so you seal one hole by using a browser with a large number of other holes.

smart. S.M.R.T.!

Re:Important: Use a safe browser

[grahammm]

I doubt that Firefox will need a patch. The only patch which should be needed is to libz, then all applications which use is will benefit from the fix.

Re:Important: Use a safe browser

[Anonymous Coward]

Fedora Core was patched yesterday... before I even know about this issue.

Re:Important: Use a safe browser

[yeremein]

Because Firefox renders PNG completely, it is prone to these sort of errors. However there is one browser that won't need a patch issued to be safe from this bug, which is Internet Explorer. While IE can render PNG a little, it hasn't implemented the full technology. By using IE, you ensure that you will be safe from any bugs that arise from new technologies, such as PNG.

Wow, that's some attempt at spin. Too bad it's completely wrong.

The fact that Microsoft doesn't support the PNG alpha channel, antialiasing, shadow masking, etc. is completely immaterial--IE still has to decompress the PNG, which means it is vulnerable to any zlib bugs.

By the way, the "new technology" known as PNG was approved by the W3C on October 1, 1996. The fact that Microsoft still doesn't properly support it does not mean it's some bleeding edge thing. Microsoft is just stuck in the stone age.

Re:Important: Use a safe browser

[aussie_a]

It was a parody! I went so overboard so that it would be obvious at a glance that I couldn't be serious. I typed it up in Firefox! I even added a disclaimer at the end of the post. I know PNG isn't new, I slipped it in to hopefully make it obvious I was joking.

Re:Important: Use a safe browser

[dancallaghan]

Except the Windows version (surely only an insignificant number of users though :-P), which would be statically linked against libz.

Re:Important: Use a safe browser

[ratnerstar]

Were you born without a sense of humor, or did you have it surgically removed?

Re:Important: Use a safe browser

[antic]

"Wow, that's some attempt at spin. Too bad it's completely wrong."

-1, Completely missed the joke. And also completely missed the disclaimer on the joke noting it as such. Wow.

Re:Important: Use a safe browser

[jonwil]

The question is, why didnt microsoft (who have used zlib in a few places in windows like IE PNG handling and shell zip support) add zlib or the compression algorithim it implements as a dynamic library and make it a core part of the OS (i.e. with a zlib.h in the platform SDK)

Re:Important: Use a safe browser

[noda132]

By using IE, you ensure that you will be safe from any bugs that arise from new technologies, such as PNG.

That's wrong, IE is just as vulnerable with regard to PNGs.

Also, zlib compression isn't only used in PNGs. It is used to transfer a good proportion of web pages, transparently. In fact, this very web page was transmitted to you using zlib, if you're using IE or Firefox or Opera or Safari or Lynx or....

Re:Important: Use a safe browser

[Anonymous Coward]

You'll have to upgrade to the latest trunk version of Firefox (it will be in Deer Park alpha 2) :
https://bugzilla.mozilla.org/show_bug.cgi?id=29944 5

Re:Important: Use a safe browser

[imsabbel]

God. i know people on slashdot are slow to get humor, but come on, he even WROTE IT IN THE POST that its supposed to be an ironic joke...

Re:Important: Use a safe browser

[RealBorg]

Albert Einstein postulated "Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.". You have just proven the latter.

Re:Important: Use a safe browser

[millette]

yeremein post is surely a parody too !

Re:Important: Use a safe browser

[Anonymous Coward]

Did nothing on 98se in IE ..nor Mozilla. So what xp windows component causes the problem with that link? Is it limited to XP alone or is 2kpro affected as well?

For those with IE on XP,2003

[A beautiful mind]

please visit this link:
http://www.hunger.hu/win.html

Warning: the link causes BSOD because of a flaw in the image rendering algorithm of some windows component when it tries to render a huge image, the waiting times out and the kernel decides its better just to panic.

The link causes no problems on linux, as it's implementation is not faulty.

I think there is a patch for this fault on windowsupdate, but i can't be certain as i'm not using windows.

So long with the zealotry of IE's safeness.

Re:For those with IE on XP,2003

[Anonymous Coward]

It crashed in Mozilla Firefox 1.0.4 as well (blue screen of death). The device driver that failed was that of my graphics card.

Re:For those with IE on XP,2003

[A beautiful mind]

Yes! Exactly.

That was mainly the point of my GP (sorry if my sarcasm wasn't apparent enough).

It is fairly plausible to assume that there are some platform specific bugs or design flaws, where a specific browser is not at fault, like in the case of this zlib bug or the case of the demonstrated windows image bug, therefor we cannot say that firefox sucks or ie sucks, just that "hey look, there is a bug in zlib which is used by this and that" or "hey look, there is a bug in this windows component/design which causes the whole system to halt when it encounters a bug in a device driver!".

Btw, Opera 8.x is said to be immune from this windows crashing design flaw because it uses it's own image rendering library (afaik).

Re:For those with IE on XP,2003

[smallstepforman]

Opera 8.01 works fine. But what else would you expect from Scandinavians.

Re:For those with IE on XP,2003

[Soul-Burn666]

Indeed. This crashes using Firefox aswell ^_^

Re:For those with IE on XP,2003

[Anonymous Coward]

Sorry man, I just thought perhaps you had just said Internet Explorer because it's so tied in to the operating system.

Re:For those with IE on XP,2003

[jacksonj04]

No crash here (Fully patched XPSP2 with Firefox 1.0.4)

Re:For those with IE on XP,2003

[Hektor_Troy]

The Swedish Bikini Team!

Re:For those with IE on XP,2003

[imsabbel]

Oh yeah great.
My latest firefox made a BSOD, too.
Who needs uptime? Or unsaved documents? Grrr
(note to self: never click on stupid links because you think "firefox wont crash, because its good open sourse" )

Re:For those with IE on XP,2003

[John Courtland]

"Btw, Opera 8.x is said to be immune from this windows crashing design flaw because it uses it's own image rendering library"

Looks like it is :)

Re:For those with IE on XP,2003

[Anonymous Coward]

holy crap, that crashed my lynx browser and made windows BSOD.

Re:For those with IE on XP,2003

[AlexMax2742]

Opera 8.01 user here, no crash, though zooming in and out on the picture produced....odd...results.

Re:For those with IE on XP,2003

[Kagami001]

For me, in both IE6 and Firefox 1.0.4, my system slowed to an almost uncontrollable crawl while it tried to render the image. It eventually finishes, although any attempt to scroll the image starts it up again. Keyboard/mouse input gets through occasionally during the rendering, so you can hit alt-F4 and wait a minute and it'll stop. I can bring up Task Manager, but it's almost impossible to control while the rendering is going on.

My system is an up-to-date Windows XP SP2 installation, P4 3.4GHz EE with HT turned on, Intel 875PBZ motherboard, 4GB of RAM, a GForce 5950 Ultra on AGP w/driver ver 61.77 and a Voodoo 3 2000 on PCI with default MS drivers. (I only tested in a normal LUA, not an admin account, on this system.)

The results are the same regardless of whether my browser window is on a screen driven by the GForce or the screen driven by the Voodoo; does that mean it's not a video driver issue? What causes this effect?
The process doesn't appear to use much RAM, just CPU. Anyone here with multiple CPUs care to report?

---
While my real system did not suffer a BSOD, a test XP SP2 system inside VMWare viewing the link with IE as admin does blue screen.

Re:Check out Microsoft's misdeeds

[Anonymous Coward]

Hi eepity you spamming fuck!

http://slashdot.org/comments.pl?sid=155309&cid=130 19918

Will posting dupes cause widespread trouble?

[Anonymous Coward]

I just wanted to ask Timothy is posting dupes would cause widespread trouble for the interenet. Seems like a flaw that not even open source can fix.

Already patched

[Anonymous Coward]

Both Debian and Ubuntu released the patch for this problem 2 days ago. I assume the other big names in the Linux world have or will follow suit shortly.

Re:Already patched

[jnelson4765]

Fedora Core 4 has an update to zlib - ran yum as soon as I saw this article :)

Re:Already patched

[i_like_spam]

What took 'em so long?

Gentoo announced the bug July 5th and had the patch a day later.

Re:Already patched

[Anonymous Coward]

Downside with gentoo is you have to wait three days before the emerge --sync finishes ;-P

Re:Already patched

[stutterbug]

But what about the dozens of binaries you run on Debian, Ubuntu, Fedora and all the other OSs recently shown as patched on Secunia. If you're reading this in anything other than a text-only browser and you haven't recompiled from source, you are vulnerable, regardless of the application sitting in /bin/. That's precisely the point of the article.

Re:Already patched

[Anonymous Coward]

Thu Jul 7 13:47:41 2005 >>> sys-libs/zlib-1.2.2-r1
merge time: 40 seconds.

Re:Already patched

[udippel]

004: SECURITY FIX: July 6, 2005

On OpenBSD

Re:Already patched

[Anonymous Coward]

I'm talking about emerge --sync, which takes ages on my hardware, even though it's not that outdated. not to say i don't like gentoo ofcourse

Re:Check out Microsoft's misdeeds

[Anonymous Coward]

Parent is an attempted zlib buffer overflow hack. (The irony!) Don't click it.

Not just IE

[mikael_j]

Not just IE on WinXP, Firefox works equally well with this website if you are looking for a BSOD. (I just had to try it).

/Mikael

Re:Not just IE

[TekBoy]

It is sites like that this that make Firefox's Session Saver extension a bad idea. I just had to click it. :-) Then firefox just had to restore it the next time loaded. :-0

My stupidity squared.

Re:Not just IE

[Anonymous Coward]

I'm using Firefox 1.0.4 on XP w/ SP2 and the other rubbish up to date, and after about 10 sec the system reboots itself w/o warning.

Re:Not just IE

[PReDiToR]

Oh, you too?

I had to edit the Prefs.js file to stop it doing it a third time. I know that Session Saver will throw up an error eventually telling you that one of your pages is screwing things, but I didn't want to wait.

OUR stupidity is squared. I wonder how many gurus are getting tech support calls right now with this?

even on Microsoft Windows

[Turn-X Alphonse]

even on Microsoft Windows

NOT WINDOWS! I was just about to move to it from this Linux thing!

welcome to four days ago: [ GLSA 200507-05 ]

[Anonymous Coward]

http://forums.gentoo.org/viewtopic-t-356659.html

Re:welcome to four days ago: [ GLSA 200507-05 ]

[zoloto]

wait, something on gentoo came before slashdot? don't you have to wait for the information to compile? is slashdot THAT slow? ladies and gentleman, the day slashdot is slower than gentoo I'll eat my hat!

.click. [SUBMIT]

.click. [SUBMIT]

.click. [SUBMIT]

does anyone have any ketchup?

Re:welcome to four days ago: [ GLSA 200507-05 ]

[Anonymous Coward]

silly /. troll. It was gentoo who found this bug in the first place.

Glad I'm running Linux.

[yog]

I'm running RHN alert notification on Fedora Core 3, and my version of zlib has already been updated with a patch for CAN-2005-2096, the zlib overflow bug.

It's interesting to read about these as they occur, but it's a nice feeling that my operating system is so well taken care of. Too bad that all personal computers aren't set up for this kind of timely response. I wonder about those millions of library computers, home PCs, small business computers, and other institutional setups where no one even understands the concept of an update, let alone regularly runs the Windows "security" update program.

Another reason to use Linux!

Re:Glad I'm running Linux.

[A beautiful mind]

Sorry to take a stab at this, but using linux in itself is NOT a guarantee to get rid of luser stupidity and ignorance. The biggest weakness, around 40-50% of all insecurities in a given system is the user, statistically. We can mitigate the other risks, but the user factor is the worst problem.

I am writing this from my debian desktop.

Re:Glad I'm running Linux.

[moonbender]

Are all the apps on your system that have zlib compiled into them statically patched, too? (Although I'm not sure if that's even an issue.)

Re:Glad I'm running Linux.

[Anonymous Coward]

I applied the update RPM over 48 hours ago before seeing this story here. I love Slashdot, but:

Corollary: Don't depend on /. for security announcements.

Re:Glad I'm running Linux.

[The_Wilschon]

Personally, when I was running windows, I didn't allow the patches to be installed automatically, and I don't on Linux either. Maybe I'm paranoid, but I want to know exactly what goes onto my system before it goes on.

Re:Glad I'm running Linux.

[wasabii]

In a modern Linux distro, it's not.

In Debian for instance, there is one zlib package that everything uses, and compiling static is against policy. I assume redhat is similar.

Re:Glad I'm running Linux.

[Vantage13]

In Debian for instance, there is one zlib package that everything uses

Not entirely true. See this list from the debian-security mailing list for a few which are statically linked.

Re:Glad I'm running Linux.

[Anonymous Coward]

I thought this was another reason to use Lynx :)

despite being OSS...

[forgoil]

...there has been no security audit that found this flaw. ...everybody's computers won't be automatically updated just because there is a patch that all total geeks love. ...the source haven't been read by all who uses it, and the flaw wasn't found.

Let's face it, going through code isn't that much fun, and only big projects find stuff because many people are activly changing the code (and thus forced to read parts of it since the documentation isn't there(proper documentation, not a shabby man page and some out of date howto)). Which is probably why we've got new programming languages which do their best to avoid "security flaw by programming language" :)

Re:despite being OSS...

[Anonymous Coward]

..there has been no security audit that found this flaw.

Been at the crack pipe again? "Provided and/or discovered by: Tavis Ormandy, Gentoo Linux Security Audit Team" here.

Re:despite being OSS...

[djmurdoch]

...there has been no security audit that found this flaw. ...everybody's computers won't be automatically updated just because there is a patch that all total geeks love. ...the source haven't been read by all who uses it, and the flaw wasn't found.

In fact, this flaw was found by a security audit of an open source project, not by any of the closed source projects (like Microsoft Office) that make use of it.

Re:despite being OSS...

[Anonymous Coward]

Wasn't found? How can you be writing about the flaw if it wasn't found?

Re:despite being OSS...

[pohl]

In fact, this flaw was found by a security audit of an open source project

Are you sure? How can that be the case if reading source code isn't fun for the auditors?

Re:despite being OSS...

[djmurdoch]

Who said auditing isn't fun?

Re:despite being OSS...

[pohl]

In a grandparent post, forgoil wrote: "Let's face it, going through code isn't that much fun".

A case for packaging systems

[eldacan]

We've seen many posts on slashdot recently explaining why the packaging systems are no longer desirable (if they ever were), that dependencies are a PITA (even with systems à la APT), etc.
But when you have a flaw in a very popular library like this, you'll be happy to know that all 354 programs using this library on your system will be safe once the shared library is upgraded... Windows users must upgrade every software manually, and they often won't be able to know precisely what software may be affected...

Re:A case for packaging systems

[IamTheRealMike]

You're assuming every program dynamically links to libZ. That isn't the case at all: statically linked or private copies of this library are common.

What's more, Windows not having a package manager doesn't mean they're necessarily worse off. Think about it - if the zlib code was in Windows, they could just update the affected DLLs using Windows Update. If it isn't in Windows, then by definition it wouldn't have been packaged anyway.

Re:A case for packaging systems

[eldacan]

Ah you're right about the presence of statically linked programs. I'm not sure about what you say concerning Windows however... we're not talking about official libs in the OS, but of third party libs commonly used in third party programs. In linux distibutions, there's a culture of using shared libraries when possible, and advanced package tools make this quite convenient. AFAIK there's no such thing on Windows, so most of the time programs have to include their own copies of third party libraries. (Sure, I remember having programs that asked for some visual basic 5 dll I had to fetch on internet, but it's a (rather annoying) minority.)

Re:A case for packaging systems

[spitzak]

I think what he meant is that packaging means it is possible for a program on Linux to use a shared library even if it is not part of the base system.

However, first of all zlib is part of the base system on Linux and not on Windows, this is the much bigger difference in this case, so whatever happens here does not prove anything about packaging.

Second, I really have not seen packaging work for the sort of third-party programs that would be vulnerable. If say zlib was not part of Linux, but Debian noticed that it was used by hundreds of their programs, they would package it and put a dependency in all of *their* programs. But a third party, not paying close attention to Debian, or interested in running on multiple Linux versions, would still be statically-linking in zlib.

I'm in favor of app-dirs type distribution. In this case a program that needs zlib might (I hope) include libzlib.so in the appdir (and arrange things so the appdir is first in LD_LIBRARY_PATH). If there is a vulnerability, the zlib could be replaced, or removed/renamed so that the fixed main zlib is visible. Now I admit this is far from perfect but it does seem to me that this makes installable applications without dependencies, but still fixible, at least *possible*.

Re:A case for packaging systems

[Jimmy_B]

Remember the GDI image handling vulnerability awhile back? The one where Microsoft released a tool to scan for programs that kept private copies of the vulnerable DLL, but which tended to miss things? That's a very common situation in the Windows world; there's no dependency management, so every program bundles every dependency. And if a common dependency happens to be exploitable, then there's no easy way to collect all the different copies.

They've mostly taken care of programs overwriting eachothers' copies with broken or back-leveled versions, at least. That's an improvement. But Microsoft's package management (letting installers do whatever they want) is still broken.

Re:A case for packaging systems

[Anonymous Coward]

Seems like it wouldn't be all that hard to find all copies on the system and patch them. Probably be about six lines of Perl.

Re:A case for packaging systems

[julesh]

I think what he meant is that packaging means it is possible for a program on Linux to use a shared library even if it is not part of the base system.

This is equally true on Windows. There are many installation programs that will check for the presence of a non-core DLL in %WINDIR% and install it if it is not present. It doesn't happen enough though, largely because there's a culture of putting DLLs in a private directory, and many open source packages don't have a standardised naming convention for a DLL file on windows (e.g. the makefiles distributed with zlib just produce a "zlib.dll" file, which is next to useless: you wouldn't be able to tell if you were overwriting a different (potentially incompatible) version when you installed a replacement). Also, people still remember windows 3, where if two applications had a DLL with the same filename, the second to load would actually be linked to the DLL from the first to load.

If you link with zlib the right way, easy to fix

[Ed Avis]

And this, my friends, is why 'dependency hell' is a good thing. A flaw is found in zlib - no trouble, just run the normal update program that comes with your distribution, 'yum update' or whatever, the centrally installed zlib library will be updated, and all applications will start using it.

The trouble comes with those software authors that wanted to be clever and to 'cut down on dependencies' and included a whole copy of zlib statically linked into their application. Now you have to replace the whole app to remove the zlib security hole. The dependency on zlib is still there, just better hidden, and in a way that makes upgrading a lot harder.

If Microsoft had any sense, a zlib.dll would be bundled with Windows and then Office (and many other apps) could use it. But they wouldn't want to do that, partly because it would involve admitting that they use such third-party libraries.

Open Source zlib

[canuck57]

It affects countless software applications, even on Microsoft Windows.

I thought Microsoft was proprietary and didn't use open source like zlib? Snicker. I guess Microsoft is being assimilated.

Re:Open Source zlib

[tcdk]

>It affects countless software applications, even on Microsoft Windows.

I thought Microsoft was proprietary and didn't use open source like zlib? Snicker. I guess Microsoft is being assimilated.

Yeah, all windows applications are written by Microsoft....

Re:Open Source zlib

[compm375]

Yeah, all windows applications are written by Microsoft....

I think Microsoft Internet Explorer is...

Re:Open Source zlib

[Isaac-Lew]

Here is the zlib license, taken from http://www.zlib.net/zlib_license.html :

/* zlib.h -- interface of the 'zlib' general purpose compression library version 1.2.2, October 3rd, 2004

Copyright (C) 1995-2004 Jean-loup Gailly and Mark Adler

This software is provided 'as-is', without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this software.

Permission is granted to anyone to use this software for any purpose, including commercial applications, and to alter it and redistribute it freely, subject to the following restrictions:

1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.

Jean-loup Gailly jloup@gzip.org
Mark Adler madler@alumni.caltech.edu */

Re:Open Source zlib

[m50d]

No, they don't use "viral" open source. They're happy to take advantage of bsd-licensed code, like zlib, libpng, or the bsd tcp stack. And why not?

Perspective of non-C Programmers

[putko]

It is really something that this flaw impacts so many applications.

This situation is unnecessary; the problem is that C is not a type-safe language, like ML, CAML, Haskell, Common Lisp, Scheme, Java, etc.

You could write that code in SML/CAML/Common Lisp and likely get it to run as fast or faster than the original (particularly if you did some space/time tradeoffs ala partial evaluation). Integration with the applications in the form of a library would be the tough part.

Here's a provocative bit from Paul Graham (Lisp expert entrepreneur) on buffer overflows.

Re:Perspective of non-C Programmers

[Florian Weimer]

Common Lisp (the language) is not completely safe, it permits unsafe constructs which can even lead to classic buffer overflows. Most implementations omit bounds checks which are not mandated by the standard when optimizing, so these problems can occur in practice.

Re:Perspective of non-C Programmers

[Anonymous Coward]

the problem is that C is not a type-safe language

Please. This is a very boring misconception about types. It's not a type error. It's a pointer arithmetic error. Nothing a type system à la ML, Java, CL, whatever would have corrected.

However, mandatory bound checking on arrays, at runtime, in those languages would have caught the problem.

There exist type systems that can catch these kind of errors, but they are very cumbersome, and not very practical.

Re:Perspective of non-C Programmers

[Anonymous Coward]

BWAHAHAHAHAHAHAHAAHAAHAAHAAHAA! Best troll of the month! And you even mentioned Haskell! EXTRA POINTS!

Re:Perspective of non-C Programmers

[Coryoth]

However, mandatory bound checking on arrays, at runtime, in those languages would have caught the problem.

There exist type systems that can catch these kind of errors, but they are very cumbersome, and not very practical.

Obviously the right thing to do would have been to use Ada or SPARK which have runtime checks, and in the caes of SPARK extended static checking and formal verification. The real question is: How secure do you want to be? For a random desktop appliation such measures might be going a bit far, for core libraries like zlib where such errors have broad and far reaching effects, maybe it's worth going to a little extra trouble. Not that I see that as ever happening of course...

Jedidiah.

Re:Perspective of non-C Programmers

[putko]

the problem is that C is not a type-safe language

Please. This is a very boring misconception about types. It's not a type error. It's a pointer arithmetic error. Nothing a type system à la ML, Java, CL, whatever would have corrected.

No, you are wrong. The "pointer arithmetic error" that you mention is called (by experts) a "runtime type error".

A language like C that allows you to make type errors is "not type safe", which is why I wrote what I wrote.

Furthermore, not all programs that use arrays require dynamic type checking (as you call it "mandatory bound checking") -- if the program is simple enough. Such type systems don't need to be uwieldy. E.g. this compound statement is type safe, uses arrays, and doesn't need a complicated type system:

{char x[2]; ... Foo(x[0],x[1]);}

So I'm afraid you are the one with the "interesting" misconceptions about types.

Re:Perspective of non-C Programmers

[Anonymous Coward]

Ok, let's get into the gory details... :)

No, you are wrong. The "pointer arithmetic error" that you mention is called (by experts) a "runtime type error".

As I said, there *exist* type systems in which these errors (or some of them) are caught. In those systems, and in those sytems *only*, these are type error (runtime or not).

However, in Java, ML, CL, etc., this is not a *type* errors. Nothing in the type system of each of these languages prevent this. By type system I mean: the set of formal rules that define the type of an expression (e.g. in a sequent calculus, for a fonctionnal language).
Of course, since one could imagine a type system in which these errors are caught, one could be tempted to call them "type error" anyway. But the fact of the matter is that the type system of the language, which is something perfectly well defined, does not contain that kind of things.

If not so, I'd like you to point out what in a ML-like type system enforces this. In particular, Caml is statically typed. There is nothing in the type system that concerns runtime. So obviously, a runtime check cannot be called a type verification (if you do so, you're talking about a different, undefined, imaginary type sytem; which is not Caml type system, that you can find in the specification of the language.).

Your last argument is just fricking naive, even with static arrays, it is not possible (ie. undecidable) to check bounds at compile time in general. And very easily so in practice. It's quite easy to consider a type system which enforces some of the bound checks on static arrays at compile time; and even all of them at the price of big restrictions of what computations you are allowed to performs on your indices. But it's just impossible/ impractical to use them.

So yes, it is a type error in some (academic) languages. Sadly those are not usable in practice. And Java, ML, Haskell, CL, etc. do not have such an advanced type system. So this error in zlib would not be caught if zlib was programmed in Java because of Java type system, but because of runtime checks that are independent from its type system.

Re:Perspective of non-C Programmers

[MrBadbar]

I beg your pardon, but think about how all your favorite type-safe languages are implemented (ML, CAML, Haskell, Common Lisp, Scheme, Java, etc). These languages are all interpreted or compiled at some level by C code. Imagine how many programs will be affected when (not if, IMHO) somebody finds a bug in the implementation of one of these languages.

High level languages and tools are great, but you've got to realize that computers still only speak assembly. You can think about things at as high a level as you want, but any program has to be translated to something very low level to actually do anything at all. As long as this is the case, we will always have these kinds of bugs and exploits.

Re:Perspective of non-C Programmers

[po8]

Why does this topic bring the, uh, technically challenged out of the woodwork?

I'm a Ph.D. computer science professor with 20 years of experience in design and implementation of programming languages, and the co-author of a C-like programming language featuring static and dynamic typing and runtime operand checking. The parent poster is confused.

Static type checking involves automatically recognizing type-unsafe operations at compile time. In many programming languages, including C, if you write"s" - 1 the + operation is ill-defined, because the left-hand operand is of the wrong type: i.e., there is no string that is a legal operation to the - operator. The compiler can detect this at compile time and refuse to compile the program.

Dynamic type checking involves automatically recognizing type-unsafe operations at run time. In many programming languages, such as Python, if you write"s" - 1 inside a function definition, the compiler will not detect the problem, because the general problem of detecting this kind of error is unsolvable unless one restricts what programmers can write. Instead, the execution engine can detect the problem at runtime when the - is evaluated and refuse to continue executing the program.

Runtime operand checking involves automatically recognizing at runtime that an operation has an operand that, while of a type that might be legal for the operation being performed, is nonetheless illegal for that operation. In many languages, including Python, if you write 1 / 0 no error will be reported at compile time, because detecting such errors is in general impossible. Instead, the execution engine can detect the problem at runtime, and prevent execution from continuing.

(Of course, there also is such a thing as static operand checking, which bears the same relation to runtime operand checking that static type checking does to runtime type checking. This is a hot research topic right now.)

C's problem is that it (a) does not have a "safe" static type system, (b) does not have any dynamic type checking, and (c) has no operand checking. This combination of misfeatures is incredibly and obviously error-prone; offhand, I can think of no other popular language (not derived from C) that is broken in this fashion. Fixing (a) and/or (b) is not sufficient---(c) also needs to be fixed. Java, for example, has shown that this can be done in a C-like compiled language without noticeable loss of efficiency. (This was shown for PL/I more than 30 years ago, so it's no surprise.)

The parent post gives an example in which the index argument to an array dereference is of the correct type and has correct operands. If x[2] was evaluated, this would be an operand error, since the combination of arguments x and 2 is not a legal operand combination for the array dereference operator. With the statement as given in the parent post, I'm not sure what principle it was trying to illustrate. I think, though, that it doesn't much matter.

Re:Perspective of non-C Programmers

[putko]

By type system I mean: the set of formal rules that define the type of an expression (e.g. in a sequent calculus, for a fonctionnal language).

Your definition is mistaken. CAML and SML have a static (what you call a type system) and dynamic type system. Java similarly has a dynamic type system (for downcasts, arrays, etc.). A dynamic type system ensures that the program is type-safe, because the static type system is not sufficient.

Your remark, "So this error in zlib would not be caught if zlib was programmed in Java because of Java type system, but because of runtime checks that are independent from its type system." again displays your faulty understanding of the dynamic type system of Java.

You claim that Java's dynamic type system is separate from the type system. That is consistent with your faulty definition of "type system", but is nevertheless incorrect. Here's something from the language spec (page 313):

"A basic principle of the design of the Java programming language is that the runtime type system cannot be subverted by code written in the language...

So what you call "runtime checks that are independent from its type system." is what the Java spec (and I) call a "dynamic type system". I would assume the same is true of CAML/SML, but those specs are not available online (and I don't have them handy).

Here's more, from Computer Science professor. Please look here -- this Professor is suitably precise to mention:

There is some dynamic typechecking that occurs in statically typed languages too. For instance, in Java, downcasts are verified at run-time and can raise exceptions. Out-of-bounds array accesses are also checked at run-time in Java and Caml, and are thus dynamically typed. Array accesses are not typed in C or C++, since no check is performed at all. Note that the type of an array (i.e. int, float) is statically checked, but the size is dynamically checked.

Re:Perspective of non-C Programmers

[Anonymous Coward]

No they aren't. Most Common Lisp compilers are written in Common Lisp. GHC is written in Haskell. SML/NJ is written in SML. The Jikes RVM is written in Java.

Re:Perspective of non-C Programmers

[Anonymous Coward]

Here's a neat project that addresses the points you've mentioned: how do you know your programming language implmentation is correct.

That project featured a verified compiler for a high-level language that created assembly code.

No C in there at all!

Re:Perspective of non-C Programmers

[El Cubano]

I'm a Ph.D. computer science professor with 20 years of experience in design and implementation of programming languages...

In light of the below statement, I find the above statement to be dubious at best.

C's problem is that it (a) does not have a "safe" static type system, (b) does not have any dynamic type checking, and (c) has no operand checking. This combination of misfeatures is incredibly and obviously error-prone; offhand, I can think of no other popular language (not derived from C) that is broken in this fashion. Fixing (a) and/or (b) is not sufficient---(c) also needs to be fixed. Java, for example, has shown that this can be done in a C-like compiled language without noticeable loss of efficiency. (This was shown for PL/I more than 30 years ago, so it's no surprise.)

The reason I do not believe that you are a CS prof with 20 years experiences writing languages is becuase a real CS prof with 20 years experience writing languages would not describe as broken the exact features put into the language by the designers to provide the flexibility they needed to write an OS!

C was designed by a group of very experienced computer scientists and programmers for use by other very experienced computer scientists and programmers to develop operating systems and other system-level utilities. Any first year CS student, let alone a professor, should know this. The fact that C was adopted by academia, industry, and everybody and his grandmother as a general purpose application language is not the fault of the designers!

In fact, when you are coding things like process and memory mangement routines and libraries, it is very handy to be able to do arithmetic with and compare to variables that are not "exactly" the same type, if the comparison or operation otherwise makes sense. Hence, things like the boolean FALSE and integer 0 being equal (which Java will complain about) are handy.

Not only that, but the lack of dynamic type checking, operand checking and bounds checking allows the programmer to write low level or system code that gets out of the way of higher level code. Imagine the performance degradation at the kernel if every comparison was dynamically checked for type, operand and bounds.

Re:Perspective of non-C Programmers

[Anonymous Coward]

"Your definition is mistaken."

Well, indeed, I guess that we disagree on the definition of "type system". I don't think it is really interesting to consider each and every consistency check (runtime or compile time) as a type verification. It's far too general.

In this case, syntax becomes a property of the type system, any operand check is a type check. For example, a division by zero can be a type error with your definition, just because the runtime (CPU) checks that (with camlopt).

On the other hand, you could devise a type system (a quite complicated one), in which division by zero would be a type error, with my definition; and even syntax. You need two types for int instead of one:
_ int
_ int0
Define Int ::= int | int0. With the rules (positive int):
_ 0 : int0
_ n+1 : int (where n : Int)
Then, the arithmetic operations are of type :
+ : Int * Int -> Int
- : Int * Int -> Int
* : Int * Int -> Int
/ : Int * int -> Int
Then of course, you could add stronger rules. Like:
* : int0 * Int -> int0
* : Int * int0 -> int0
But you would need even more rules, and some restrictions on the kind of arithmetic expressions you allow, to make it work correctly. Depending on the kind of arithmetic expressions you accept, it may or not need runtime checks. In whole generality, you would need them. I hope you get the idea.
This I call a type system. And I am not the only one, obviously.

Of course, you could argue that implicitely, any null int in Caml has the type int0, and that / has the type Int * int -> Int, since there is a runtime check. But that certainly not the same thing that the explicit system I describe.

Why do I want to have my "own" definition (not my own by the way; look at the manual of Caml)? Very easy: how do you differentiate my type system (and the way it checks division by zero) from the way Caml does it (by letting the CPU check for it)?
And then, why no one says that the std::vector in C++ are dynamically typed? Or why, then, would glib's g_array_* not be dynamically typed?

One last thing: why do Java people use "runtime type check" instead of "runtime operand check"? Because no one has ever used that expression for C++ std::vector, so it look's new. Because "types" are fashionable. But this usage of "type" is completely in contrast with decades of research on programming languages and strongly typed lambda-calculus (ever heard of System F? That's a real type system).

Re:Perspective of non-C Programmers

[po8]

We're pretty badly off-topic here, but what the hey...

C was first designed and implemented in the time period from 1969-1973. It is hardly a critique of its original designers and implementors that we have learned a lot about programming language design and implementation in the succeeding 30+ years, and that many of the constraints of the computing environment have been weakened or removed during that time. Indeed, some of the original designers of C and UNIX spent a lot of time 10+ years ago developing an alternative language and runtime for writing operating system and application code that fixes the problems with C that I described.

"In fact, when you are coding things like process and memory mangement routines and libraries, it is very handy to be able to do arithmetic with and compare to variables that are not "exactly" the same type, if the comparison or operation otherwise makes sense. Hence, things like the boolean FALSE and integer 0 being equal (which Java will complain about) are handy."

If by "handy", parent meant "tempting" but "error-prone" and "potentially insecure", I think there's about 30 years' experience to back up this claim. Things as fundamental or important as my operating system's process or memory management routines are occasionally broken in particularly dangerous ways because their programmer did something that seemed to "make sense" at the time, even though a "safe" programming language wouldn't allow it. Go look at the changelogs of a recent UNIX kernel for plenty of examples.

"The lack of dynamic type checking, operand checking and bounds checking allows the programmer to write low level or system code that gets out of the way of higher level code." I'm sorry, but I don't know what this means.

"Imagine the performance degradation at the kernel if every comparison was dynamically checked for type, operand and bounds." One would prefer that operations be checked statically whenever possible. This is not so much for performance as because failed runtime checks in low-level code are difficult to handle gracefully. That said, as I mentioned in my previous post on this topic, we have known for a long time how to build programming languages so that a combination of static and runtime type and operand checking will provide some correctness guarantees without signficantly impacting execution performance. IMHO, it's way past time to start using that knowledge.

Re:Perspective of non-C Programmers

[flynn_nrg]

All those languages are very nice until you check the requirements. Let's see an example: darcs. Darcs is an SCM written in Haskell, which is pretty cool. So, in order to build darcs, you need a haskell compiler, which is written in haskell! GHC needs a bootstrapped compiler in order to be able to compile itself.

This same problem exists with java (see how the FreeBSD port pulls a linux-jdk to build the native version), modula3 and IIRC mono. As long as this problem exists those languages will never be that popular. It's true that you need a C compiler to build gcc, but cross compiling C is a mastered art, it's been done for years. Take a look at how you can build the whole NetBSD operating system even from Windows/cygwin and for a different target CPU.

Re:Perspective of non-C Programmers

[po8]

We're way off-topic here, but still...

First, thanks much for the cool book ref! It looks very nice.

That said, I think a little more of a depth discussion would make the conventional description of the current PL situation with dynamic checks more clear.

A traditional view of types, due to Strachey and Scott, is that a type is defined by the set of possible values that comprise it. As such, one might reasonably say that the operand type of the right hand operand of the C short int divide operator "/" is the set {-32768..32767}-{0} (given 16-bit shorts, but don't get me started).

In practice, however, and motivated by the above example, programming languages tend to restrict the set of types usable in programming. One usual consequence of this restriction is that static typechecking moves from being undecidable to being tractable. Another is that some valid programs are excluded by the type system.

The statement in the Java programming language reference manual that "the runtime type system cannot be subverted by code written in the language" is not limiting. Indeed, Chapter 4 of the spec makes it clear that it considers the base types of the language to be only those "standard" types that any first-year programming student would expect. Section 15.5 makes it clear that integer division by 0 is not regarded as a type error in Java. Instead, 0 is a legally-typed right operand for the Java integer divide operator---it merely causes a runtime exception to be raised when it is encountered. From the perspective of most normal folks, this is the same as saying "there's a runtime operand check for a right-hand side of 0, and an error is raised (but may be handled) when it occurs."

There are other views than the standard one of what a type is. A quite modern view is to regard "types" as a synonym for "properties" of a computation. This view leads to some interesting programming language ideas, but they're still pretty much in the research stage. Note also that the Strachey-Scott view is in some sense the one rejected by parent: that types are defined by the axioms and inference rules of a type system. It's just that Dana Scott's work on domain theory allows such a definition to be sound.

Hope this helps.

Re:Perspective of non-C Programmers

[putko]

I looked at the CAML manual for how they describe the runtime checks (dynamic type system). I couldn't find "runtime operand check" nor "dynamic ...". I figure the SML has to be have a better spec, but that's not freely available.

It seems you are in the mold of the type theory people who view type systems as primarily syntactic (like Benjamin Pierce). So I guess "dynamic type system" is a bit anathema to you. I just went by the prof's online book and the Java spec.

In any case, my original point was that C is not type-safe, in the sense that you can apply nonsense operands to operators (perhaps obtained via your "pointer arithmetic error" or "array bounds error"). I meant type-safe in the sense of Milner:
"Well-typed programs never go wrong."

C programs go wrong -> C is not type-safe.

Re:Perspective of non-C Programmers

[MillionthMonkey]

This happened to someone else, not to me, so my recollection may be faulty.

This guy had a routine in Java to decompress and grab a very small file out of a very large ZIP file containing thousands of tiny to midsize files. The version of the JVM he was using (1.3 I think, maybe this still happens in 1.5) implemented zip using JNI to delegate to a native ZLIB layer.

This routine worked fine, until we got complaints from a customer (a big pharma company). It turned out that the very first thing that ZLIB did when extracting anything was attempt to memory map the entire zip file. That caused an OutOfMemoryError once the ZIP files got too big to fit on the heap. (This was in a scientific app, so potential datasets within the ZIP could have unlimited size.) I think he ended up changing his protocol to a multiple-zip format as a workaround.

It was rumored that Sun was working on a Java-layer implementation for later JVM versions. In theory (knowing Sun) they could just have moved ZLIB's memory-mapping technique to the Java level. (I'm too lazy to check.)

Re:Perspective of non-C Programmers

[Threni]

> It is really something that this flaw impacts so many applications.

Yeah, like division by zero. If we just all deprived the number zero of the oxygen of publicity it would go away and we wouldn't have this problem.

Re:Perspective of non-C Programmers

[Richard W.M. Jones]

Darcs is an SCM written in Haskell, which is pretty cool. So, in order to build darcs, you need a haskell compiler, which is written in haskell! GHC needs a bootstrapped compiler in order to be able to compile itself.

Or you type apt-get install darcs

Installing a C compiler from source is really no easier than installing a Haskell compiler from source (think: what if you don't have a C compiler to compile gcc with?). For most people, it'll be a non-issue since they'll be using a binary package.

Rich.

Re:Perspective of non-C Programmers

[Anonymous Coward]

In many programming languages, including C, if you write"s" - 1 the + operation is ill-defined, because the left-hand operand is of the wrong type: i.e., there is no string that is a legal operation to the - operator.

Actually, in C that's pointer arithmatic. Since it results in a pointer before the start of an array, I believe it results in undefined behavior.

Re:Perspective of non-C Programmers

[po8]

Oops--you're absolutely right, of course. Boy, I must have not been paying attention when I typed that---how embarassing. s/C/Java/ and thanks for the correction!

A skilled worker chooses his tools carefully

[Goosefood]

Ok guys, its about time we start looking at the tooling used in the field of programming. The widespread use of software is constantly bringing to light the same types of software faults. Hardly a day goes by without someone being negatively affected by a buffer overflow/pointer type of bug in software. Would we accepts this from the engineers who build our bridges and buildings?

The direct cause of this of course is programmer error. But lets be honest, we cannot realisticly expect a programmer to check every single pointer related statement in systems running into the thousands of KLOCs. You can certainly invest a lot of effort and reduce the number of potential bugs, but even then you cannot be sure that you've found them all. We really need to start using tooling (languages, compilers, APIs etc etc) that removes this task from the progammer. I'm not saying that every software project out there is suitable for this, but most userspace programs should be.

I'd far rather have an application that today is a little slower, but secure instead of fast but unsecure. Think about it, today it'll be slow and secure, in a year time it'll be fast and secure once the hardware cycle has caught up with the extra overhead introduced.

Re:A skilled worker chooses his tools carefully

[noda132]

I'd far rather have an application that today is a little slower, but secure instead of fast but unsecure.

Zlib compression and decompression probably takes up a sizeable proportion of all CPU power in the world. Make it take twice as much processing power, and hell, you'd be impacting the environment because of the extra wattage used around the globe :P.

Seriously, though: its entire purpose is to be quick and transparent. If you double the amount of time it takes to compress everything, suddenly zlib becomes half as useful.

Re:Mods on crack!?

[atrocious cowpat]

Mods: This is not an attempt at troll, but a parody of the typical "This is why you should switch to Firefox" posts whenever a vulnerability involving IE. It should be painfully obvious, but then again most of you are on crack.

Slander! I only mod people down when I'm off crack!

fuck zlib

[sl4shd0rk]

We don' neet no steenking zeeleeb. -LZO lib

Re:Golly Gee!

[tripslash]

Troll? Come on, I know it's early, but lighten up!

How 'bout -1 Lame Attempt at Humor using awkward attempt to make fun of the phrase "even on Microsoft Windows."

Note to self: Don't go for the obvious M$ joke early on Sunday morning. (Damn, did it again.)

Note to self/others: Lighten up!

Ah ... that's better.

The answer is Defense in Depth

[n0-0p]

Yes, both Visual C++ and the GCC ProPolice extensions provide stack and heap protection. And in general these techniques have a minimal impact on execution speed. Unfortunately, this does not solve the problem. There are still viable attacks that can be preformed by avoiding the stack canaries or heap allocation headers and overwriting other vulnerable data. The probability of developing a successful exploit is lower, but it's still there.

I don't disagree that building secure applications is hard, but it's certainly not impossible. Modularized code just adds another layer of compilcation and potentially confusion. Most of this can be addressed by documenting the design and interface constraints, and ensuring that they're followed. At that point even most security vulnerabilities are primarily implementation defects. Defects will of course still occur, but the trick is to build systems that fail gracefully.

Developers must to account for defects and expect that every form of security protection will fail given enough time and effort. This is why the concept of "Defense in Depth" is so important. By layering protective measures you provide a level of security such that multiple layers have to fail before a compromise becomes truly serious. Combine that with logging and monitoring, and a successful attack while usually be identified before damage is done.

Take the above vulnerabiliy and assume it exists in an exploitable form in a web app running on Apache with a Postgres backend. If the server had been configured from a "Defense in Depth" perspective it would be running in a chroot jail as a low privilege account. Any database access required would be performed through a set of stored procedures or a middleware component that validates the user session and restricts data access. SELinux or GRSecurity would be used for fine grained user control on all running processes. All executables would also be compiled with stack and buffer protection.

In the above scenario, you see that this single exploit wouldn't get you much. However, most systems are deployed with one layer of security, and that's the problem.

Smile :)

[Anonymous Coward]

MSN Messenger uses PNG for some smilies, I guess it's time to disable them ;) 8-) :)

It's the runtime...

[argent]

The problem with writing code in a typesafe language is, as you have noted, integration with libraries. The main reason that so much code is writen in C is that C needs pretty close to zero runtime support.

Not only do languages like Lisp need a fairly extensive runtime, they need dynamic memory allocation and garbage collection, and when you share garbage-collected objects between languages (potentially between multiple languages each with their own allocation models) you're asking for a whole new kind of hurt.

Find a type-safe language where the generated code is standalone and has an easy mechanism for importing and exporting data, and you'll get some traction. And much as I like lisp-family languages myself, that's not the way to go... nor is anything like Java. Perhaps there's something in the Modula family that will work, though my own experience with Modula 2 has not been encouraging.

Re:It's the runtime...

[Richard W.M. Jones]

Find a type-safe language where the generated code is standalone and has an easy mechanism for importing and exporting data, and you'll get some traction.

That would be OCaml then. It compiles to native executables which require no outside support, and it's relatively easy to integrate with C code.

Rich.

Re:It's the runtime...

[Jimmy_B]

They require LOTS of runtime support, they just bundle it in the executable. OCaml is "easy to integrate with C code" in the sense that an OCaml program can use a C library, NOT the other way around. If every library fired up its own garbage collector and loaded its own language runtime, your average large program would immediately run out of memory.

Re:It's the runtime...

[ZenShadow]

Not to mention, I think OCaml is one of the few languages with a "tie fighter" operator. Everywhere I look it's (this :=:*/+ the other thing)...

I was asked to maintain an OCaml program once. I refused. :-)

--S

Re:It's the runtime...

[Richard W.M. Jones]

It's true that OCaml's runtime has an efficient, lightweight garbage collector. C's runtime has a thing called malloc which is also bundled, and of nearly equivalent complexity - take a look at the source to it some time.

It is also possible to call OCaml code from C. Don't believe me? Have a look at mod_caml - that's Apache (C) calling into OCaml CGI scripts.

My original call was for people to start using sensible languages which have support for bounds checking. You can write daemons and applications in OCaml perfectly well, and they will run as fast as C programs, using similar amounts of memory. The technology to exceed C in many areas has been around since the 1970s - for goodness sake, let's start using it!

Rich.

Re:It's the runtime...

[Anonymous Coward]

Cook me flapjacks. Now!

Only a CVE candidate?

[i_like_spam]

Can someone explain why this zlib buffer overflow is only considered as a candidate for inclusion in the CVE list?

Has it not been completely verified, or is there just a lot of red-tape involved in being accepted by the CVE board?

Re:Only a CVE candidate?

[Anonymous Coward]

nothing gets automatically accepted, in fact it can take years to get accepted, so yes, lots of red tape.

Of course, that'd be...

[Anonymous Coward]

as in Oscar.

Wow.

[Anonymous Coward]

It's a good thing we have modern features like shared libraries to make fixing this easier. Right? RIGHT?

I'll just sit down now.

very complex code

[ep385]

Has anyone read the zlib code? While the author clearly tried to make it readable it's still very complex and it's very hard to see at a glance or even after many glances where potential buffer overflow problems may exist (or even where it might fail to implement the deflate algorithm). C is great language for writing an operating system where all you care about is setting bits in a machine register but this algorithm really taxes its abilties.

For comparison here is the deflate algorithm written in Common Lisp. It all fits neaty into a few pages. This is a far better language comparison example than the oft-cited hello world comparison.

Re:very complex code

[Anonymous Coward]

The code you pasted appears to use a linked list, so obviously it won't be nearly as efficient in a real computer with limited CPU cache, unless you always work on small amounts of data.

Re:very complex code

[Beryllium Sphere(tm)]

It's not a linked list, it's a stream datatype.

Re:very complex code

[noidentity]

Is the Common Lisp version written within the same efficiency constraints as the zlib (C) code?

Re:very complex code

[KidSock]

Why is a stupid plug for lisp a 5?

Re:very complex code

[Anonymous Coward]

Thank you for that. It's interesting to me to see simple, common functions like zlib compression done in languages other than C. (Anyone got a link to inflate implemented in Java or other languages?)

Question: does your Lisp code do fast table lookup for short Huffman codes?

zlib reads 8 bits (I think) at a time from the input stream, then looks up this value in a 256-element table. The table returns two things:
1. actual width of the Huffman code that was detected inside the 8-bit value (0 if no code was detected)
2. corresponding output value for the Huffman code

So, narrow Huffman codes (8 bits or less) can be decoded with just a table lookup, which is O(1). Walking through a binary tree, which is O(log2(N), is done only for Huffman codes longer than 8 bits.

Re:very complex code

[julesh]

That's an answer to a different question ("Could the Common Lisp version possible be written to the same performance constraints as the C one?") not the question originally asked: is it?

The main reason for zlib's apparent complexity is that it is an optimization of the traditional deflate algorithm that uses less memory than most other implementations. I would guess that this Lisp version may run as quickly as zlib (although on a modern processor it's unlikely to make a lot of difference -- both will be IO bound by a substantial degree), but probably it uses a significant amount more memory to do so.

Zlib is designed to be useful for embedded systems.

Is i my imagination...

[MSDos-486]

or does it seem the end of the world will be caused by a buffer overflow?

Re:Is i my imagination...

[Anonymous Coward]

No, the world will actually end Tuesday, January 19, 2038. http://www.2038bug.com/

Re:Is i my imagination...

[MSDos-486]

Which will becaused by a buffer overflow.

Re:Is i my imagination...

[Anonymous Coward]

I dunno, I feel fine...

Re:Is i my imagination...

[shish]

IIRC All the times the internet has gone down due to flooding (nimda, codered, msblast, etc) were buffer overflows -- when we get to a point where society depends upon the internet, a buffer overflow could well take out the world as we know it...

Re:Is i my imagination...

[Geoffreyerffoeg]

If society collapses from overpopulation, then it would be a "buffer overflow" in a sense...

Re:Is i my imagination...

[/dev/trash]

Yes. The sun exploding will cause a huge buffer overflow

+++
I once was a great hacker.

BSD Status

[emidln]

For the undead crowd out there:

OpenBSD is affected, and was patched on the 6th of June
FreeBSD is affected, and was patched on the 6th of June
NetBSD base system is not affected, but a zlib from pkgsrc is, and was patched on the 8th of June

Re:BSD Status

[i_like_spam]

Oops... You meant 'July', not June.

Re:BSD Status

[Anonymous Coward]

And the biggest one? Darwin? Hello?

Re:BSD Status

[emidln]

You're right, I did. I'm still in June. Time is an illusion. My management of it doubly so.

Re:BSD Status

[Some Dumbass...]

For the undead crowd out there:

OpenBSD is affected, and was patched [openbsd.org] on the 6th of June

Oops... You meant 'July', not June.

Yes, apparently the undead have psychic powers, perhaps from eating all those delicious brains.

Re:BSD Status

[anethema]

Ubuntu also seems to have patched..or at least the updater downloaded another zlib a few days ago.

Strange? nobody is commenting that this is good

[Portal1]

I see a lot of people complaining that it was badd that so many programs used zlib. Bit i think it is good.
Why??
Well they found one severe bug in one library and probably fixed 1000 programs within 24 hours from a critical bug.
well that is what i call a good track record.
Using mature and stable libraries is good, and it would have been better if they were even more rigorly tested and reviewed, but maybe that is how they found it.

I would however advice to use on stable components rigorous test and analysis tools that look for anything that can cause memeory errors and can advice on speed optimizations. Not just accepting them as beeing stable.

Greetings

Re:Strange? nobody is commenting that this is good

[jcuervo]

Well they found one severe bug in one library and probably fixed 1000 programs within 24 hours from a critical bug.

That's a double-edged sword. Those 1000 programs were also vulnerable from this same severe bug in this one library.

Re:Strange? nobody is commenting that this is good

[Portal1]

I totaly agree,

But i rather use standard mature code with maybe a flaw, in stead of a propiatary solution with likely a lot more severe errors

programming is difficult

[Anonymous Coward]

I think this only underscores how difficult a programmers job really is, if they have any idea what they are doing.

To use compiled libraries and build upon someone elses experiences or to make your own. To use open code and to compile or use it incorrectly, or to reinvent the wheel and spin your own.

Not one has solved the problem.

I find it frightening when i can't trust data files.

Where'd you get those stats?

[sqlrob]

I'm surprised it's anywhere near that low

Re:Where'd you get those stats?

[A beautiful mind]

Based on the world's 3rd largest data recovery firm's statistics, where they have the record of more than 30'000 recoveries (according to an auditor giving a lecture about data recovery and system failures a year ago).

Re:Where'd you get those stats?

[InvalidError]

It is probably higher.

But the real important part to keep in mind is that whatever this percentage may be, there is no and never will be any sure-fire absolute fix for it. In a technically secure system, human are the weakest link.

Better yet...

[Tyler Durden]

...instead of looking for information on the NX bit from the buffer overflow page in Wikipedia, look up the thing specifically here. Yeah.

Correct input validation is the key

[timbrown]

It's all very good advocating the use of languages that mitigate against heap and stack overflows et al. But as the recent XML RPC vulnerability in PHP applications should reinforce, the bad guys are migrating to higher level language attacks too.

The fundamental problem is that all too often, different components of a system have are implemented with different input validation. For example a web browser component running an web application may accept all text input, whereas the backend database is only expecting a subset of text inputs.

Developers should establish what input the lowest level component and highest level components will require to get the job done and validate input into all components subject to these requirements. Where the lowest and highest level components have different requirements then the developer needs to define some method of encoding values that would otherwise be considered invalid, and ensure that all components enforce this encoding.

Der..

[rve]

because it would involve admitting that they use such third-party libraries.

A company of that size doesn't sneakily use 3rd party software. They pay $$$ for 3rd party software they include, and they would only use 3rd party software if it was patented by a 3rd party, or prohibitively expensive to develop themselves. I'm pretty sure that does not include z-lib.

Re:Der..

[Pius II.]

A company of that size doesn't sneakily use 3rd party software.

I see. That must be why they not only use zlib, but also remove the copyright strings.
(Search for "microsoft")

Re:Der..

[rve]

I stand corrected. The copyright notice is even in some of the binaries. Thanks

So, what's wrong with modularization?

[mi]

If anything, this story is about how good modularization is. Simply updating one shared library (libz.so or zlib.dll) will fix the problem for all of your installed applications. No?

Re:So, what's wrong with modularization?

[elflord]

If anything, this story is about how good modularization is

Yes.

Simply updating one shared library (libz.so or zlib.dll) will fix the problem for all of your installed applications. No?

No. Some applications ship their own zlib and/or statically link to it, circumventing the benefits of modularity.

Re:So, what's wrong with modularization?

[Anonymous Coward]

If anything, this story is about how good modularization is. Simply updating one shared library (libz.so or zlib.dll) will fix the problem for all of your installed applications. No?

Only if "size of patches to download" is a measure of security for you. It's not for me. I don't care if one byte fixes the problem or 10 megs fixes it. My apps are vulnerable, and I have to download a patch.

Re:So, what's wrong with modularization?

[Anonymous Coward]

In this case you have to download 1 patch, not 10,000 patches.

Re:If you link with zlib the right way, easy to fi

[macemoneta]

If the argument were that simple, static linking would never occur.

The flip side of the argument is that installing a broken zlib will break all application that are dynamically linked, but have no effect on those that are statically linked.

Remember too that an upgrade to a dynamically linked function means that proper testing must include all software that uses that function. A statically linked application can be tested as a standalone unit.

The resulting isolation of points of failure and lower MTTR is often seen as an advantage in production environments.

I remember this specific situation occurring in a production environment I worked in. A common library was updated, causing the failure of multiple critical applications. The ones not impacted? Statically linked.

Both sides of the discussion clearly have advantages and disadvantages; they have to be weighed to determine the proper risk/benefit.

Re:If you link with zlib the right way, easy to fi

[twiddlingbits]

But what if you are running it in an embedded system that has been deployed on a Spacecraft?It's NOT easy at all to make that change! Zlib is in a LOT of places where data compression is needed as the algorithm and code are free. We found this zlib bug close to 3 years ago at NASA IV&V duing Code Analysis for the SWIFT mission using an Automated Tool (CodeSurfer). The tool told us we has a possible buffer /memory overflow problem in the zlib code at a certain spot. We had to figure out what was causing it which we non-trivial. TWe reccommended a fix then, and I'm pretty sure it was patched by the developers at least for the SWIFT mission. I'm surprised it has taken this long for the problem to be noticed in the mainstream software community. As others have said, keeping your libs up to date can resolve the problem. But once the code is released it becomes a much bigger task. It's theoretically possible to test software for all know conditions that might be serious enough to crash the system or open doors that should not be open. However, that task is VERY timeconsuming and the software DOES have to get to market! It's a hard line to walk to determine how much testing is enough and where to test deeper and where not to. Profilers and Analysis tools are very helpful but in the end it comes down to a human judgement (mistake) that things are "good enough"/

No...

[Junta]

Not in the least bit, observe, just verified with ldd that Xorg and firefox have libz dynamically linked in on my system, which means on program restart, it will pick up the code from the shared library at runtime. It's the whole point of a dynamicly linked library.

Now once upon a time, a lot of distributions (and open source projects out of the box even) would just static link in libz for some reason or another, but after some security issues in the past that caused massive headaches for package maintainers, that practice has largely ceased.

Re:If you link with zlib the right way, easy to fi

[slavemowgli]

You don't have to test every piece of software that uses a dynamic library. Proper unit tests for the library itself will be enough, as long as they're comprehensive and cover the entire API; the only apps that might still break are those that use the library in ways it's not intended to be, but that's really the app developers' own fault then.

Re:If you link with zlib the right way, easy to fi

[Anonymous Coward]

Hey shit for brains, living off of tax payer money, DID YOU FILE A BUG REPORT WITH THE ZLIB MAINTAINERS?

'm surprised it has taken this long for the problem to be noticed in the mainstream software community

It doesn't sound like you filed a bug report. If you didn't, go fuck yourself.

BO protection

[Anonymous Coward]

Software like entercept has buffer overflow protection. I got this on my machines in my network. I doubt I will even have to worry about this one.

C Still Rules

[Anonymous Coward]

People have been proposing the we code our critical operating systems, libraries and applications in a sandbox for decades now.

Well. Where is the p-system? Where is Javagator? Name me a multi-operating system application written in Visual Basic?

Name me a program not written in "C" or C++ ?
I mean a real one like "Linux" or "Windows Kernel" or "Mozilla" or "Word" or "Excel" or "ssh".

Hell, perl is written in C.

Well. I'm waiting. The fact of the matter is that programmer's need control to make truly
usable applications. The sandbox of Java or Visual Basic or Perl make them too slow. They are for making cute little toys, not systems. If you are making toys, use one of these. If you are making serious system tools, use "C". And check for buffer overflows and memory leaks. That's just the way it is.

Re:C Still Rules

[Goosefood]

I did not claim that *all* software should be written using something else than C. Like my post's title says, its all about selecting the right tool for the right job.

My daily work is on a server system deployed in telco networks. Its all written in C, even things like postprocessors for data analysis. To be honest, only a few or the dozen required processes really need to be implemented in C, the rest are more or less IO bound. For us things like GUIs, data analyzers and some non-latency sensitive protocol converters could easily be implemented in a 'slower' language. You might call them toys, we call them part of our product. Customers even see them as the product.

In fact, fixing bugs on these 'toys' is almost as expensive as fixing them on the server-kernel. If we are able to reduce the number of bugs and LOCS on the non-kernel code, then the business is free to deploy its software development resources in way that generates profits instead of decreasing them .

Re:If you link with zlib the right way, easy to fi

[eldacan]

The flip side of the argument is that installing a broken zlib will break all application that are dynamically linked, but have no effect on those that are statically linked.

That's why some people like to use (say) debian stable in production environments: security fixes are backported to the well-tested version of the lib, making a breakage quite unlikely.

News for nerds, [and Windows users?]

[Swedey]

I patched this already.

only a dos attack

[Anonymous Coward]

"The overflow does not
involve user supplied data, and therefore does not allow the execution of
arbitrary code. However, it could still be used by a remote attacker to
create a denial of service."

Re:If you link with zlib the right way, easy to fi

[NtroP]

[snip]We found this zlib bug close to 3 years ago at NASA IV&V duing Code Analysis for the SWIFT mission[/snip]

If this was found 3 years ago why wasn't it patched? WTF is my money going to? I pay YOU to help forward human knowledge and understanding. I fully expect that if you are going to use my public code and find a flaw in it that you report it and maybe even provide a working patch for it!

If your post isn't total bullshit, I'm pissed! I think there should be repercussions for this totally irresponsible attitude we see coming from NASA. When you say "the developers" I'm assuming you mean the developers of the SWIFT mission software, not the developers of zlib, because otherwise we wouldn't be seeing this article on Slashdot!

If, for some reason this turns into another Sasser or Code Red, I think penalties aught to be sought against these fuckwads. Using OSS has responsibilities associated with it too, you know!

Not complex

[Anonymous Coward]

Zlib is not that complex, sorry.

Look ... the reason people use "C" is that it produces small, fast code. Period. If you're writing a library routine that going to get called billions of times a day, it needs to be fast. If you write it in Lisp, it's too slow.

Here's why you should use C and not Lisp. This is form the "shootout" benchmarking programming languages. C wins across the board pretty much all the time against all other languages. Imagine if your computer was twice as slow as it is now. That'd what'd be like if your word process, operating system, browser, etc. weren't written in C. You'd buy another system if it ran twice as fast. This is why systems and serious applications are written in "C" not lisp or Java or perl or whatever.

Re:Not complex

[Anonymous Coward]

The language shootout is crap. They don't even try to measure throughput, so a lot of the measurements for C are very small number and thus have a lot of error. They should have run the tests until a certain time had been reached (run sorting test for 10 seconds, say) and measured how much work was done. But they didn't because doing this trivial thing was 'too difficult'. They claim to measure Java startup time by using a baseline program that does a println, but this is completely bogus since they use other classes in the benchmarks, the load time of which is not included in that baseline. For Java, they also compare Character.isUpper() which implements locale-based semantics (2 method calls and 2 if condidions) with a C version that uses a freakin lookup table to avoid branch delays. I'm sure the same lack of objective fairness applies to other languages as well.

In short, the shootout data is crap. Basing your conclusions on it is about as intelligent as getting your news from Fox.

Re:Not complex

[Anonymous Coward]

I'm missing programs. Have some? Hmmmmmm?

Re:Not complex

[julesh]

That's a terrible benchmark. They've not even tried to be fair -- I can see no other reason why you'd see results like this:

Language / time / mem usage
(binary-trees)
C gcc 5.95 4,496
C++ g++ 18.71 16,176

(count-words)
C gcc 0.18 260
C++ g++ 1.31 744

I mean, you pay a slight penalty for using c++, sure, but it doesn't make your code 3 times (or even 6 times) slower, and use 4 times as much memory. That's bullshit if ever I saw it.

I mean look at the code for the count-words test. Here's the C++:

OK. I don't get to paste the code in, case /.'s lameness filters are rejecting it. You can look at it yourself if you like.

It's a reasonable implementation. I'd optimise it by not using a streambuf (I don't know whether the 'bumpc()' method will be inlined or not, so I'd rather not chance it), but just read data into a character array and use that. I'd also use unsigned characters and check "c Your comment violated the "postercomment" compression filter. Try less whitespace and/or less repetition. Comment aborted.

Yeah, fuck you too. Anyone'd think this wasn't a site where programmers regularly tried to discuss coding techniques.

...as zlib is wildely used...

[PontifexPrimus]

Wildely used?

RARRRGH! ZLIB SMASH! ZLIB COMPRESS!

I think the author meant "widely". Proofreading can't be that hard, can it?

Re:...as zlib is wildely used...

[TheOtherChimeraTwin]

Wildely used?

Yes, Wildely used. Wilde-ly.

Oscar Wilde (1854-1900), Irish poet and dramatist whose reputation rests on his comic masterpiece The Importance of Using Zlib (1895).

For example:

echo "One should always play fairly when one has the winning cards." | zlib
Cheater

echo "Anyone who lives within their means suffers from a lack of imagination." | zlib
Cheap

echo "A little sincerity is a dangerous thing, and a great deal of it is absolutely fatal." | zlib
Liar

echo "There are only two kinds of people who are really fascinating: people who know absolutely everything, and people who know absolutely nothing." | zlib
Stupid

Re:If you link with zlib the right way, easy to fi

[hikingpete]

You're not actually comparing statically linked libraries with shared libraries. What you're actually doing, is comparing upgrading with not upgrading. The library in the statically linked program did not get upgraded.

Warning: OpenSSH uses zlib

[Mike Gleason]

It's unclear how exploitable this bug is with OpenSSH. Just to be sure, apply the patch, recompile zlib, then relink and reinstall ssh.

If you don't want to go to the trouble of doing that right now, a quick fix would be to disable compression in the sshd_config file ("Compression no") and restart sshd. In fact, zlib has had a string of several security bugs recently, so it may be best to just do this anyway.

Re:Warning: OpenSSH uses zlib

[AaronW]

Fortunately though, at least in my distribution, ldd shows that zlib is dynamically linked. In other words, when zlib is fixed, a ton of applications like ssh will also be fixed.

Out of curiosity I did a check in my /usr/bin and found 351 programs with a dependency on zlib.

Any second thoughts on dynamic linking?

[Chris Snook]

This is a perfect example of why dynamic linking is Usually A Good Thing. People are always going to be using libraries, and, like all code, those libraries invariably have bugs.

Appallingly, so many people still insist on static linking of published binaries, with no more compelling reason for this than a barely measurable performance boost they could make up for ten times over by profiling the app and recompiling with that data. This of course assumes that they're testing the final binary.

I don't mean to start up the static vs. dynamic linking flamewar, because there are plenty of circumstances where static linking is perfectly appropriate, but if you can't make a good argument for it, dynamic linking should be the default.

Now we have countless windows apps out there with overflows in both JPEG and PNG (recall the overflow in JPEG a few months ago). It looks like the old "images are safe" mantra that tech support has been spouting out to users regarding attachment safety is falling apart.

Re:Any second thoughts on dynamic linking?

[argent]

1. Dynamic linking carries its own set of security issues and its own history of exploits.

2. Dynamic linking is pretty much the norm these days anyway. It's definitely the norm on the two big closed-source platforms where this might be an issue: Windows and Macintosh.

3. Where static linking is common, it's also common to have either source or relocatable object so you can rebuild or relink anyway.

But given that dynamic linking really is the norm, I'm not sure I get what you're talking about. Who are the "so many people"?

Re:Any second thoughts on dynamic linking?

[Chris Snook]

I agree that dynamic linking has vulnerabilities, but generally speaking, if someone's got enough priviliges on your box to be exploiting any of those, they own it anyway. It's just a matter of convenience.

The "so many people" are typically the novice developers who do short contract work and then are difficult to find when something goes wrong. They build custom tools that countless businesses use all-day every day, and are either mission-critical or somewhere close to it. Large companies are generally organized well enough that this isn't a problem, since they either do it in-house, go to a reputable contractor, or make very certain that they keep the code themselves. The problem is, most of the economy is small business. They decide "It would be worth $5000 dollars if I had a program that did foo." and then they ask around and find someone willing to write that program for $5000. This someone decides they want to make the whole thing completely self-contained, so they statically link it. There's plenty of share/demo-ware out there that's shipped like this too, and small business uses that like mad. Most of them don't quite get F/OSS yet.

This happens ALL THE TIME. It's generally in software you've never heard of, but it's running on computers all around you. The comfort is, the variety of the programs susceptible to this generally makes writing a generic exploit using code execution difficult, but even denial of service is bad.

Re:Any second thoughts on dynamic linking?

[argent]

The "so many people" are typically the novice developers who do short contract work and then are difficult to find when something goes wrong.

If you haven't kept the code for stuff like this, you're hosed anyway.

There's plenty of share/demo-ware out there that's shipped like this too, and small business uses that like mad.

Switch to the Mac, application bundles really wipe out most of the reasons for shipping statically linked code.

Re:Any second thoughts on dynamic linking?

[Chris Snook]

If you haven't kept the code for stuff like this, you're hosed anyway.

Most small businesses don't have a single employee who knows what a compiler is. There's a whole bunch of "If you don't do this, you're hosed" stuff that small businesses just don't have the economies of scale to do. That's why they hire consultants in the first place, but sometimes they don't really know which one to hire or what to ask of them.

Re:Any second thoughts on dynamic linking?

[Sigma 7]

Appallingly, so many people still insist on static linking of published binaries, with no more compelling reason for this than a barely measurable performance boost they could make up for ten times over by profiling the app and recompiling with that data. This of course assumes that they're testing the final binary.

I'm sure that someone else already mentioned this, but DLLs have their own set of problems.

- Windows 3.11 through Windows 98 have a problem known as DLL hell. Most applications insist on installing their own DLL version, breaking applications that require a newer version.
- Windows XP SP1 had a "critical update" in the autoupdater that fixed a minor bug (KB824141) but caused instability in some applications.
- If you need a DLL file for each individual task (e.g. a DLL for Zlib, DLL for TAR files, DLL for Gzip, DLL for TGZ fgiles, etc), you'll end up with a ton of DLL files in your system directory. You can't remove one without knowing whether or not your system will break.
- Finally, there are some cases where the application relies on an existing DLL to be installed (and doesn't include a copy or otherwise give a list.)

Re:Any second thoughts on dynamic linking?

[Chris Snook]

All these things you've described are implementation problems, not design problems. DLL Hell is a result of developers completely missing the point of dynamic linking, implementing it as though it were just a form of static linking that's slower, less stable, and randomly breaks other applications or system libraries.

Package managers solve all of these problems. My familiarity is with apt and rpm with yum/up2date on Linux, but I understand that as of WinXP, the system installer attempts to enforce a certain degree of discipline on application installers, and things like Installshield help the developer distribute an installer that is not likely to either cause or suffer from DLL Hell.

Granted, I have suffered plenty of pain from dependency problems with apt and rpm (all software sucks), but the package manager has always given me enough information to figure out the cause, and resolving it has generally required little more research than reading a man page. In the Windows (95, 98, occasionally XP) world, I was rarely ever able to do any better than determining (by trial and error) that I needed to install my applications in one precise order to get them to work. Without the aid of registry trackers, even that wouldn't have been possible.

Even on Microsoft Windows???

[OsirisX11]

A vulnerability has been found on a Microsoft product?
This changes my whole world. Now I'm going to have to stop purchasing Microsoft products..

oh wait.. :)

Re:If you link with zlib the right way, easy to fi

[ishepherd]

That's a tad optimistic. Think about it, were that true, there would be no need for any sort of testing other than unit testing. You'd simply test all the units separately (per their APIs) and that would be enough to prove the whole system will work.

APIs always leave something to the interpretation.

Take Java. There are thousands of API methods carefully documented, and thousands of unit tests which Sun can run on any Java implementation to check that it is totally compliant. Does that mean you can run your app on a different version of Java and count on it to 'just work', with no extra testing? You'd be a brave guy if you did.

Zlib loaded with Spyware

[yajacuk]

I ran the AOL Spyware protection twice this week and both times I found spyware in the Zlib library.
Here is a sample of the Scan log.

ASP Version: 1.0.77 Definition Date: 01-05-05 Date: 7/6/2005 5:02:02 PM
Action: Found: c:\Program Files\daimonin\client\zlib.dll
Spyware Name: Diablo Keys

Correct interface design.

[argent]

Input is just one interface... every internal interface in a system also needs to be designed so that it only accepts known safe or encapsulated data, and if there's limits on safe input they need to be something that the upper layer can reasonably check without having to effectively replicate the component it's passing the data to.

Let's say, for example, that this overflow involves a pathological artificially created compressed stream. To test for it, you may have to implement most of the algorithms in zlib. To test for it in an image file, you'd have to implement a decoder for that image format that would at least extract the compressed streams to pass to zlib... as well as implementing zlib.

If the problem in the underlying component is a bug that involves the structure of a complexly encoded data stream, it can't really be described in the input or any internal interfaces. Which means that you're not going to catch it with input validation.

Which means that while resiliant interface design is important, it's not going to be enough to block deep attacks. The only way to do that is to make the components themselves secure and resilient. And depending on input validation can cause problems, just ask anyone with an unusual surname.

Re:Correct interface design.

[timbrown]

Erm, I'll give you +4 elloqence as you've just given a much clearer description of what I meant.

When I referred to all components, I didn't just mean those components to which the user of the application was exposed, but every piece of code that took input in i.e. every function. The encoding I referred to should take place between these components, precisely to avoid scenarios such as unusual surnames.

Re:Correct interface design.

[argent]

OK, when people talk about "input validation" I start worrying about whether they're talking about Perl "taint mode". (shudder)

But the point I was getting at was that there's some components that are doing such complex things with the input that the only way to validate them is to duplicate them. And zlib's one of them.

Re:Correct interface design.

[timbrown]

The points I was trying to make were a) that merely using higher level languages to abstract yourself away from how the machine executes, won't make a blind bit of difference, because the bad guys can do the same and have done, to use SQL injection as an example and b) just because you validate in your program at the initial user input, someone might use your lower level functions without doing so, so every function must (complexity aside) check its input.

Re:Correct interface design.

[argent]

that merely using higher level languages to abstract yourself away from how the machine executes, won't make a blind bit of difference

Sure it will.

Well, first of all, it's not so much "using a higher level language", it's "using a language that guarantees certain behaviour". There's some quite low-level languages that have bounds checking and type checking in the definition of the language, and quite high level ones that don't.

Anyway...

Just because it doesn't prevent all failure modes that may be exploited, it eliminates one possible failure mode that may be exploited. That increases the security and robustness of the system as a whole.

The counterargument that the bad guys will just exploit something else is pretty much dead. It's like a Windows user saying it's OK for IE to have really bad security, because if you fix that they'll just use social engineering instead. Just because there's other avenues of attack doesn't mean that improving one of the barricades isn't a good thing.

So, like, when I said "fixing the interfaces isn't the whole solution" I didn't mean "don't fix the interfaces", I meant "still fix the deeper exploits, if you can".

just because you validate in your program at the initial user input, someone might use your lower level functions without doing so, so every function must (complexity aside) check its input

Yes, that's right, I had no intention of implying otherwise, and I apologise if I led you to make that inference.

Yawn

[ChiralSoftware]

As long as we're using unsafe languages to handle untrusted data, we will keep having these problemss.

Zlib itself contains 8000 lines of code. Not very big, is it? It's been around for years and is widely used, so in theory a lot of people have been able to look at it. And yet, after all these years, they are still finding buffer overflows in these 8000 lines of code.

Zlib was not written by monkeys. It was written by very smart, experienced coders. And yet somehow they are not able to write 8000 lines of code without multiple serious buffer overflows.

As long as code like this is written in C we're going to have these problems.

Saying, "there's a critical buffer overflow in a library written in C" is as newsworthy as saying "when I bang my head against the wall I get a headache."

Re:Yawn

[Anonymous Coward]

That's brilliant. You neglect to mention when they've ironed all those buffer overflows out and have tight, bulletproof code they'll be thankful they didn't write it in anything other than C.

Re:Yawn

[Mornelithe]

So, after it takes them 10 years or more (according to the zlib site, it's been around since 1995) to iron out all the buffer overflows in their small library, they'll be glad they wrote it in C?

That sounds like a good time frame to you?

Re:Yawn

[MacDork]

As long as we're using unsafe languages to handle untrusted data, we will keep having these problemss.

Error: Found array of arrays. Expecting array of objects.

Message processing aborted: Next time, please use type safe English.

;-)

Re:Yawn

[Dahan]

And yet, after all these years, they are still finding buffer overflows in these 8000 lines of code.

You make it sound like zlib was written years ago, and the only changes that have ever been made to it since then have been security fixes. That is incorrect. This particular security bug was not present in version 1.0 of zlib. It wasn't even present in version 1.1. It was introduced in version 1.2.1 of zlib, which has been around for less than two years.

Re:Yawn

[Anonymous Coward]

Yeah, seriously, I don't get the whole C thing. Code is something which you need to get things done, not something that you take pride in how long it took you to get it right. Some crazy people take pride in using a hand file to make tweezers out of a truck axel. I don't get that either.

Re:If you link with zlib the right way, easy to fi

[iabervon]

This is really an argument for versioning dynamic libraries very carefully. The Linux dynamic linker has perfectly good support for avoiding problems like this. Each shared library has a "SONAME" field. Programs linked against the library should be able to use any later version of the library with the same SONAME. If the library changes in some way that breaks desireable behavior, it is supposed to get a new SONAME. The system keeps two sets of symlinks, in addition to the object files: libfoo.so is the latest version and libfoo.so.{version} is the latest version of a compatible series. When you link a program, you use the libfoo.so to find which series is newest. When you run a program, you use libfoo.so.{version}, so you get the newest compatible version.

It sounds like the problem you had was that the common library version number wasn't changed when it should have been. If it had been changed, there would have been no effect on any of the existing programs, static or dynamic. Of course, for a two-line fix to a function which only changes the behavior of the library when dealing with corrupted files, and resulted in a buffer overflow instead of an error, the change won't break anything except attacks, so keeping the same SONAME is right.

Note that you can see this by doing "ldd {program}", which will report how it looks for each library and what it finds. And you can see that zlib claims to be backwards compatible all the way through 1.x, while openssl is only to 0.9.7.

Re:I love when this happens

[Compenguin]

> Nothing more fun than firing off a couple apt-get upgrades in the morning while watching your bsd/gentoo friends sit around rebuilding

What!!! The gentoo/bsd crowd wouldn't staticly link zlib. They would only need to rebuild one package. You sir, are full of crap.

very complex code

[ep385]

Absolutely.

Most Common Lisp compilers compile to machine code just like C compilers do. The overhead imposed by the runtime type dispatch for operations is very small (and can even go to zero in many cases). Any difference in deflate speed is likely to be insignificant part of the overall speed any application using zlib.

Woah

[FryingLizard]

Hey you guys,
Just wanted to point out that, amidst the usual "They should've written it in perl" type nonsense, can I just point out that

** This is a _really_ serious bug **

Zlib is very easy to use, effective, and standard, so has been VERY WIDELY incorporated into apps for the last half-dozen years. Those apps are all the most popular *INTERNET FACING* apps (b/c most everyone adds a dash of zlib on incoming or outgoing internet data), and they almost all trust zlib to not blow up on incoming data.

Dudes, not only PNG, but *Macromedia Flash* does and has always used zlib. Anyone fancy checking that?

Perhaps it might be an idea to write a firewall filter that inspects data to detect poisoned zip streams? I believe it can be done reasonably efficiently b/c zip streams are not that expensive to identify.

Gentoo has it's place, too

[excyl]

Luckily there were some Gentooists recompiling their systems packages every so often, since the vulnerability was found by the Gentoo Linux Security Audit Team here. I've always thought the the more projects making use of the same library, as well as, the platforms a project is ported to eventually improves the quality and stability of the code. In this case the security was improved, but in general the abstracted parts of the code improve.

You know..

[FryingLizard]

you could write the most incredibly scary 'sploit with this hole;
Naturally I don't know what I'm talking about (in terms of sploitability of the zlib hole, but it looks bad from a cursory scan) but..

Let's say that there are a buttload of vulnerable programs (let's suppose; multiple browser/IM client PNG implementations, Flash Player(!), email browsers, etcetera) and they mostly link in (statically or dynamically) one of say 4 different builds of zlib. I am making assumptions here which someone will be happy to correct me on, but as this code is right in the core of the decompressor, the local stack frame (regardless of which vulnerable app is running) will likely be usually very similar, making a near-universal buffer overflow sploit at least more practicable.

Now this may be conjecture, but it seems possible that sufficiently misguided individual(s) could work hard and produce an outrageouly virulent nasty that could infect a multitude of different apps. Dunno how hard it would be to get the payload working though.

Certainly busting into a web server will get much easier for a lot of people - there are a lot of server apps that accept (or look inside) zip files, not to mention _virus scanners_. Oh the irony - Your virus checker gets you 0wned. ;-)

Disclaimer: Actually implementing any or all of the above sounds like, at best, a waste of one's precious lifetime. Go hang out with some friends instead.

Re:You know..

[Anonymous Coward]

you certainly don't. look at the code. it's not a user-fed buffer that gets overrun. it's a premature return -1. so that badly crafted png will be oh no, not displayed. it's a dos vulnerability but just exits from the zlib decompression routine so no harm done.

Salza! (was Re:Perspective of non-C Programmers)

[drew crampsie]

zlib written in CL, sans buffer overflows: http://www.cliki.net/Salza

Re:If you link with zlib the right way, easy to fi

[Etyenne]

And this, my friends, is why 'dependency hell' is a good thing. A flaw is found in zlib - no trouble, just run the normal update program that comes with your distribution, 'yum update' or whatever, the centrally installed zlib library will be updated, and all applications will start using it.

Er... no. Long running software (ie daemon) will continue to use the old version, which would reside in memory (in their code segment, I guess). We got beaten by this a little while back, when there was a worm exploiting Apache thru a bug in OpenSSL. The guy responsible for applying the security updates did in fact update the lib, but did not restart Apache. Result ? One of our machine got owned, even though the bug was patched.

Re:I love when this happens

[eklitzke]

Why do you think you only had to upgrade a single package to get this fixed? It's because zlib is dynamically linked into the other programs. You would only need to rebuild all of your packages if you had zlib statically linked into your programs, which isn't done by anyone (even the bsd/gentoo folks) who isn't insane. Right now i'm using Gentoo, and all I had to do was update zlib which didn't take more than a couple minutes.

Re:If you link with zlib the right way, easy to fi

[menscher]

A flaw is found in zlib - no trouble, just run the normal update program that comes with your distribution, 'yum update' or whatever, the centrally installed zlib library will be updated, and all applications will start using it.

Ah, but when? Do they start using it immediately, or do running programs cache the old library, and therefore need a restart? And, if you don't know which programs are running it, then you have to reboot, right?

I agree that it's useful to patch one library rather than several programs, but it doesn't necessarily mean that you're safe just from updating the library. You also need to restart all the programs that use it.

Modularized code is the solution, not the problem

[LibrePensador]

Given that software is always bound to have flaws, modularized code allows you to fix the problem once and have it fixed in all applications that use that library, which is what I just did in my Debian system.

To pretend that replicating the same fuctionality across thousands of incompatible libraries in different operating systems is the solution is as close to blindness as you can get.

You say that modular code helps in maintainability. Yeah, and a big part of the maintainability is making the code maintainable from a security point of view. In summary, I want to fix things once and have them fixed across the board in all applications that use a library.

Of course, you appear to be coming from the world of proprietary operating systems, where the concept of a distribution of software that works well together and is maintained as a whole of does not exist.

The fact that on Windows, you may have to get the same fix from Microsoft, Adobe, Corel and a thousand other vendors only highlights the fact that open source code is both more reusable and more maintainable and thus ultimately more secure because the security process can be optimizied through modularity: Fix once, fix everywhere!

Re:Mods on crack!?

[dodongo]

The reason I don't have mod points this week is because my pipe is broken, in fact.

Face it.

[Anonymous Coward]

Java is slow. If you don't like the shootout, make your own bechmarks. Go ahead, rig it up so that Java is calling assembly language routines so that you can say that Java is faster than C. Go ahead. Have fun.

The rhetorical questions remains: where's Javagator?

Re:I love when this happens

[Anonymous Coward]

(Nicely appropriate username, by the way)

For some reason, half the time when I've used apt-get, it breaks. Updates stop happening, and the installation of new packages doesn't work. Considering that this happens WITHOUT changing configuration files or using any funky tags, then I would have to say that it's a flawed system.

Portage only breaks if I do it myself. And since I'm not an idiot, it's never happened to me :p

Re:I love when this happens

[ciroknight]

Um, I'm sorry but plenty of progams that the Gentoo/BSD crowd use statically link to zlib, and thus, must be patched and rebuilt, thus meaning that the grandparent is correct.

While setting up Gentoo for the first time myself, I remarked about this, and of course, my friends told me it would be "no big problem". Sadly, the 500MHz Celeron system I installed it on will now require to be shut down for nearly a week while it goes through everything effected and rebuilds it all. Of course, ccache and distcc will help speed this up, but neither is the perfect solution (distcc requires the computers be up and running and on the network during the entire compile process, ccache can only ccache objects that don't change and can take up a ton of space with cached objects).

Meanwhile all of my file servers will be perfectly alright with an apt-get upgrade and a manditory restart (system policy; I'd much rather insure the newer libraries are loaded into memory than risk it).

dumb question - why stack so close to data

[Anonymous Coward]

I agree.

Can someone explan why the stack is accessible so close to the data? Seems like there should be a way to prevent writing application data onto the stack where it can collide with instructions in the first place. And, it seems that this is a compiler implmentation issue, and it should not matter what language you are using. And, even if the language standard requires ambiguity, with a switch you could make it "standards compliant" vs "safe and usable".

Yea, I'm probably oversimplifying, which is why I consider it a dumb question.

Any references to books that explain how this works would be appreciated.

even on Microsoft Windows

[Anonymous Coward]

It affects countless software applications, even on Microsoft Windows.

And linux.

Re:I love when this happens

[DJ Rubbie]

What are you talking about? All I had to do was to rebuild zlib and restart apps that used zlib. Oh, I did that while I ssh back home from work, all I had to do was to fire off a couple commands to do the same fix as apt-get.

Please, learn how a Linux/BSD system work, how linked/static binaries work before start critiquing other distributions with non-sense.

Re:I love when this happens

[hitchhacker]

$ uname -a
Linux mars 2.6.10-gentoo-r6 #1 Fri Jan 28 04:06:48 CST 2005 i686 AMD Duron(tm) processor AuthenticAMD GNU/Linux

$ for x in /usr/bin/xchat-2 /usr/bin/abiword-2.0 /usr/bin/gnome-panel /usr/bin/gnome-session /usr/bin/galeon /usr/bin/gaim /usr/bin/enlightenment; do
> ldd $x | grep libz
> done
libz.so.1 => /lib/libz.so.1 (0xb7716000)
libz.so.1 => /lib/libz.so.1 (0xb7ca4000)
libz.so.1 => /lib/libz.so.1 (0xb7655000)
libz.so.1 => /lib/libz.so.1 (0xb7785000)
libz.so.1 => /lib/libz.so.1 (0xb73d5000)
libz.so.1 => /lib/libz.so.1 (0xb773a000)
libz.so.1 => /lib/libz.so.1 (0xb7dba000)

$ ldd /usr/bin/firefox
not a dynamic executable

so.. of the programs that I could think of that link to zlib, only firefox doesn't link dynamically by default.

-metric

Re:I love when this happens

[Compenguin]

that's because /usr/bin/firefox is a shell script. The real firefox binary is loacted somewhere like /usr/lib/mozilla-firefox/firefox-bin.

$ ldd /usr/lib/mozilla-firefox/firefox-bin | grep libz
libz.so.1 => /usr/lib/libz.so.1 (0xb76b4000)

Re:If you link with zlib the right way, easy to fi

[MarcQuadra]

...I think penalties aught to be sought against these fuckwads. Using OSS has responsibilities associated with it too, you know!


No. You REALLY don't want to put the burden of responsibility on anyone wo happens upon a bug. If this person is working on project X for NASA, and they're getting paid for it and they have deadlines, I certainly DO NOT want him going off on a tangent to fix a bug in software library Y.

When you're working, your number one priority should be getting the assigned task done. There are people at NASA who do contribute oodles of code to Linux and other projects, and that's part of their job description. It's none of this guy's business to be tweaking zlib when he's getting paid to make SWIFT work.

Granted, it would be polite to drop a line to the maintainers letting them know about the bug.

As a snide post-script of sorts; people like you scare the crap out of the vast majority of businesses out there who are thinking about using OSS.

Re:I love when this happens

[Compenguin]

> Um, I'm sorry but plenty of progams that the Gentoo/BSD crowd use statically link to zlib, and thus, must be patched and rebuilt, thus meaning that the grandparent is correct.

OK name a few then, and binary programs like ut2004 or mozilla-firefox-bin that aren't compiled from source don't count.

Not surpising

[PingXao]

Have you ever looked at the zlib source code? I used it to embed some PNG capability into a utility I wrote several years ago. It's a mess. There are typedefs, macros, typedefed maros and macro'ed typedefs all over the place. I guess it's for portability but that doesn't necessarily make it programmer-friendly. I am not surprised at all.

Re:I love when this happens

[m50d]

Huh? USE=-static set, just emerge zlib, takes maybe 20 minutes on a slow system, and there you are. And it will be a faster version. The linux scheduler is good enough that if you run it at nice 19 the compile won't affect your use of the system.

more free advertising

[Anonymous Coward]

Anyone else find it odd how Whitedust seem to be using this flaw (and the apache one) to their benefit to advertise their web site.

Re:If you link with zlib the right way, easy to fi

[Anonymous Coward]

This is exactly why Windows performs the file level locking that it does and asks for a reboot when a shared library currently in use is updated. Windows installer is fully capable of cutting the file lock and replacing the file, and the reboot is more often than not just a recommendation, but a reboot is the one way to fully and completely ensure that the older version of the code is no longer being executed.

bull puckey...

[Anonymous Coward]

You cannot bind dynamic-length strings into a language and match the efficency (at least on a function-level) of a static language.

By exposing every bit of it, C allows you to keep the amount of data manipulation to a bare minimum where it counts. By the same token, it also make it necessary to do every little operation (grow the string, etc.) where the slowdown wouldn't matter.

But anyway, in a tight operation like that which makes up most of the total time spent to accomplish a task (compress a file), a Lisp implementation will be slower than a well-coded C version.

But I'm not saying that this means that C is the right language for other, less CPU-intensive tasks.

El Cubano Fullo Shito

[Anonymous Coward]

Why don't you argue that lead pipes and asbestos insulation are good building materials, while you're at it?

Rhetorical questions

[Anonymous Coward]

"The rhetorical questions remains: where's Javagator?"

Remains what, unanswered? If it's rhetorical, why are you trolling for an answer?

And they laughed at him...

[Anonymous Coward]

...for reimplementing zlib from scratch on PuTTY...

Update

[PhotoGuy]

Hmmmm, I had a minimal Fedora 3 installation on a wonderful coLinux installation, which seldom gets updated packages, but I noticed zlib was updated the other day (via freshrpms, apt-get). Kinda surprised me that something as mundane as a compression library would need updating. But I'm guessing this is why, and several days before the news hits slashdot, awesome.

Re:If you link with zlib the right way, easy to fi

[Randseed]

The half-assed solution to that is after you finish updating a lib with a security vulnerability, run a hacked version of (more or less) 'lsof' on all processes that prefixes each line with the process ID. Then grep the output for the library in question. Then, at least, you know what to fix.

I'd like to say that this shouldn't even be an issue, but unfortunately that would damn us to Windows-like constant reboot-on-install-and-update hell. (I assume this is a large reason of why Windows demands reboots after updates.)

A somewhat less foolproof solution would be to drop the machine to single user, then bring it back to multiuser. That has its own set of obvious problems that aren't acceptible in every situation.

Calling captain obvious...

[Anonymous Coward]

I'm not saying that every software project out there is suitable for this, but most userspace programs should be.

Zlib is not a userspace program. (-1 Offtopic)

What is the big deal

[Anonymous Coward]

What is the big deal? The bug was fixed last week on Gentoo.

http://bugs.gentoo.org/show_bug.cgi?id=98121

Possible bandaid:

[The Master Control P]

I tried ./Configure --help with OpenSSL-0.9.7g for Linux, and noticed something that might help in the output:

[[no-]zlib|zlib-dynamic]

Would it help to re[configure & compile & install] with the no-zlib option? I know it's just a bandaid, but every little bit helps, right?

Fixed in Oct 2004?

[kybred]

According to the CERT bulletin, zlib 1.2.2, released in Oct 2004, fixes this issue. Is this really patched or not?

Re:Fixed in Oct 2004?

[kybred]

To answer my own question, the problem noted in the summary is new; zlib 1.2.2 has this problem.

The CERT bulletin I looked at was old (why don't they post the date at the top of the bulletin!).

kybred

very complex code

[ep385]

> You cannot bind dynamic-length strings into a
> language and match the efficency (at least on a
> function-level) of a static language.

Actually it turns out you can Anonymous Coward, at least in Common Lisp. Through the use of type declarations and the 'optimize' declaration you can declare your intentions that it's more important that a piece of code run fast than it be safe from bad input. In that case string operations are as fast as C.

Generally you only worry about speeding up the key loops in your code (there's that old saying that N% of the time is spent in M% of the code for N >> M). Before removing type checks through the optimize declaration you add code to ensure that the input is ok and then you let the compiler generate optimized code for the loop. Thus you're only vulnerable in a small part of your code and with your own checks you eliminte that vulnerability. With C you're vulnerable in all your code by default. And that brings us back to zlib...

Re:If you link with zlib the right way, easy to fi

[syousef]

Remember too that an upgrade to a dynamically linked function means that proper testing must include all software that uses that function.

Wrong. Totally wrong way to go about it. If you're building with a library the library should be treated as a black box. If you can't trust it to work as a black box, don't use it.

The habbit of doing this kind fo regression testing is a bad habbit brought about by people relying on bad libraries. Think about it. If a mechanic replaces an air or oil filter, he may test the car to make sure it runs but he sure as hell isn't going to go through the rigorous checks the engine went through at the factory. If mechanics worked that way they'd be out of business. So too with coders.

Re:If you link with zlib the right way, easy to fi

[snookums]

So, does anyone know a good way for me to identify binaries on my system which are statically linked to zlib?

Will this work? Is "zlib" a good enough string to search for?


#!/bin/bash
grep -rl zlib /{,usr/,usr/X11R6/,usr/local/}{s,}bin 2>/dev/null | while read FILE ; do
if ! (ldd ${FILE} | grep libz >/dev/null); then
echo ${FILE}
fi
done


And yeah, I know there are other locations where I have binaries (e.g. ~/bin, /opt/*). This is just a proof-of-concept.

Re:If you link with zlib the right way, easy to fi

[Anonymous Coward]

Don't assume you found the same vulnerability. If you look at the zlib checkins you'll see there are plenty of similair patches which aren't so wildly publicized. This one's obscure too and an automated too would have no way of detecting it.

Re:If you link with zlib the right way, easy to fi

[Anonymous Coward]

>If you're building with a library the library should be treated as a black box. If you can't trust it to work as a black box, don't use it.

I take it you don't use many libraries, do you?

Re:If you link with zlib the right way, easy to fi

[Anonymous Coward]

I think you mean pwned.

Re:I love when this happens

[Lost Found]

I, for one, was pleased to be a user of Gentoo Linux when this bug hit. I got notification via e-mail before the CERT report was even live, and after an emerge --sync and emerge zlib (the compiling of zlib took hardly any time at all), I ran lsof | grep libz and restarted the affected packages. Fixing my cluster at work took a few minutes, and I fixed my machines at home over SSH.

bug free security problems

[xixax]

And just what is "bug free" anyway?

To use a building analogy, we can create a perfect, bug-free classes, doors, windows, chimneys, plumbing, etc. etc. But even with "perfect" classes, you'll get some "idiot" using them in unexpected ways, like uing a shower curtain instead of a security screen for the front door.

Do we counter this by restricting how the "shower screen" class can be used? Or by limiting what kinds of classes can be inserted into ExteriorWall()? Can we reasonably anticipate how people will combine things?

To paraphrase The Young Ones, "Well it doesn't say, 'do not fill VCR with dish washing detergent'"

Xix.

We need static relinking!

[Deven]

There are good arguments for both static and dynamic linking -- neither is inherently better than the other for all situations.

What we really need is the ability to take a statically-linked binary and relink it with a new version of an included library like this, without requiring the source or object files. (Of course, this would require retaining relocation information in the binary.)

If relinking were possible, needed security fixes could be safely relinked into existing static binaries on demand, without risking random breakage from library upgrades...

Re:If you link with zlib the right way, easy to fi

[Ed Avis]

or do running programs cache the old library, and therefore need a restart?

This is a damn good point. The same applies to replacing some executable - if the old version is still running then you're still vulnerable. Or a data file; because of Unix filesystem semantics you can have the old version open, it gets unlinked and replaced, but a process can have an open filehandle to the old version. In fact, anything loaded into memory from disk is a place for old copies to lurk.

And yet aren't these the advantages of Unix-like systems over Windows? That you don't have to reboot after every update, that you can gracefully restart services?

Maybe the only way to _really_ fix the problem is to go to a managed code system (Java, or .net, or some Lisp edifice) which can dynamically load and unload bits of code, and guarantee that the old version is no longer running.