DieHard, the Software

Roland Piquepaille writes "No, it's not another movie sequel. DieHard is a piece of software which helps programs to run correctly and protects them from a range of security vulnerabilities. It has been developed by computer scientists from the University of Massachusetts Amherst — and Microsoft. DieHard prevents crashes and hacker attacks by focusing on memory. Our computers have thousands times more memory than 20 years ago. Still, programmers are privileging speed and efficiency over security, which leads to the famous "buffer overflows" which are exploited by hackers."

Comments

Vista already doing some of this

[PurifyYourMind]

Along the same lines anyway... a new feature in Vista: Address space layout randomization (ASLR) is a computer security technique which involves arranging the positions of key data areas, usually including the base of the executable and position of libraries, heap, and stack, randomly in a process' address space. http://en.wikipedia.org/wiki/Address_space_layout_ randomization

Re:Vista already doing some of this

[Anonymous Coward]

This came out in OpenBSD 3.3 over three years ago. Nice to see Microsoft keeping up with the times.

Re:Vista already doing some of this

[Salvance]

Sure, but wouldn't it be better if everything ran in it's own virtual session (or within a virtual secure space)? This was Microsoft's original plan with it's Palladium component of Longhorn, but my understanding is that this was almost entirely scrapped to get Vista out the door.

Part of the other problem is that most home users expect secure data, but they aren't willing to do anything about it (e.g. set up non-admin users, install virus checkers/firewalls/etc).

Re:Vista already doing some of this

[Ristretto]

Hi Slashdot readers,

DieHard's randomization is very different from what OpenBSD does, not to mention Vista's address-space randomization. I've added a note to the FAQs that explains the difference in some detail, and answers several other questions, but in short: "address-space randomization" randomizes the base address of the heap and also mmapped-chunks of memory, leaving the relative position of objects intact. By contrast, DieHard randomizes the location of every single object across the entire heap. It also goes further in that it prevents a wide range of memory errors automatically, like double frees and illegal frees, and effectively eliminates heap corruption.

-- Emery Berger

Re:Vista already doing some of this

[strider44]

You could have just looked it up and seen that it's been in Linux for a similar length of time (in 2.6.x). I just googled for "linux address randomization" and clicked the top link.

Re:Vista already doing some of this

[nacturation]

Seems like OpenBSD's implementation does what DieHard claims, or at least some of it. See this interview from August 2005 for information:

http://kerneltrap.org/node/5584

Any thoughts?

Re:Vista already doing some of this

[Tim C]

Vista has been in development for around 5 years; unless you were expecting this to be released as a service pack for XP or Server 2003, what's your point? It's in MS's latest release, what more do you want? (Yeah, a shorter release cycle would be nice - except that then people would bitch about the upgrade treadmill...)

Re:Vista already doing some of this

[iamacat]

No, it won't. Programs need to interoperate - you might want to explicitly upload a photo to shutterfly using your web browser, but you don't want a rogue website to just siphon off all your private photos by exploiting a memory bug in one of the endless plugins.

The real solution is programming in a language with secure memory management, such as .Net, Java or even LISP. I suspect that overhead is far smaller than running 3 copies of the program at once like DieHard does.

Re:Vista already doing some of this

[Ristretto]

Hi,

Here's a more detailed answer -- I'll add it to the FAQ.

OpenBSD (a variant of PHKmalloc) does some of what DieHard's allocator does, but DieHard does much more. On the security side, DieHard adds much more "entropy"; on the reliability side, it mathematically reduces the risk that a programmer bug will have any impact on program execution.

OpenBSD randomly locates pages of memory and allocates small objects from these pages. It improves security by avoiding the effect of certain errors. Like DieHard, it is resilient to double and invalid frees. It places guard pages around large chunks and frees such large chunks back to the OS (causing later references through dangling pointers to fail unless the chunk is reused). It attempts to block some buffer overflows by using page protection. Finally, it shuffles some allocated objects around on a page, randomizing their location within a page.

DieHard goes much further. First, it completely segregates heap metadata from the heap, making heap corruption (and hijack attacks) nearly impossible. On OpenBSD, a large-enough underflow on OpenBSD can overwrite the page directory or local page info struct (at the beginning of each page), hijacking the allocator. This presentation describes several ways OpenBSD's allocator can be attacked. By contrast, none of DieHard's metadata is located in the allocated object space.

Second, DieHard randomizes the placement of objects across the entire heap. This has numerous advantages. On the security side, it makes brute-force attempts to locate adjacent objects nearly impossible -- in OpenBSD, knowing the allocation sequence determines which pages objects will land on (see the presentation pointed to above).

DieHard's complete randomization is key to provably avoiding a range of errors with high probability. It reduces the worst-case odds that a buffer overflow has any impact to 50%. The actual likelihood is even lower when the heap is not full. DieHard also avoids dangling pointer errors with very high probability (e.g., 99.999%), making it nearly impervious to such mistakes. You can read our PLDI paper for more details and formulae.

-- Emery Berger

Correction

[realmolo]

"Still, programmers are privileging speed and efficiency over security..."

Speed and efficiency of *development*, maybe.

Which is the problem. Modern software is so dependent on toolkits and compiler optimizations and various other "pre-made" pieces, that any program of even moderate complexity is doing things that the programmer isn't really aware of.

Re:Correction

[MBCook]

This is one of the arguments for a language running on a VM like Java, C#, or Python. They can do runtime checking of array bounds and such and throw an exception or crash instead of silently overwriting some other variable that only may or may not cause a crash or some other noticeable side effect later.

Re:Correction

[AKAImBatman]

"Still, programmers are privileging speed and efficiency over security..."

Speed and efficiency of *development*, maybe.

No, it was right the first time. Java is several orders of magnitude more secure by default than any random C or C++ program. Yet mention Java on a forum like, say, Slashdot, and you'll hear no end to how much Java sucks because "it's slow". (Usually ignoring the massive speedups that have happened since they last tried it 1996.) It doesn't matter that the tradeoff for that speed is flexibility, security, and portability. They want things to be fast for some undefined quantity of fast.

In fact, I predict that someone will be along to argue just how slow Java is in 3... 2... 1...

Re:Correction

[Anonymous Coward]

No, we want things to be as fast as they can be.

Maybe, but most programs are not written in a way which will achieve this goal.

Programmer time is a limited resource. This is true even on a hobby project with no deadlines and everybody working for free; you want to ship sometime. Making programs run fast takes a lot of programmer time, even when you use a language which is supposedly fast by default such as C or C++.

C and C++ make you spend a lot of time working around weaknesses in the language and fixing bugs that other languages can never have. A great deal of programmer time is put into developing the broken and slow implementation of half of Common Lisp that every sufficiently complex program must contain.

All of this time spent is time that does not go into making the program fast.

By using a language that makes programmers more productive, you get a lot more time to make the program fast. You can do this by optimizing in the "slow" language you started with, by rewriting inner loops in C, by changing the whole algorithm to run on the GPU, etc.

The 90/10 rule says that your program spends 90% of its time in only 10% of its code, and that optimizing the other 90% of the code is basically a waste. And yet people who want their programs to "go fast" are writing that 90% in a low-level language, effectively wasting a large amount of effort.

You may also end up getting your program working, realize that it actually is fast enough despite being written in a really slow interpreted language, and spend the time you saved making more cool software. Or you can go back and make the original product fast. It's up to you.

There are many good reasons to use C, and many good reasons to write entire programs in C, but "it's fast" is not a particularly good reason. An app written in pure C is probably not as fast as it can be unless its scope is very limited.

Re:Correction

[Jeremi]

He must be criticizing open source programmers only. Because in business, programmers aren't focussed on speed and efficiency

Business software isn't the problem. The software that is the problem is the software that runs on every naive home user's PC ... Windows, Outlook, IE, Mozilla, AIM, etc etc. This is the software whose security problems allow spam, credit card fraud, virus outbreaks, etc. And last time I checked, all of that stuff is still written in C or C++, not in any VM.

Berger sounds like a VM-language bigot (or paid ($30K from MS) .Net Runtime shill)
who doesn't understand how most software is really made, and prefers to believe in caricatures of programmers.

Great, you've called the guy a bigot, a shill, and an idiot, without even having understood what he was talking about.

Re:Correction

[evilviper]

It doesn't matter that the tradeoff for that speed is flexibility, security, and portability. They want things to be fast for some undefined quantity of fast.

I've got to call you on the "portability" crap.

Java is about as portable as Flash... Sure, the major platforms are supported, but that's it. 3rd parties spent a lot of time trying to impliment java, but never did get everything 100%. Licensing issues, above all else, made it a real hassle to get Java on platforms like FreeBSD.

Meanwhile, C and C++ compiler are installed in the base system by default.

The only "portability" advantage Java has is perhaps in GUI apps, and that's at the expense of a program that doesn't look or work remotely similar to any other app on the system...

There are a great many reasons people don't use java. Performance is only a minor one.

Re:Correction

[Anonymous Coward]

"Java is slow" is the stated reason. As you noted, it is not the actual reason. To tell the actual reason is difficult, but in short Java reminds us too much of what it should have been.

The basic complaints I have heard are these:

Complaint 1: Java is slow.
  As you stated, this is not a meaningful complaint.

Complaint 2: Garbage Collection stinks
  GC is an obvious requirement of a "safe" language. As implemented in Java, it is downright stupid. When doing something CPU intensive, the GC never runs, leading to gobbling up memory until there is no more and thrashing to death. I'm sure that somebody is going to dig up that paging-free GC paper, but pay attention: that is a kernel-level GC.

Complaint 3: Swing is ugly/leaks memory
  The first is a matter of opinion. The second is well-known. Swing keeps references to long-dead components hidden in internal collections leading to massive memory leaks. These memory leaks can be propagated to the parent application if it is also written in Java.

Complaint 4: Bad build system
  Java cannot do incremental builds if class files have circular references. In a small project of about ten classes I was working on, the only way to build it was "rm *.class ; javac *.java"

Complaint 5: Tied class hierarchy to filesystem hierarchy
  This was just stupid and interacts badly with Windows (and anything else with a case insensitive filesystem). It is even worse for someone who is first learning the language. It also makes renaming classes have a very bad effect on source control.

Complaint 6: Lack of C++ templates
  C++ has some of its own faults. Fortunately its template system can be leveraged to fix quite a few of them. Java's generics have insufficient power to do the same thing.

Complaint 7: Lack of unsigned integer
  These are oh-so-necessary when doing all kinds of things with binary formats. Too bad Java and all its descendents don't have them.

Complaint 8: Verbosity without a point
  It has gotten so bad in places that I am strongly tempted to pass Java through the C preprocessor first, but I can't do that very well because of 4.

Re:Correction

[RAMMS EIN]

In support of your point, I want to share the followind PDF: Lisp as an Alternative to Java. The study compares programs written in Java, C/C++ (they lump these togeter), and Common Lisp / Scheme (again, lumped together). The findings are basically:

1. The fastest programs are written in C/C++
2. On average, Lisp/Scheme programs are fasted, followed by C/C++ programs, and Java programs are way behind.
3. Development time is shortest for Lisp/Scheme, with Java and C++ being more or less equal.
4. C/C++ programs used the least memory, with Lisp/Scheme and Java being about equal.
5. There was very little variation in the run time and development time of Lisp/Scheme programs, and a lot of variation everywhere else.

The PDF contains some nice graphs illustrating all this.

Re:Different program?

[j00r0m4nc3r]

Yeah, he should have named his project Die Harder

Re:Different program?

[baldass_newbie]

Or DieHardWithAVengeance...

You must remember...

[jd]

...the number of programmers like ourselves who learned how to code correctly is vanishingly small in comparison to the number of coders who assume that if it doesn't crash, it's good enough. Whether you validate the inputs against the constraints, engineer the program such that constraints must always be met, or force a module to crash when something is invalid so that you can trap and handle it by controlled means - the method is irrelevant. What matters is less that you're using a method than you remember to use a method.

Even assuming nobody wants to go to all that trouble, there are solutions. ElectricFence and dmalloc are hardly new and far from obscure. If a developer can't be bothered to link against a debugging malloc before testing then you can't expect their software to be immune to such absurd defects. A few runs whilst using memprof isn't a bad idea, either.

This assumes you're using a language like C, which is not a trivial language to write correct software in. For many programs, you are better off with a language like Occam (provided for Unix/Linux/Windows via KROC) where the combination of language and compiler heavily limits the errors you can introduce. Yes, languages this strict are a pain to write in, but the increase in the initial pain is vastly outweighed by the incredible reduction in agony when debugging - if there's any debugging at all.

I do not expect anyone to re-write glibc in Occam or any other nearly bug-proof language. It would be helpful, but it's not going to happen.

Re:Different program?

[Werkhaus]

I thought the random number generator was DieBold?

nothing to do with VMs - just exception handling

[Anonymous Coward]

Ada's been doing that kind of runtime checking and throwing exceptions for 20 years now without needing a VM to enable exception handling.

Lots of memory available?

[NorbrookC]

In reading this article, I started to wonder a lot about this. writing to conserve memory is a bad thing? I will say that I haven't noticed that in most software, regardless of whether it's OSS or closed-source. If anything, there seems to be a variation of Parkinson's Law in effect. Yes, computers these days have a lot more memory available, however, the number of applications and the size demands of each application has grown almost in lock-step with that. 15 or so years ago, yes, you had one OS and one application running - maybe, if you were lucky or were running TSR apps, two or three. These days, the OS takes up a hefty chunk, and it's not uncommon to see 8 or 9 (if not more) applications running at once. What they all seem to have in common is that they assume they have access to all the RAM, or as much of it as they can grab.

I have to wonder if he's actually looked at things these days. I don't see where programming (properly done) to conserve memory is a bad thing. If anything, it seems that few are actually doing it.

Re:Different program?

[Joebert]

No, you're thinking of the random Vote generator.

Re:What a load of shit.

[Anonymous Coward]

I did a quick read of the whitepaper and sort of see it as heap randomization+. I have very little faith in the claims of low overhead. But leaving that aside, there are 2 major problems here:

1) If there is a program crash, it may be possible to reproduce the bug on the same computer, but probably not on 2 different ones, such as the user's and the developer's.

2) It discourages programmers from good design and thorough testing by leading them to believe that bugs won't occur.

The claim for DieHard (from the whitepaper) is that it "tolerates memory errors and provides probabilistic memory safety". But bugs will still happen! I once added about 10 lines of code to log a bug our team was having a hard time tracking down. It turned out to have its own bug that would be hit if:

- Two threads were accessing the same buffer
AND
- One of them was swapped out during the execution of 3 machine instructions (out of about a million)

It took my moderately sized customer base 2 weeks to hit it. The only way to avoid memory errors is to make the code bulletproof, which means fixing it when bugs are reported.

Speed - buffer overflow?

[BagOCrap]

Still, programmers are privileging speed and efficiency over security, which leads to the famous "buffer overflows"... Am I the only one who finds the above sentence just strange? In my books and experience, speed optimisations most certainly don't result in buffer overflows. Recklessness does, however.