DieHard, the Software

Roland Piquepaille writes "No, it's not another movie sequel. DieHard is a piece of software which helps programs to run correctly and protects them from a range of security vulnerabilities. It has been developed by computer scientists from the University of Massachusetts Amherst — and Microsoft. DieHard prevents crashes and hacker attacks by focusing on memory. Our computers have thousands times more memory than 20 years ago. Still, programmers are privileging speed and efficiency over security, which leads to the famous "buffer overflows" which are exploited by hackers."

Comments

Vista already doing some of this

[PurifyYourMind]

Along the same lines anyway... a new feature in Vista: Address space layout randomization (ASLR) is a computer security technique which involves arranging the positions of key data areas, usually including the base of the executable and position of libraries, heap, and stack, randomly in a process' address space. http://en.wikipedia.org/wiki/Address_space_layout_ randomization

Re:Vista already doing some of this

[Anonymous Coward]

This came out in OpenBSD 3.3 over three years ago. Nice to see Microsoft keeping up with the times.

Re:Vista already doing some of this

[Salvance]

Sure, but wouldn't it be better if everything ran in it's own virtual session (or within a virtual secure space)? This was Microsoft's original plan with it's Palladium component of Longhorn, but my understanding is that this was almost entirely scrapped to get Vista out the door.

Part of the other problem is that most home users expect secure data, but they aren't willing to do anything about it (e.g. set up non-admin users, install virus checkers/firewalls/etc).

Re:Vista already doing some of this

[Ristretto]

Hi Slashdot readers,

DieHard's randomization is very different from what OpenBSD does, not to mention Vista's address-space randomization. I've added a note to the FAQs that explains the difference in some detail, and answers several other questions, but in short: "address-space randomization" randomizes the base address of the heap and also mmapped-chunks of memory, leaving the relative position of objects intact. By contrast, DieHard randomizes the location of every single object across the entire heap. It also goes further in that it prevents a wide range of memory errors automatically, like double frees and illegal frees, and effectively eliminates heap corruption.

-- Emery Berger

Re:Vista already doing some of this

[strider44]

You could have just looked it up and seen that it's been in Linux for a similar length of time (in 2.6.x). I just googled for "linux address randomization" and clicked the top link.

Re:Vista already doing some of this

[nacturation]

Seems like OpenBSD's implementation does what DieHard claims, or at least some of it. See this interview from August 2005 for information:

http://kerneltrap.org/node/5584

Any thoughts?

Re:Vista already doing some of this

[Tim C]

Vista has been in development for around 5 years; unless you were expecting this to be released as a service pack for XP or Server 2003, what's your point? It's in MS's latest release, what more do you want? (Yeah, a shorter release cycle would be nice - except that then people would bitch about the upgrade treadmill...)

Re:Vista already doing some of this

[iamacat]

No, it won't. Programs need to interoperate - you might want to explicitly upload a photo to shutterfly using your web browser, but you don't want a rogue website to just siphon off all your private photos by exploiting a memory bug in one of the endless plugins.

The real solution is programming in a language with secure memory management, such as .Net, Java or even LISP. I suspect that overhead is far smaller than running 3 copies of the program at once like DieHard does.

Re:Vista already doing some of this

[Alioth]

OpenBSD prevents double frees and illegal frees and heap corruption too, and has been doing so for at least a couple of years. The code is BSD licensed too, so you can use it in closed source products like Windows. OpenBSD also has had something called W^X (Write XOR execute) for several years now, even for CPU architectures that don't support making executable pages read-only.

Re:Vista already doing some of this

[Ristretto]

Hi,

Here's a more detailed answer -- I'll add it to the FAQ.

OpenBSD (a variant of PHKmalloc) does some of what DieHard's allocator does, but DieHard does much more. On the security side, DieHard adds much more "entropy"; on the reliability side, it mathematically reduces the risk that a programmer bug will have any impact on program execution.

OpenBSD randomly locates pages of memory and allocates small objects from these pages. It improves security by avoiding the effect of certain errors. Like DieHard, it is resilient to double and invalid frees. It places guard pages around large chunks and frees such large chunks back to the OS (causing later references through dangling pointers to fail unless the chunk is reused). It attempts to block some buffer overflows by using page protection. Finally, it shuffles some allocated objects around on a page, randomizing their location within a page.

DieHard goes much further. First, it completely segregates heap metadata from the heap, making heap corruption (and hijack attacks) nearly impossible. On OpenBSD, a large-enough underflow on OpenBSD can overwrite the page directory or local page info struct (at the beginning of each page), hijacking the allocator. This presentation describes several ways OpenBSD's allocator can be attacked. By contrast, none of DieHard's metadata is located in the allocated object space.

Second, DieHard randomizes the placement of objects across the entire heap. This has numerous advantages. On the security side, it makes brute-force attempts to locate adjacent objects nearly impossible -- in OpenBSD, knowing the allocation sequence determines which pages objects will land on (see the presentation pointed to above).

DieHard's complete randomization is key to provably avoiding a range of errors with high probability. It reduces the worst-case odds that a buffer overflow has any impact to 50%. The actual likelihood is even lower when the heap is not full. DieHard also avoids dangling pointer errors with very high probability (e.g., 99.999%), making it nearly impervious to such mistakes. You can read our PLDI paper for more details and formulae.

-- Emery Berger

Different program?

[Anonymous Coward]

I thought DieHard was a random number generator test suite. It is annoying when people dont even look around for other programs with the same name and do similar things.

Re:Different program?

[j00r0m4nc3r]

Yeah, he should have named his project Die Harder

Re:Different program?

[baldass_newbie]

Or DieHardWithAVengeance...

Re:Different program?

[Werkhaus]

I thought the random number generator was DieBold?

Re:Different program?

[Joebert]

No, you're thinking of the random Vote generator.

Re:Different program?

[pacinpm]

Problem is it is not so random.

Correction

[realmolo]

"Still, programmers are privileging speed and efficiency over security..."

Speed and efficiency of *development*, maybe.

Which is the problem. Modern software is so dependent on toolkits and compiler optimizations and various other "pre-made" pieces, that any program of even moderate complexity is doing things that the programmer isn't really aware of.

Re:Correction

[MBCook]

This is one of the arguments for a language running on a VM like Java, C#, or Python. They can do runtime checking of array bounds and such and throw an exception or crash instead of silently overwriting some other variable that only may or may not cause a crash or some other noticeable side effect later.

Re:Correction

[AKAImBatman]

"Still, programmers are privileging speed and efficiency over security..."

Speed and efficiency of *development*, maybe.

No, it was right the first time. Java is several orders of magnitude more secure by default than any random C or C++ program. Yet mention Java on a forum like, say, Slashdot, and you'll hear no end to how much Java sucks because "it's slow". (Usually ignoring the massive speedups that have happened since they last tried it 1996.) It doesn't matter that the tradeoff for that speed is flexibility, security, and portability. They want things to be fast for some undefined quantity of fast.

In fact, I predict that someone will be along to argue just how slow Java is in 3... 2... 1...

Re:Correction

[Anonymous Coward]

No, we want things to be as fast as they can be.

Maybe, but most programs are not written in a way which will achieve this goal.

Programmer time is a limited resource. This is true even on a hobby project with no deadlines and everybody working for free; you want to ship sometime. Making programs run fast takes a lot of programmer time, even when you use a language which is supposedly fast by default such as C or C++.

C and C++ make you spend a lot of time working around weaknesses in the language and fixing bugs that other languages can never have. A great deal of programmer time is put into developing the broken and slow implementation of half of Common Lisp that every sufficiently complex program must contain.

All of this time spent is time that does not go into making the program fast.

By using a language that makes programmers more productive, you get a lot more time to make the program fast. You can do this by optimizing in the "slow" language you started with, by rewriting inner loops in C, by changing the whole algorithm to run on the GPU, etc.

The 90/10 rule says that your program spends 90% of its time in only 10% of its code, and that optimizing the other 90% of the code is basically a waste. And yet people who want their programs to "go fast" are writing that 90% in a low-level language, effectively wasting a large amount of effort.

You may also end up getting your program working, realize that it actually is fast enough despite being written in a really slow interpreted language, and spend the time you saved making more cool software. Or you can go back and make the original product fast. It's up to you.

There are many good reasons to use C, and many good reasons to write entire programs in C, but "it's fast" is not a particularly good reason. An app written in pure C is probably not as fast as it can be unless its scope is very limited.

Re:Correction

[Jeremi]

He must be criticizing open source programmers only. Because in business, programmers aren't focussed on speed and efficiency

Business software isn't the problem. The software that is the problem is the software that runs on every naive home user's PC ... Windows, Outlook, IE, Mozilla, AIM, etc etc. This is the software whose security problems allow spam, credit card fraud, virus outbreaks, etc. And last time I checked, all of that stuff is still written in C or C++, not in any VM.

Berger sounds like a VM-language bigot (or paid ($30K from MS) .Net Runtime shill)
who doesn't understand how most software is really made, and prefers to believe in caricatures of programmers.

Great, you've called the guy a bigot, a shill, and an idiot, without even having understood what he was talking about.

Re:Correction

[evilviper]

It doesn't matter that the tradeoff for that speed is flexibility, security, and portability. They want things to be fast for some undefined quantity of fast.

I've got to call you on the "portability" crap.

Java is about as portable as Flash... Sure, the major platforms are supported, but that's it. 3rd parties spent a lot of time trying to impliment java, but never did get everything 100%. Licensing issues, above all else, made it a real hassle to get Java on platforms like FreeBSD.

Meanwhile, C and C++ compiler are installed in the base system by default.

The only "portability" advantage Java has is perhaps in GUI apps, and that's at the expense of a program that doesn't look or work remotely similar to any other app on the system...

There are a great many reasons people don't use java. Performance is only a minor one.

Re:Correction

[Anonymous Coward]

"Java is slow" is the stated reason. As you noted, it is not the actual reason. To tell the actual reason is difficult, but in short Java reminds us too much of what it should have been.

The basic complaints I have heard are these:

Complaint 1: Java is slow.
  As you stated, this is not a meaningful complaint.

Complaint 2: Garbage Collection stinks
  GC is an obvious requirement of a "safe" language. As implemented in Java, it is downright stupid. When doing something CPU intensive, the GC never runs, leading to gobbling up memory until there is no more and thrashing to death. I'm sure that somebody is going to dig up that paging-free GC paper, but pay attention: that is a kernel-level GC.

Complaint 3: Swing is ugly/leaks memory
  The first is a matter of opinion. The second is well-known. Swing keeps references to long-dead components hidden in internal collections leading to massive memory leaks. These memory leaks can be propagated to the parent application if it is also written in Java.

Complaint 4: Bad build system
  Java cannot do incremental builds if class files have circular references. In a small project of about ten classes I was working on, the only way to build it was "rm *.class ; javac *.java"

Complaint 5: Tied class hierarchy to filesystem hierarchy
  This was just stupid and interacts badly with Windows (and anything else with a case insensitive filesystem). It is even worse for someone who is first learning the language. It also makes renaming classes have a very bad effect on source control.

Complaint 6: Lack of C++ templates
  C++ has some of its own faults. Fortunately its template system can be leveraged to fix quite a few of them. Java's generics have insufficient power to do the same thing.

Complaint 7: Lack of unsigned integer
  These are oh-so-necessary when doing all kinds of things with binary formats. Too bad Java and all its descendents don't have them.

Complaint 8: Verbosity without a point
  It has gotten so bad in places that I am strongly tempted to pass Java through the C preprocessor first, but I can't do that very well because of 4.

Re:Correction

[tulrich]

Java is several orders of magnitude more secure by default than any random C or C++ program.

Do you know what "several orders of magnitude" means? For variety, next time you should write "... exponentially more secure ..." or "... takes security to the next level!"

BTW, it's funny you should mention Java performance in this thread -- one of the DieHard authors published this fascinating paper on Java GC performance: http://citeseer.ist.psu.edu/hertz05quantifying.htm l -- executive summary: GC can theoretically be as fast as explicit malloc/free, if you're willing to spend 5x memory size overhead (gulp).

Re:Correction

[smallfries]

This implies is that because memory is larger less attention can be paid to efficiency, but the hapless programmers don't know better. I used to use quicksort when I had 640 KiB of RAM, but now that I have 8 GiB, I'll just use bubble sort. Brilliant. You are really misrepresenting his point here. We both know that bubble sort would run much slower on a 8Gb dataset than quicksort. The real comparison is "should we some really tricky and nasty code for this particular function or should it be a giant lookup table?" When memory is (relatively) cheaper than processor time, the set of tradeoffs changes. Some of these tradeoffs then mean than code can be written more correctly (securely) at the expense of higher memory usage. These tradeoffs are intuitively bad to someone who cut their teeth on a 16bit processor with hardly any memory, but as I'm sitting writing this there is only 130Mb out of 1Gb in use (not counting the cache) - who is to say that doubling memory usage is bad if it removes a whole class of bugs and holes?

Re:Correction

[RAMMS EIN]

In support of your point, I want to share the followind PDF: Lisp as an Alternative to Java. The study compares programs written in Java, C/C++ (they lump these togeter), and Common Lisp / Scheme (again, lumped together). The findings are basically:

1. The fastest programs are written in C/C++
2. On average, Lisp/Scheme programs are fasted, followed by C/C++ programs, and Java programs are way behind.
3. Development time is shortest for Lisp/Scheme, with Java and C++ being more or less equal.
4. C/C++ programs used the least memory, with Lisp/Scheme and Java being about equal.
5. There was very little variation in the run time and development time of Lisp/Scheme programs, and a lot of variation everywhere else.

The PDF contains some nice graphs illustrating all this.

Re:Correction

[durdur]

> Sure, the major platforms are supported, but that's it

What platforms do you want? Java is on 32- and 64-bit Windows, 32- and 64-bit Linux, Solaris (SPARC + Intel), AIX, HP-UX, OS X (Intel + PPC), IBM z/OS and iSeries, AND FreeBSD (http://www.freebsd.org/java/) among other platforms. FreeBSD guys negotiated a license (but with Java GPL now they won't need to in future).

I would also disagree about portability of code. J2SE apps are very portable. Usually with zero platform specific code changes. J2EE is another story.

Let's solve voting security.

[Wilson_6500]

If you were somehow to install DieHard software on a DieBold machine, does the universe collapse in on itself? This is one of those pasta plus antipasto situations, I think.

Buggy

[The MAZZTer]

Firefox 2 crashed for the first time ever (I've used it since beta 1 came out) for me today... suspiciously, less than five minutes after I turned DieHard on. Hrm.

You must remember...

[jd]

...the number of programmers like ourselves who learned how to code correctly is vanishingly small in comparison to the number of coders who assume that if it doesn't crash, it's good enough. Whether you validate the inputs against the constraints, engineer the program such that constraints must always be met, or force a module to crash when something is invalid so that you can trap and handle it by controlled means - the method is irrelevant. What matters is less that you're using a method than you remember to use a method.

Even assuming nobody wants to go to all that trouble, there are solutions. ElectricFence and dmalloc are hardly new and far from obscure. If a developer can't be bothered to link against a debugging malloc before testing then you can't expect their software to be immune to such absurd defects. A few runs whilst using memprof isn't a bad idea, either.

This assumes you're using a language like C, which is not a trivial language to write correct software in. For many programs, you are better off with a language like Occam (provided for Unix/Linux/Windows via KROC) where the combination of language and compiler heavily limits the errors you can introduce. Yes, languages this strict are a pain to write in, but the increase in the initial pain is vastly outweighed by the incredible reduction in agony when debugging - if there's any debugging at all.

I do not expect anyone to re-write glibc in Occam or any other nearly bug-proof language. It would be helpful, but it's not going to happen.

nothing to do with VMs - just exception handling

[Anonymous Coward]

Ada's been doing that kind of runtime checking and throwing exceptions for 20 years now without needing a VM to enable exception handling.

Lots of memory available?

[NorbrookC]

In reading this article, I started to wonder a lot about this. writing to conserve memory is a bad thing? I will say that I haven't noticed that in most software, regardless of whether it's OSS or closed-source. If anything, there seems to be a variation of Parkinson's Law in effect. Yes, computers these days have a lot more memory available, however, the number of applications and the size demands of each application has grown almost in lock-step with that. 15 or so years ago, yes, you had one OS and one application running - maybe, if you were lucky or were running TSR apps, two or three. These days, the OS takes up a hefty chunk, and it's not uncommon to see 8 or 9 (if not more) applications running at once. What they all seem to have in common is that they assume they have access to all the RAM, or as much of it as they can grab.

I have to wonder if he's actually looked at things these days. I don't see where programming (properly done) to conserve memory is a bad thing. If anything, it seems that few are actually doing it.

Re:What a load of shit.

[Anonymous Coward]

I did a quick read of the whitepaper and sort of see it as heap randomization+. I have very little faith in the claims of low overhead. But leaving that aside, there are 2 major problems here:

1) If there is a program crash, it may be possible to reproduce the bug on the same computer, but probably not on 2 different ones, such as the user's and the developer's.

2) It discourages programmers from good design and thorough testing by leading them to believe that bugs won't occur.

The claim for DieHard (from the whitepaper) is that it "tolerates memory errors and provides probabilistic memory safety". But bugs will still happen! I once added about 10 lines of code to log a bug our team was having a hard time tracking down. It turned out to have its own bug that would be hit if:

- Two threads were accessing the same buffer
AND
- One of them was swapped out during the execution of 3 machine instructions (out of about a million)

It took my moderately sized customer base 2 weeks to hit it. The only way to avoid memory errors is to make the code bulletproof, which means fixing it when bugs are reported.

Randomness. Nooooo!

[istartedi]

The worst bugs are the ones that are hard to reproduce. In fact, when faced with a bug that's difficult to reproduce, I've been known to quip "yet another unintentional random number generator". The suggestion that they're going to apply a pseudo-fix that involves random allocations raises all kinds of red flags. I'd much rather have fine-grained control over which sections of code are allowed to access which sections of memory, and be able to track which sections of code are accessing a chunk of memory. I'd much rather have strict enforcement of a non-execute bit on memory that's only supposed to contain data (there is some support for this already). Introducing randomness into memory allocation? Worst. Idea. Ever. It's like throwing in the towel, and if they put that in at low levels in system libs and things like that, we're screwed in terms of every being able to *really* fix the problem. If their compiler is going to link against an allocator that has this capability, I hope they provide the ability to disable it.

Corrections

[SuperKendall]

Basically almost every point you raised can be addressed simply by saying "get your head out of five years in the past". Moderm GC can take little overhead, and will run when needed even with the CPU being consumed.

Swing does not really have the problems you speak of any longer, if you are using it right... heck, it didn't really have those problem to any great degree about seven years ago when I was building a large custom client app all in swing for only desktop deployment.

Complaining about the build system is like saying GCC has a bad build system - really it has no build system, and you should use something made for building Java. That is why we have Ant and the like...

Of the remainder, I really only think #7 has much in the way of merit. Have you looked into the java.nio package? This makes working with binary data much simpler...

Do NOT INSTALL THIS

[cowholio4]

BTW Do not install this crap... I do software development and this program has made it impossible to compile/run my programs (even after I uninstalled it) and also while it was running it would not allow Eclipse to run. So basically this program screwed me over big time. I am writing a database that is is to be deployed tomorrow morning. **&**&**&*&* ..... Note to self: Do this crap in a virtual machine and not while developing a program.

It's the programmers fault?

[X-Phile]

I don't think it's typically the programmers fault that there are security issues with the software. If the programmer was taught how to do things properly, then they would do things properly. Also, if they weren't so rushed to get a product out the door, they would be able to do a proper review and test of the code and find a majority of the bugs before the product hits the streets (or the server room in the case of custom software)

Typically, a programmer is doing their job. The programmers manager is doing their job, by squeezing the work and deadlines of the programmers.

My $0.02 CDN

Speed - buffer overflow?

[BagOCrap]

Still, programmers are privileging speed and efficiency over security, which leads to the famous "buffer overflows"... Am I the only one who finds the above sentence just strange? In my books and experience, speed optimisations most certainly don't result in buffer overflows. Recklessness does, however.

Why the restriction to "non-commercial" use?

[grandpa-geek]

Why not just license under the GPL, LGPL or some other open source license? This business of being "free for non-commercial use" restricts users who use open source software for commercial purposes. This software is really "non-free" according to any definition of the FSF or Open Source Initiative, which explicitly forbid discrimination against fields of endeavor. Perhaps you should say "non-free, but gratis for non-commercial use."