First, there is no magic solution to this. IT IS A LOT OF WORK. Also, my company predominately develops software for non-Linux platforms, so I'm not going to recommend any Linux specific tools.
I recommend the following (by no means a complete list):
1. Fuzz testing
Fuzz testing is throwing directed yet random inputs at a program to see how it fails. Extremely long strings, null terminated strings, invalid files, files that are "almost" valid, etc. It's good for security but it also helps reliability. Even if all your input comes from trusted sources, by protecting against invalid input you also protect against bugs in these sources.
2. Dynamic Analysis Tools
There's a wealth of tools that'll simulate disk-read errors, out of memory errors, and other failures like this. Even if you expect to always have enough memory, OOM conditions may happen even temporarily. Tools like AppVerifier help detect heap buffer overruns, underruns, and bad API usage. Run your test passes under tools like these.
3. Static Analysis
There's a host of tools which can analyze source code and look for problems. Run these as often as possible and fix all the issues which come up. If they are quick to run, make a clean run checkin requirement.
4. Establish a feedback loop
Even with the strictest coding standards, strict testing, and excellent tools, crashes will happen. Eventually, your code will run in an unexpected environment, some external influence on the program will corrupt its environment, or some maintenance coder two years down the road will checkin a "fix" that introduces a crashing regression for some customers. Have someway for your customers to send you dump files whenever a crash does happen. If you happen to support Windows, this is really easy. Microsoft has a site for getting access to all the crash data that the customer would send for your product. Establishing an account is free (as in beer), but does require you to provide a VeriSign ID to establish identity, so noone else will try to get at your data. My company uses this, and it allows us to focus on the top N crashes that occur in our products so we get the most bang for our bug.
Even if you do all of the above, there will still be some crashes in the product.
What not to do:
1. Swallow all exceptions
This'll make your code appear more stable on the surface, but by blindly swallowing exceptions you are forcing your code to operate in a state you never designed for. All you really do is turn an easy to diagnose crash into an impossible to diagnose crash, or worse, a bug that just results in silent data corruption.
2. Believe that using library x/Java/.Net/STL/etc. will fix your problems
All of the above are just tools, but it is still possible to have crashes even if you use these tools 100%. An OOM exception in any of the above is more graceful and more recoverable than an access violation, but you're still going to have to do a lot of work to make sure you eradicate the sources of exceptions in your code as well as make sure the exceptions you do expect and can recover from you can actually rollback/retry/etc. to leave your data in a valid state.
I could have sworn that when a 386 (and to some extent a 286) was running in protected mode, different areas of memory could be marked as 'code' for execution and for 'data' that could not be executed.
Sort of. You could mark entire segments as being no-execute, but not individual pages. The segmented memory model isn't as flexible as a flat, paged model, and all Win32 OS's use a flat paged model with the code segment, data segment, and stack segment being equal.
Indirect calls through a function pointer shouldn't affect branch prediction too much.
First, these methods will always be done with a call instruction on x86 and a non-conditional jump on other architectures. The CPU will immediately predict this jump as "always taken."
What can be problemetic is the branch target. The CPU probably can't speculatively execute the instructions at the branch target at least until the load of the function pointer has been completed. To get around this, many CPU's use a branch target buffer. When the branch instruction is decoded, a cached copy of the branch target will be fetched and the instructions at that address will be executed. If which virtual method is executed is fairly random, this will quickly be a very bad predictor, but this is true of any branch prediction algorithm.
They're not saying it's OK for attackers to cross the old line. They merely want to have additional layers of security in case the attackers manage to cross the first line.
For example, my house has a lock on the front door. If someone manages to break through that lock, I'm not just going to say, ok, have your run of the house. The alarm system will go off also. If they manage to get past the alarm system (because I, or someone else forgot to enable it, or they figured out the code) they still don't get everything for free. Irreplacable items are kept in a safe. If they crack the safe, they still can't get a large chunk of my money since I store that in a different location.
Putting your entire trust in one layer isn't very good security.
I don't know why I respond to idiotic comments like this, but what the hell.
calloc() does this. malloc() will not zero the memory out. Some operating systems that have security certifications will just fill with random bits.
calloc() and malloc() are functions in a user-mode library. When they run out of space on their heap, they request more memory from the operating system. In Windows NT, this is with the VirtualAlloc call.
No, it hasn't always done this. I wrote a page-dumper for NT that would print out every string in memory from start to finish. It was user space, didn't run as administrator.
You successfully dumped every string in your process. If you are more clever than I think you, you might have dumped memory for every other process created by you by posing as a debugger. Big deal. You did not get memory from another user memory space. If you think you did, please provide the code!
Uhm, do you know what virtual memory is?
Do you have any clue how memory management works at any level in the operating system?
Any operating system with some sort of security certification will write zeros to a page before giving it to the application. Windows NT has always done this. If not, all an app would have to do to look at another process's memory space would be to keep allocating pages. The OS usually bypasses this for kernel-mode allocations (what's the point?), which is why there was a vulnerability discovered recently in ethernet drivers that didn't bother zero'ing their memory blocks before sending the blocks out over the wire.
malloc() takes chunks of the application's virtual memory that has already been allocated by the operating system and paritions it outs. It won't bother zeroing the memory for performance reasons.
But, isn't one of those situations he mentions in the interview (namely, running a large database server) what this chip is designed to be doing?
Sure, but it doesn't really do it significantly better than some of the more common RISC architectures (Sparc, Power, Alpha), and it's a lot more expensive.
Do you have any data to back that up?
According to this press release, the current record for the TPC-Cbenchmark on a 32-way machine is held by an NEC server with Itanium 2's.
An attacker with only local access to the machine and a sledgehammer is capable of launching a permanent denial of service attack on the box.
I know for a fact this works with Windows XP, but I presume this vulnerability exists in other OS's.
Have you even looked the IA64 ISA? It is a significant departure from x86. IA64 supports x86 instructions through emulation only (which is why x86 perf on IA64 is lagging.) It is not a 64-bit extension of IA32, which is an extension of IA16.
With IA64, there are 128 integer and floating point registers, 64 1-bit predication registers, eight branch registers, instructions are fixed length, every instruction can be predicated, speculative execution is supported at the instruction level, registers are preserved across function calls with register stacks, and register rotation can help prevent explicitly prevent antidependencies in tight instruction loops. Each of these is a departure from x86
I'm not too sure of this fact (and I am too lazy to double check) but I'm pretty sure IA64 requires a 64-bit operating system to even boot, unlike x86 which boots to 16-bit real-mode, and then is switched to 32-bit protected mode by the OS.
So what if the key can be cracked by groups of people with clusters of machines, lots of memory, and no day job, within a few months (or less)?
What matters, is that someone who steals the hard drive cannot read it, and will most likely not have the CPU cycles to crack it. Meaning, the thief that steals your bag can't go browsing through your credit card numbers, or your wife can't see that you plan on filing for divorce.
Of course, if you're want to transport extremely sensitive information (i.e., information that these people with the means to crack the key might want) you shouldn't depend on this hard drive, but that's obvious.
Slashdot Mirror
First, there is no magic solution to this. IT IS A LOT OF WORK. Also, my company predominately develops software for non-Linux platforms, so I'm not going to recommend any Linux specific tools.
I recommend the following (by no means a complete list):
1. Fuzz testing
Fuzz testing is throwing directed yet random inputs at a program to see how it fails. Extremely long strings, null terminated strings, invalid files, files that are "almost" valid, etc. It's good for security but it also helps reliability. Even if all your input comes from trusted sources, by protecting against invalid input you also protect against bugs in these sources.
2. Dynamic Analysis Tools
There's a wealth of tools that'll simulate disk-read errors, out of memory errors, and other failures like this. Even if you expect to always have enough memory, OOM conditions may happen even temporarily. Tools like AppVerifier help detect heap buffer overruns, underruns, and bad API usage. Run your test passes under tools like these.
3. Static Analysis
There's a host of tools which can analyze source code and look for problems. Run these as often as possible and fix all the issues which come up. If they are quick to run, make a clean run checkin requirement.
4. Establish a feedback loop
Even with the strictest coding standards, strict testing, and excellent tools, crashes will happen. Eventually, your code will run in an unexpected environment, some external influence on the program will corrupt its environment, or some maintenance coder two years down the road will checkin a "fix" that introduces a crashing regression for some customers. Have someway for your customers to send you dump files whenever a crash does happen. If you happen to support Windows, this is really easy. Microsoft has a site for getting access to all the crash data that the customer would send for your product. Establishing an account is free (as in beer), but does require you to provide a VeriSign ID to establish identity, so noone else will try to get at your data. My company uses this, and it allows us to focus on the top N crashes that occur in our products so we get the most bang for our bug.
Even if you do all of the above, there will still be some crashes in the product.
What not to do:
1. Swallow all exceptions
This'll make your code appear more stable on the surface, but by blindly swallowing exceptions you are forcing your code to operate in a state you never designed for. All you really do is turn an easy to diagnose crash into an impossible to diagnose crash, or worse, a bug that just results in silent data corruption.
2. Believe that using library x/Java/.Net/STL/etc. will fix your problems
All of the above are just tools, but it is still possible to have crashes even if you use these tools 100%. An OOM exception in any of the above is more graceful and more recoverable than an access violation, but you're still going to have to do a lot of work to make sure you eradicate the sources of exceptions in your code as well as make sure the exceptions you do expect and can recover from you can actually rollback/retry/etc. to leave your data in a valid state.
I could have sworn that when a 386 (and to some extent a 286) was running in protected mode, different areas of memory could be marked as 'code' for execution and for 'data' that could not be executed.
Sort of. You could mark entire segments as being no-execute, but not individual pages. The segmented memory model isn't as flexible as a flat, paged model, and all Win32 OS's use a flat paged model with the code segment, data segment, and stack segment being equal.
Indirect calls through a function pointer shouldn't affect branch prediction too much.
First, these methods will always be done with a call instruction on x86 and a non-conditional jump on other architectures. The CPU will immediately predict this jump as "always taken."
What can be problemetic is the branch target. The CPU probably can't speculatively execute the instructions at the branch target at least until the load of the function pointer has been completed. To get around this, many CPU's use a branch target buffer. When the branch instruction is decoded, a cached copy of the branch target will be fetched and the instructions at that address will be executed. If which virtual method is executed is fairly random, this will quickly be a very bad predictor, but this is true of any branch prediction algorithm.
They're not saying it's OK for attackers to cross the old line. They merely want to have additional layers of security in case the attackers manage to cross the first line.
For example, my house has a lock on the front door. If someone manages to break through that lock, I'm not just going to say, ok, have your run of the house. The alarm system will go off also. If they manage to get past the alarm system (because I, or someone else forgot to enable it, or they figured out the code) they still don't get everything for free. Irreplacable items are kept in a safe. If they crack the safe, they still can't get a large chunk of my money since I store that in a different location.
Putting your entire trust in one layer isn't very good security.
One of our customers is the U.S. military, who, surprisingly, doesn't like any backdoors or easter eggs in the software we deliver.
At my company, implementing a backdoor or an easter egg in shipping software is a terminable offense.
I don't know why I respond to idiotic comments like this, but what the hell.
calloc() does this. malloc() will not zero the memory out. Some operating systems that have security certifications will just fill with random bits.
calloc() and malloc() are functions in a user-mode library. When they run out of space on their heap, they request more memory from the operating system. In Windows NT, this is with the VirtualAlloc call.
No, it hasn't always done this. I wrote a page-dumper for NT that would print out every string in memory from start to finish. It was user space, didn't run as administrator.
You successfully dumped every string in your process. If you are more clever than I think you, you might have dumped memory for every other process created by you by posing as a debugger. Big deal. You did not get memory from another user memory space. If you think you did, please provide the code!
Uhm, do you know what virtual memory is?
Do you have any clue how memory management works at any level in the operating system?
Any operating system with some sort of security certification will write zeros to a page before giving it to the application. Windows NT has always done this. If not, all an app would have to do to look at another process's memory space would be to keep allocating pages. The OS usually bypasses this for kernel-mode allocations (what's the point?), which is why there was a vulnerability discovered recently in ethernet drivers that didn't bother zero'ing their memory blocks before sending the blocks out over the wire.
malloc() takes chunks of the application's virtual memory that has already been allocated by the operating system and paritions it outs. It won't bother zeroing the memory for performance reasons.
But, isn't one of those situations he mentions in the interview (namely, running a large database server) what this chip is designed to be doing?
Sure, but it doesn't really do it significantly better than some of the more common RISC architectures (Sparc, Power, Alpha), and it's a lot more expensive.
Do you have any data to back that up?
According to this press release, the current record for the TPC-Cbenchmark on a 32-way machine is held by an NEC server with Itanium 2's.
An attacker with only local access to the machine and a sledgehammer is capable of launching a permanent denial of service attack on the box.
I know for a fact this works with Windows XP, but I presume this vulnerability exists in other OS's.
Have you even looked the IA64 ISA? It is a significant departure from x86. IA64 supports x86 instructions through emulation only (which is why x86 perf on IA64 is lagging.) It is not a 64-bit extension of IA32, which is an extension of IA16.
With IA64, there are 128 integer and floating point registers, 64 1-bit predication registers, eight branch registers, instructions are fixed length, every instruction can be predicated, speculative execution is supported at the instruction level, registers are preserved across function calls with register stacks, and register rotation can help prevent explicitly prevent antidependencies in tight instruction loops. Each of these is a departure from x86
I'm not too sure of this fact (and I am too lazy to double check) but I'm pretty sure IA64 requires a 64-bit operating system to even boot, unlike x86 which boots to 16-bit real-mode, and then is switched to 32-bit protected mode by the OS.
So what if the key can be cracked by groups of people with clusters of machines, lots of memory, and no day job, within a few months (or less)?
What matters, is that someone who steals the hard drive cannot read it, and will most likely not have the CPU cycles to crack it. Meaning, the thief that steals your bag can't go browsing through your credit card numbers, or your wife can't see that you plan on filing for divorce.
Of course, if you're want to transport extremely sensitive information (i.e., information that these people with the means to crack the key might want) you shouldn't depend on this hard drive, but that's obvious.