Slashdot Mirror
Theo De Raadt's Small Rant On OpenSSL
New submitter raides (881987) writes "Theo De Raadt has been on a better roll as of late. Since his rant about FreeBSD playing catch up, he has something to say about OpenSSL. It is worth the 5 second read because it is how a few thousand of us feel about the whole thing and the stupidity that caused this panic."
Update: 04/10 15:20 GMT by U L : Reader badger.foo pointed out Ted Unangst (the Ted in the mailing list post) wrote two posts on the issue: "heartbleed vs malloc.conf and "analysis of openssl freelist reuse" for those seeking more detail.
Years ago the BSD guys added safeguards to malloc and mmap, but they were disabled for all platforms in OpenSSL only because they caused performance problems on some platforms. He finishes by saying that OpenSSL is not developed by a responsible team.
Theo De Raadt is a noteworthy personality, same as Linus Torvalds or Bill Gates. Their comments on important matters are important.
heartbleed vs malloc.conf
and
analysis of openssl freelist reuse. Short articles with a lot of good information.
-- That grumpy BSD guy - http://bsdly.blogspot.com/
If your intermediate certificate's signing keys are on the internet facing web servers you're doing it wrong. That intermediate signing key should be treated with the same level of security you would treat a root key with.
It is good practice to sign against an intermediate certificate. That way if it is compromised you can reject it and issue a new intermediate certificate signed by your root certificate. You can push the new certificates as updates since they would be validated against the root certificate.
You need to read up on authenticating the entire chain of certificates.
Why OpenSSL is so popular? It has FIPS-certified module, and this becomes important for selling your product to the government.
So what could be done to prevent something like this from happening in the future? People will keep writing bad code, this is unavoidable, but what automated tests could be run to make sure to avoid the worst of it? Someone with direct development experience please educate the rest of us.
As much as Theo can be an utter and insufferable prick, on this score he's right. This was an insanely trivial error which has exposed who knows how many systems to potential breaches. Right now I'm starting up a full audit of our systems. We use OpenVPN for our interoffice WAN, as well as for clients; many of them Windows, iOS and Android clients, not to mention reviewing all our *nix clients running SSH daemons. We're only a relatively small operation, and it's still a monumental pain in the ass.
The world's burning. Moped Jesus spotted on I50. Details at 11.
That's all true and correct. When you do that, though, you need to do at least as good a job as what you're circumventing. In this case OpenSSL didn't.
The trick is not to read TFA in the first place. Also, you must be new here.
May the Maths Be with you!
That code is almost a text book example of material that is submitted to the Underhanded C contest...
http://en.wikipedia.org/wiki/Underhanded_C_Contest
My understanding is Theo said: Developers on a security product made a conscious decision to turn off last line of defence security for all platforms in the interest of performance on some systems. That does not sound like and unfortunate error to me, it sounds outright irresponsible.
I don't get what you're saying, and I think that's probably because you don't know what you're talking about. Having certificate chains is only a plus, the flat structure was crap. Here's how it works:
I have a root certificate that's universally trusted. It is used *only* to sign intermediate certificates. Having the public cert in the public is fine since it only contains the public key part of the asymmetric public/private key pair. The private key, sits on a server which is physically isolated from the world. By that, I mean that the root certificate and private keys are literally on servers with *no* network connections. When you want to generate a new intermediate certificate, you put the CSR on a USB stick, plug it in, and sign it from that machine. In this way you never have to worry about external threats potentially gaining access to your private key (internal are an ever constant threat, and you put in good safeguards to prevent against that).
Now that you have a chained hierarchy, you can use different intermediates to sign different end user certificates. Remember that both the root and intermediate have their own certificate revocation lists: the root can revoke intermediates (which means anything signed by them is null and void) and intermediates can revoke server or subordinate intermediate certs.
As a result of my chained hierarchy, if an intermediate is compromised, I can revoke it, without revoking every single end user / server certificate out there. This gives me finer grained control.
Now, I said that the root isn't even connected to the internet. The intermediate ideally is not either. Ideally the user / server signing intermediate is behind a set of firewalls, and *pulls* signing requests from frontends, that it then processes and posts the resulting signed certs to. That way if you compromise the front end, you have to the compromise the firewall (which ideally is blocking all inbound connection requests / unconnected sockets / anything that is not communication required for the intermediate server(s) to send pulls to the frontends) in order to get to the intermediate.
Your flat view of the world is draconian, wrong, uneducated, and probably hurts everyone who reads it by making them a little less educated.
Theo's point isn't that OpenBSD users would have been safe. It's that had OpenSSL crashed on OpenBSD (or any OS with similar mitigation in place) it would have surfaced the bug much sooner...perhaps before a worldwide release. Once found it would have been fixed and merged upstream to benefit all users.
This is really a specific case of the larger point behind avoiding monoculture, whether OS or hardware. OpenBSD continues to support older architectures in part because it forces them to work through issues that benefit other platforms, especially the one most of us use daily: x86.
In addition, the mitigation countermeasures also prevent memory debuggers like Valgrind from finding the problem (Valgrind find use-before-init for malloc'ed blocks, but not if there is a wrapper in between that re-uses blocks), and may also neutralize code-security scanners like Fortify.
I have to admit that while my original intuition was "screwup", this looks more and more like some parts of the OpenSSL team have been compromised and did things that make this kind of bug far more likely. Like their own insecure memory allocation. Like not requiring time-of-use boundary checks or having any secure coding guidelines in the first place. Like documenting everything badly so potential reviewers get turned away. Like not having working review for patched or a working fuzz-testing set-up (which would have found bug this easily).
After all, the NSA does not have to sabotage FOSS crypto software. They just have to make sure the quality is low enough. The bugs they can exploit will follow. And the current mess is just a plain classic. Do enough things wrong and eventually stuff breaks spectacularly.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Oh, and read this: http://www.tedunangst.com/flak/post/analysis-of-openssl-freelist-reuse
In effect at some points OpenSSL does:
free (rec);
rec = malloc (...);
and assumes that rec is the same.
Eeew,
Watch this Heartland Institute video
Well, what you are pointing out is that a CA is a single point of failure -- Something actual security conscious engineers avoid like the plague. What you may not realize is that collectively the entire CA system is compromised by ANY ONE of those single points of failure because any CA can create a cert for ANY domain without the domain owner's permission. See also: The Diginotar Debacle.
The thing is, nobody actually checks the the cert chain, AND there's really no way to do so. How do I know if my email provider switched from Verisign to DigiCert? I don't, and there's no way to find out that's not susceptible to the same MITM attack.
So, let's take a step back for a second. Symmetric stream ciphers need a key. If you have a password as the key then you need to transmit that key back and forth without anyone knowing what it is. You have to transmit the secret, and that's where Public Key Crypto comes in, however it doesn't authenticate the identity of the endpoints, that's what the CA system is supposed to do. Don't you see? All this CA PKI system is just moving the problem of sharing a secret from being the password, to being which cert the endpoint is using -- That becomes the essential "secret" you need to know, and it's far less entropy than a passphrase!
At this time I would like to point out that if we ONLY used public key crypto between an client and server to establish a shared secret upon account creation, then we could use a minor tweak to the existing HTTP Auth Hashed Message Authentication Code (HMAC) proof of knowledge protocol (whereby one endpoint provides a nonce, then the nonce is HMAC'd with the passphrase and the unique-per-session resultant hash provides proof that the endpoints know the same secret without revealing it) to secure all the connections quite simply: Server and client exchange Nonces & available protocols for negotiation, the nonces are concatenated and HMAC'd with the shared secret stored at both ends, then fed to your key-stretching / key expansion system AND THAT KEYS THE SYMMETRIC STREAM CIPHER SIMULTANEOUSLY AT BOTH ENDS so the connection proceeds immediately with the efficient symmetric encryption without any PKI CA system required.
PKI doesn't really authenticate the endpoint, it just obfuscates the fact that it doesn't by going through the motions and pretending to do so. It's a security theater. SSL/TLS and PKI are essentially the Emperor's New Secure Clothes. At least with the shared secret model I mention above, there's just that one-time small window of PK crypto for secret exchange at worst (failing to intercept account creation means no MITM) and at best you would actually have the CHANCE to go exchange your secret key out of band -- Visit your bank in person and exchange the passphrase, etc. then NO MITM could intercept the data. HTTP Auth asks for the password in a native browser dialog BEFORE showing you any page to login (and it could remember the PW in a list, or even generate them via hashing the domain name with a master PW and some salt so you could have one password for the entire Internet). That's how ALL security should work, it ALL relies on a shared secret, so you want the MOST entropic keyspace not the least entropic selection (which CA did they use). If you're typing a password into a form field on a web page, it's ALREADY game over.
Do this: Check the root certs in your browser. For Firefox > Preferences > Advanced > Certificates > View. See that CNNIC one? What about the Hong Kong Post? Those are Known bad actors that your country is probably at cyber war with, and THEY ARE TRUSTED ROOTS IN YOUR FUCKING BROWSER?! Not to mention all the other Russian ones or Turkish, etc. ones that are on the USA's official "enemy" list. Now, ANY of those can pretend to be whatever domain's CA they want, and if your traffic bounces through their neck of the woods they can MITM you and you'll be n
but if your explicit reason for overriding the kernel is to do something as-good-or-better than the kernel, and you fail to achieve that (let alone validate that you've achieved it), that's a rather spectacular failure. mistakes are common, sure. but this is sort of like saying the seatbelts that came with your car suck (they very well might), so you replace them with something "better", that ends up being little more than a piece of string that you daintily lay across your lap. kind of "unreasonable", no...?
As I read his analysis, OpenSSL relies on releasing a buffer, reallocating it, and getting the PREVIOUS contents of that buffer back -- or else it will abort the connection. (Search for the string "On line 1059, we find a call to ssl3_release_read_buffer after we have read the header, which will free the current buffer." in his article referenced by the parent post).
Now, IMO, this goes way beyond sloppy. Releasing a buffer before you're done with it, and relying on a wacky LIFO reallocation scheme giving you back that very same buffer so you can process it, is either 1) an utterly incompetent coding blunder that just happened to work when combined with an utterly terrible, insecure custom allocation scheme, or 2) specifically designed to ensure that this insecure combination is widely deployed to provide a custom-made back door, as it works only with the leaky custom allocator.
If 1), then I must agree with Theo that the OpenSSL team were indeed irresponsible, since at least one of these two cooperating blunders ought to have shown up in a decent security audit of the code, and any decent set of security-oriented coding standards would forbid them both.
If 2), then it was deliberate, and the tinfoil-hat crowd is right for once.
"My strength is as the strength of ten men, for I am wired to the eyeballs on espresso."
Theo De Raadt makes comments about Linux being for losers. Linus Torvalds makes comments about OpenBSD users being masturbating monkeys. You usually have to take some of their comments with a grain of salt.
He said, prophesying a future Ubuntu release name.
It must have been something you assimilated. . . .
Theo's "rant" isn't brought about by a typical bug in the sense of a mistake in the code. His rant is brought about by the fact that OpenSSL deliberately introduced wrapper functions for malloc/free, making it impossible for a system to provide hardened ones, then made those wrappers default. He further rants about the fact that, because it's default, they later introduced a bug that means you can't turn the wrappers off.
The heartbeat bug itself was a mistake and a bug in the traditional sense. The "hey let's replace malloc/free" is much closer to "bad decision" than "mistake."