Heartbleed OpenSSL Vulnerability: A Technical Remediation
An anonymous reader writes "Since the announcement malicious actors have been leaking software library data and using one of the several provided PoC codes to attack the massive amount of services available on the internet. One of the more complicated issues is that the OpenSSL patches were not in-line with the upstream of large Linux flavors. We have had a opportunity to review the behavior of the exploit and have come up with the following IDS signatures to be deployed for detection."
Was this badly translated from another language, or have I been out of system administration too long?
We have to thank the security researchers that chose to break the embargo on the news before OpenSSL coordinated with downstream project.
Thank you for the mess, guys!
Someone knows if OpenVPN is affected? The tests do not work on it since it uses TLS in an unusual way.
An IDS provides the means to detect malicious patterns in traffic. It is by no mean a remedy.
If it's using one of the affected versions, how did it get past the famed OpenBSD audits?
General Relativity: Space-time tells matter where to go; Matter tells space-time what shape to be.
NSA, heartbleed, whatever. you'll tell your grandchildren about "back in the day" internet.
It's amusing how much talk is going on about this. Patching the vulnerability is trivial. All of the major IPS and IDS products out there already have signatures published to remediate it for organizations who for whatever reason can't patch. This is getting silly.
I'm running Linux Mint Olivia -- the next to current version -- an no openssl patch is yet available as of this afternoon. I image there are quite a few similar distros. Since I have actual work to do, and can't risk wasting two hours on a potentially borked upgrade, I'm stuck to trying not to use programs affected by the exploit for the duration.
While something tells me this exploit is somewhat overblown, what really ticks me off is that this is all the result of delegating memory management to C pointers and basically mmap. As far as I'm concerned, in this day and age, that amounts to spaghetti code and I can't say it endears me to the reliability of openssl.
Please, we need SSL to be secure, not fast. Just use a less efficient method to make things more secure.
May the Maths Be with you!
PFsense VRT rules (paid version) already updated, I'm sure every maintained IDS has been updated for this.
"If any question why we died, Tell them because our fathers lied."
Uh, Olivia's been out of support for a couple of months, hasn't it?
There is well written C, and there is poorly written C. I've been through the bowels of OpenSSL, and there are parts of it that frighten me. Ninety percent of the issues in OpenSSL could be solved by adopting a modern coding style and using better static analysis. While static analysis tools can't find vulnerabilities, they can root out code smell that hides vulnerabilities. If, for instance, I followed the advice of two of the quality commercial static analyzers that I ran against the OpenSSL code base, I would have been forced to refactor the code in such a way that this bug would have either been obvious to anyone casually reviewing it, if the refactor did not eliminate the bug all together.
C and C++ are not necessarily the problem. It's true that higher level languages solve this particular kind of vulnerability, but they are not safe from other vulnerabilities. To solve problems like these, we need better coding style in critical open source projects.
Before anyone uses this as a attack against open source, then please remember the compromise of RSA by the NSA. That was a deliberate, wilful violation of trust with wide-reaching consequences.
Also, don't forget that little issue in the Apple source code recently...
BTW, is there any evidence this was a NSA job as opposed to just a screwup by a developer ?
I've now read that: "No versions of OS X or OS X Server are affected by the OpenSSL Heartbleed bug, because the last version of OpenSSL shipped by Apple in an OS was 0.9.8y, which is a branch not affected by this bug. So unless you've installed OpenSSL via MacPorts or Homebrew, your public-facing OS X servers/services should be immune to this bug." What say the wise ones here?
DaveyJJ
Like some site that is (like "what that site is running?" (Apache, IIS etc)) where we can see who gets what fixed when. No point in changing my passwords on a still-affected site.
Slashdot's rate-of-post filter: Preventing you from posting too many great ideas at once.
Oh, and PPC too.
Let the script kiddies do their best. Bwa ha ha ha ha
There have been a number of sites.
SSLLabs scanner has been updated to check for Heartbleed, and also will report when the cert validity starts (handy if you want to see whether they're using a new cert). https://www.ssllabs.com/ssltes...
LastPass has a pretty decent scanner that just focuses on Heartbleed (without all the other info that you get from SSLLabs): https://lastpass.com/heartblee...
There are some others out there as well, of course.
There's even one for client-side testing (almost as critical):
Pacemaker is an awesome little POC script (python 2.x) for testing whether a *client* is vulnerable (many that use OpenSSL are...). https://github.com/Lekensteyn/...
There's no place I could be, since I've found Serenity...
There are many organizations that not only can't patch, do not know how to patch, or simply haven't completed patching, but also don't _have_ an IPS or IDS in place. In fact, even if a company is in a position (and has the know-how) to install one, using either one of these options may come with what is perceived as an unacceptable performance impact.
I managed to write an exploit for this issue within about 30 minutes. The bug is almost trivial to exploit. In my meager tests, i gathered usernames, passwords, session cookies, and oauth2 client tokens from an unrelated location on the internet. So, yes, i'd say the issue leans a bit closer to mountain than molehill.
That said, the fix is also trivial, and the fact that several distributions still don't have updated packages is downright embarrassing.
The Canadian Revenue Agency has shut down online reporting of taxes. 80% of Canadians use this service and most corporations file electronically anyway. Get ready for a wave of paper heading for Sudbury!
http://www.cra-arc.gc.ca/menu-eng.html
unsigned int payload; /* Use minimum padding */ /* Read type and payload length first */
+ unsigned int padding = 16;
+
+-
+- hbtype = *p++;
+- n2s(p, payload);
if (1 + 2 + payload + 16 > s->s3->rrec.length) /* silently discard per RFC 6520 sec. 4 */
return 0;
should not that be written like this:
if (payload > s->s3->rrec.length) //! s->s3->rrec.length) /* silently discard per RFC 6520 sec. 4 */
return 0;
Taken from: http://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=96db902
Is bypassing/wrapping/whateverthey'redoing OS calls changing how memory is managed not a big enough change to warrant an entry in the 1.0.0h -> 1.0.1 log?
One of my current roles is to provide technical support/advice for a group of project managers and business analysts. This morning a few of them had watched the Crash News Network over breakfast and came in convinced that privacy, as we know it, had come to an end. My job is to talk them off the ledge (and I actually enjoy it, they're smart people and as long as I explain it correctly, they get it... I've found that's pretty rare).
1. The issue only exposes 64k at a time. Let's assume that the average enterprise application has at least a 1G footprint (and that's actually on the low end of most applications I work with). That's 1,048,576K. At best, this means that this exploit can access 0.006% of memory of an applications memory at one time.
Ahh you say, I will simple make 16,667 requests and I will retrieve all the memory used by the application.
2. The entire basis of this issue is that programs reuse memory blocks. The function loadAllSecrects may allocate a 64k block, free it and then that same block is used by the heartbeat code in question. However, this code will also release this same block which means that the block is free for use again. Chances are very good (with well optimized code), that the heartbeat will be issued the same 64k block of memory on the next call. Multi-threaded/multi-client apps perturb this but the upshot is that it's NOT possible to directly walk all of the memory used by an application with this exploit. You can make a bazillion calls and you will never get the entire memory space back. (You're thinking of arguments to contrary, your wrong... you wont.)
Congratulations, much success... you have 64k internet.
3. Can you please tell me where the passwords are in this memory dump:
k/IsZAEZFgZueWNuZXQxFzAVBgNVBAMTDk5ZQ05FVC1ST09ULUNBMB4XDTEwMDMw
MzIyNTUyOFoXDTIwMDMwMzIyMTAwNVowMDEWMBQGCgmSJomT8ixkARkWBm55Y25l
There will be contextual clues (obvious email addresses, usernames, etc) but unless you know the structure of the data, a lot of time will be spent with brute force deciphering. Even if you knew for a fact that they were using Java 7 build 51 and Bouncy Castle 1.50, you still don't know if the data you pulled down is using a BC data structure or a custom defined one and you aren't sure where the boundaries start and end. The fact that data structures may or may not be contiguous complicates matters. A Java List does not have to store all members consecutively or on set boundaries (by design, this is what distinguishes it from a Vector).
Long story short. Yes, there is a weakness here. However, it's very hard to _practically_ exploit... especially on a large scale (no one is going to use this to walk away with the passwords for every gmail account... they'd be very, very lucky to pull a few dozen).
This doesn't excuse developers from proper programming practices. It's just putting "Heartbleed" in perspective.
While the proof of concept exploit used an unencrypted attack, the vulnerability can still be exploited AFTER the session is encrypted.
Since the IDS probably cannot decrypt the SSL connection... it is unlikely to detect an attack that occured after encryption was negotiated, and the extension message is invisible to the IDS
But you know the vulnerable host is running OpenSSL 1.0.1 -> 1.0.1f, so you can look at the source code to figure out what the memory around the private key is supposed to look like.
What is the point of IDS? If you detect an attack, your private keys are compromised and the game is over.
And then you try to recover, you make new keys, renew certificates, revoke the old one... but since certificate revocation is quite broken, you never recover. An attacker that stole your old private key will still be able to masquerade as the legitimate server.
Follow the proposed specification at http://heartbleedheader.com to tell your users when you've patched your servers. This eliminates the guessing: "is it OK to update my password now? Do I even need to? Can I trust that I'm not being MITMed with their old SSL key that an attacker stole?" It's bad enough using the tools at hand to detect that information from a single site, let alone the hundreds you might have in your password manager.
Dewey, what part of this looks like authorities should be involved?
This guy has retracted part of his analysis based on comments, but tries to make a case that passwords and cookies in the http headers are more likely to be exposed than keys. Remember, http-auth is still used a lot. http://blog.erratasec.com/2014...
I'm not sure where all the frenetic press came from. Surely there have been other vulns that are more severe? Like maybe one of these?
"First they came for the slanderers and i said nothing."
Can you please tell me where the passwords are in this memory dump ...
Have you ever seen a real exploited piece of data?
These are taken from Yahoo production servers, a day or two ago:
http://cdn.arstechnica.net/wp-...
http://cdn.arstechnica.net/wp-...
Can you guess where the password is, now? (And those didn't even take that many tries)
I have not seen actual SSL private keys floating around just yet, but given that the original researchers said they managed to get private keys from their own servers, I think it is reasonable to assume that some production servers must have already leaked them.
If OpenSSL is (as quite a few people who know what they are talking about have claimed) poorly written and hard to maintain, why no-one has tried to come up with a simple, easy to evaluate solution.
Or is SSL/TLS really that hard to properly implement?
I'd love to read your original comment but I can't because this beta bullshit doesn't have any parent links. Fuck you, beta!
Suppose SourceForge is/was vulnerable (I don't know that that's the case...I opened a ticket to find out).
Suppose a developer's login credentials were grabbed before SourceForge reacted and closed the hole.
Great. Now a bad character can upload malware as the latest release for any of the compromised developer's SourceForge projects.
Yeah...chew on that.
I looked at the code. The design is AMAZINGLY bad. I think it's probably a deliberately bad design that was created to hide the exploit. Probably the feature itself was designed for the exploit, but the exploit was kept out of the first version of it so that the code wouldn't be under scrutiny when the exploit was added.
I read that that part of the protocol (the ping - probably a custom add-on for open ssl) was designed by the same guy who implemented it.
It's a ping which:
1) is an unnecessary feature in itself.
2) giving it a payload is also unnecessary
3) allowing that payload to be longer than necessary to stamp, number and keep track of packets is malicious. 1 byte would be enough to be useful (if hard to use) 8 bytes should be more than enough for anyone! 65530 some odd bytes max? WTF? Who in their right mind allows that in a ping? It's designed as AT LEAST a secret side channel!
4)giving that payload a "size" field that's redundant (since the packet HAS a size) is malicious and allowed the programmer to trust a bullshit size field instead of using the actual size. And there is the anatomy of snuck in exploit - make a redundant field which should never be trusted or used - then pretend to trust it. God, no programmer in their right mind!
If anyone is in charge of openssl they should exclude the guy who wrote that and rewrite everything he's touched.
Or, Openssl should be considered trash and replaced in every package.
Every secret ssl key is vulnerable. A lot of sites won't buy new ones till their current ones expire. How long does that take, a year, a few years?
And until then, you don't necessarily have any security, even if you change your passwords. And if every system on the internet is vulnerable for the next six months, then nothing will be trustably secure for decades to come. Everything will have possible back doors, your old and new passwords possibly known etc. etc. etc.
none of those are even in the same ballpark of criticality compared to this vulnerability. All of them combined don't even come close.
http://slashdot.org/firehose.p...
Note: I am not the author of the following quote, this is a copy/paste.
http://www.reddit.com/r/h1z1
http://www.reddit.com/r/h1z1/c...
"Hi there,
I wanted to tell you about an exciting new free-to-play game we've had under wraps here at SOE for some time. It's called H1Z1. It's a massively multiplayer game in which players fight for survival in a world where death is the only sure thing. The H1Z1 virus devastated mankind and left nothing but death and destruction in its wake and a world nearly empty of human life where the remnants of humanity are in a fight against extinction against those infected with the virus. It's been 15 years since H1Z1 was first encountered and what's left of the world before is overrun with the Infected. Humanity has been reduced to hiding in the shadows, searching desperately for food and water and anything that can help to survive even for another day. But the Infected aren't the only dangers in the world. Everyday life in the Apocalypse means dealing with all kinds of wild animals and the brutality of other survivors, as well as finding your next meal and a safe place to sleep. It also means scavenging or crafting anything that can help you live just one more day. In H1Z1 every minute of every day is borrowed time and fearing for your life.unless you are the Danger (talking to you Walter), but life can and will go on.even in circumstances as dire as this. Humanity has not given in to the Infected. There are still pockets of humanity and the fight goes on!
Our vision for this game is very simple but ambitious. We are starting with what I would call "Middle America" - an "anywhere and everywhere" town. The world is massive as you've come to expect from our games. Over time we will grow the world until we have our own version of the U.S. after the death and destruction brought on during the H1Z1 epidemic. It will be our own version of America. We'll have urban cities and desolate wide open places. All connected seamlessly. Our focus is building a sandbox style of gameplay where players can build shelters out of resources in the world. They can even work together to make amazing fortresses complete with weaponry to help defend against both the Infected and other players. Players also have access to a very deep crafting system that can let players make a huge variety of awesome stuff, including weapons (I made a 1911 the other day) and things like Molotov cocktails, explosives.. and other fun surprises.
I will also go right to the heart of the question a lot of players will have - "There are a lot of survival / Zombie games.how is this one going to be any different?". First off, it's a persistent MMO that can hold thousands of players on servers we host (yes there will be multiple servers with very different rule sets). Why is that a good thing? It means a thriving economy (oh yes.there's trading). It also means you have potential allies in the all-out war on the Infected... and many an enemy as well. It uses our proprietary next-gen Forgelight engine and that means we've had a lot of really cool technology to work with to make the game we wanted to make. It's also designed from the ground up for our players to become part of the design process. The Roadmap system that we built for PlanetSide 2 will be used extensively to clearly communicate what features we're working on and what you can expect and when. You're also going to be getting awesome access to our developers. We'll be opening it up for Player Studio creations too so expect player-created items to make their way into the game. The main thing that differentiates H1Z1 from the other great games in the genre is the emphasis we are putting on player ownership and building. We want you to be able to form roving gangs that are headquartere
Forgot to write the source for the images: http://arstechnica.com/security/2014/04/critical-crypto-bug-exposes-yahoo-mail-passwords-russian-roulette-style/
What really needs to be done with certs? Do they really need to be revoked and reissued? Must the re-issued cert have a new issue date? At least one service provider I work with has claimed they somehow reissued their certificate, but it has the same, old issue date and a different signature. Is this enough?
http://en.wikipedia.org/wiki/Path_MTU_Discovery
Path MTU Discovery.
The idea is that, for speed and reliability purposes, you find the largest packet size that won't fragment in your path by sending larger and larger packets with "don't fragment" on...
The Heartbleed security bug leaks private keys and certificates for https sites (along with other stuff, such as user/password pairs for users of those sites).
Thus, if these certificates are not REVOKED, and these revocation certificates are not DISTRIBUTED to everyone, they can be used to impersionate the site.
This includes even EV certificates.
It is *not* feasible to revoke so many certificates, at all. It is downright impossible except by using OCSP + stapling, which is not only NOT enabled by default in the vast majority of the browsers, it would also cause severe privacy issues (OCSP servers know what sites you access) and operational issues (OCSP servers going down because of the load).
Therefore, the CA certificates themselves will have to be revoked (and sent to the browser vendors to be included in the internal revocation list), which means *EVERY* certificate still valid for that CA, even those of servers that were not affected, will have to be reissued. Either that, or worse, the very CA root certificates have to be replaced, invalidating just about every certificate that CA ever signed, including other CA's. These certificates have NOT been compromised, but replacing them might be the only way to revoke the millions of certificates that might have been compromised.
all the more reason to have encrypted memory
Donald 'Duck' Dunn: We had a band powerful enough to turn goat piss into gasoline.
So two yahoo accounts were affected?
You can auto-detect key material on any memory dump. You look for binary and base-64-encoded regions of high entropy. This also flags compressed data, but that's very rarely present on memory dumps. You will also find truly random data (as in random numbers used for crypto), but keys usually have internal structure, so it is not too difficult to separate both automatically.
Perhaps today is a good day to run IIS!
Oh snap, tell OP to apply lotion to the burn...
This particular bug may not have been as slam dunk bad as originally published, but for this question, assume that it was.
As a point of public debate, do we want the NSA to
a) Find and exploit these sorts of bugs?
b) Find and fix these sorts of bugs?
There is an argument that each behavior makes us 'safer'.
Plan a may catch a few bad bad guys, but leaves us open to exploits from other bad guys.
This makes doing anything important online trickey.
This will affect daily life.
Plan b probably closes an important window into what some bad folks might do.
This might prevent another 9-11, but the cost seems very high.
It's not fair to ask the NSA to do both.
It's a tough choice, but I think the wise choice is to vote for plan b.
...if Microsoft has released something with a bug like this. Somehow I doubt there'd be so much analysis. To me this demonstrates that open != secure necessarily; how long has anyone been able to read the source-code here and even known about it for months? Let's just all agree that open-sourcing code is no guarantee of any kind of quality.
throw new NoSignatureException();
The only way to get private keys is to attack a BSD system before anyone else makes a request. At least that's the only report I've found that was successful. Not likely to be practical in the real world.
"First they came for the slanderers and i said nothing."
That's only true if you believe company's secret keys are all being compromised, which they aren't.
"First they came for the slanderers and i said nothing."
my "unwilling to wade in the bulls" take on this affair is that some part of ssl on the outward face of that service is bleeding large chunks of raw memory, in response to a trivial attack.
i'm not clear on on is which parts of memory are bleeding..
only ssl services memory?
or all memory from all services in the vicinity?
or all memory in general?
i'm hearing that sll privates are amonst the things that are leaking, but can anyone please clearly define which parts of my various systems have been wide open to the world, these past two years and more?
some people are saying ssh is not affected..
this is important to me - i don't care about ssl, and consider the bulk of the applications it supports to be trivial, but i cannot work without ssh, and am not aware of any viable alternatives.
i've always assumed that ssh was built on top of ssl - as in why reinvent the wheel - but when i went digging, trying to sort out which versions i have been running, and what i need to update, etc, i seem to be finding that ssh is actually standalone, apparently?
perhaps built on similar principles, but by differerent groups, and never merged - or do i have a bent picture?
apparently i need to learn more about these things - in matters like this i prefer to be able to read the source than listen to pundits pound the keys..
i am one, so i know, don't argue with me.
anyway - hello - if anyone can please enlighten or point at the histories of both open ssl and ssh for me, it would be a great help,
and if anyone could please point at, or repeat the answer to my question about what exactly has been exposed - thank you.
https://www.robin-seggelmann.de/
No, you actually have to fix the code to add bounds checking, or download a new version of OpenSSL (which probably gets you other fixes as well, unless you were already running the latest version.)
Recompiling OpenSSL with the proper flag isn't enough to do the job - there are people who've done that and had problems keeping OpenSSL stable on their platforms, and more importantly, that still doesn't stop the Heartbleed attack from causing trouble. You need to get the code not to try to fetch memory beyond the appropriate object's array bounds, though OpenSSL should also default to using malloc()/free() instead of rolling its own badly.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
After updating my openssl in Gentoo Linux, I noticed that there is a USE flag for 'tls-heartbeat'. So, I have now removed that use flag from all of my Gentoo installs.
Is there any useful reason to keep heartbeat around?
Ops, I shuld have usd the prevuwe but in.
"SNAFU is particularly FUBAR",
what poetry!!!!
We all know what FUBAR means, and as a standard form Spanish verb it is so handy:
FUBO - I am ...
FUBAS - YOU are ...
FUBA - YOU, sir, are ...
FUBAMOS - WE are ...
FUBAN - THEY are ...
Damn. No mod points. Somebody mod the parent post up please. This is pertinent and informative.
friends don't let friends teleport drunk