The Fedora-Red Hat Crisis
jammag writes "When Linux journalist Bruce Byfield tried to dig for details about the security breach in Fedora's servers, a Red Hat publicist told him the official statement — written in non-informative corporate-speak — was all he would get. In the wake of Red Hat's tight-lipped handling of the breach, even Fedora's board was unhappy, as Byfield details. He concludes: 'If Red Hat, one of the epitomes of a successful FOSS-based business, can ignore FOSS when to do so is corporately convenient, then what chance do we have that other companies — especially publicly-traded ones — will act any better?'"
I pretty much agree: Fedora was obviously squelched by Red Hat corporate who was apparently afraid of the reaction of their paying customers. Despite the token board openings and motions about openness, after this nobody can pretend that Fedora is on anything but a *very* short leash held by Red Hat.
On the one hand, as a user I found myself trusting that Fedora's infrastructure crew were plugging away and probably handling things about as well as could be. On the other hand, the vague statements and lack of hard facts was (and still is) disturbing.
They should have come clean, and allowed the the community to vett their process.
Ob-FUD [just to poking Bruce for fun]: If they do come forth with details, it will be interesting to see if it was an ssh key compromised by the Debian flaw that caused this mess.
Thawte is Debian based. I wonder if they had a problem.
I checked our Thawte keys/certs against the SSL blacklist released by Debian. I checked several from Thawte, and could not find a potential compromised key/cert.
Also, we are a Red Hat customer. I have to agree, I prefer the way Debian handled their incident, versus the way this Red Hat incident is being handled. After reading the Red Hat Security Announcement the details are so vague, I am still not sure of the scope and reach of this vulnerability.
/^([Ss]ame [Bb]at (time, |channel.)){2}$/
This seems to be, from reading the Fedora and Red Hat statements, an ongoing investigation. The same way the police don't comment about investigations in progress, Red Hat is keeping mum. Keep in mind, the breach may be very complex and not something that they can confidently say "we understand" without a very detailed analysis.
They announced the issue immediately and took steps. For now, give them the benefit of the doubt that further details will be forthcoming once a proper investigation has been completed.
Learning HOW to think is more important than learning WHAT to think.
IT managers now know that RH is going to go unresponsive when there's a problem.
The issue isn't even fully known, so you're jumping to conclusions.
For some reason Fedora has to re-key all their repos and, while I think that's done, it's still being mirrored. One would assume a signing key has been lost.
Redhat isn't doing that. They apparently have a signing server, and a user's credentials were apparently lost, and some packages got signed, but not put in the repos. If you run a RedHat machine and get an unsolicited contact to install some new OpenSSH packages - don't.
I think Fedora has the bigger problem at the moment. Let them work through the problem, they know how to do this. When the users are safe (still an ongoing topic of discussion on how to best ensure this) my guess is they'll be releasing more information. I further suspect we'll learn that prior disclosure would have put users at more risk. We'll see.
How can they trust Red Hat again?
Historically the Fedora guys have been trustworthy to the extreme. That's why not everybody is jumping on them right now, despite the distro-partisans who smell blood in the water. Again, we'll re-evaluate our position on that once the dust settles.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
Exactly. It's not a breach of any FOSS licence. It's possibly a breach of FOSS project best practice, but that isn't clear either, because we don't know how the problem happened or what code had to be modified to fix it.
Even if some FOSS code was modified, there is no licence obligation to distribute the changes unless you are distributing the binaries.
As I understand it, the security breach was that someone gained remote access to their servers. It doesn't necessarily follow that any of the code served by the servers was faulty. Last time I checked, not all the code running Redhat sites was open-source.
And the breach could well have been down to a sys admin error, rather than a problem with the codebase itself. It would obviously be acutely embarassing if Redhat's in-house team turned out to have made the kind of mistake that causes people to fail their RHCE exam, but it wouldn't have anything to do with FOSS.
Also, there may not be a simple answer to the 'what does this mean for me?' question. In the Debian case, the answer was quite simple, and so was the solution. The Redhat announcements sounded to me like "We know there was a breach, we don't know exactly what happened as a result, we don't think anything serious happened, but, to be on the safe side, we are changing all the locks."
Redhat's PR department obviously misjudged the best way to handle this incident, but the expectations of the FOSS community also seem unrealistic. When a company open-sources some code, it doesn't mean that anyone in the world gets unfettered access to all the information in the company. Reading TFA, I can't help but think that it is at least partly motivated by the blogger's outrage that Redhat didn't roll out the red carpet all the way to the server room for his terribly important blog.
Virtually serving coffee
Someone else in the same situation might truthfully report: my vendor is keeping me the dark, I don't know the nature and degree of my own exposure.
That would be me - our RHEL5 system has the trojanized versions of OpenSSH mentioned in the Red Hat Security Advisory installed, and Red Hat did not provide the most crucial information for me: what harm these packages are able to cause (i.e. which passwords should I change, whether to look for secondary breaches on other - non-RHEL - systems, etc.), and how they got into my system. Also, they were pretty slow releasing the details. The packages were signed by their key on August 13, Fedora servers were taken offline a day or two later (so they definitely knew about the problem really soon), but the advisory was published on August 21. As far as I know I had the trojanized packages installed since August 15, so my system has been 0wned for 6 days, thanks to Red Hat delaying the information.