Slashdot Mirror
Debian Project Servers Compromised
Sean was one of many to pass along
the bad news
from the debian-announce mailing list: "Some Debian Project machines have been compromised. This is a very unfortunate incident to report about. Some Debian servers were found to have been compromised in the last 24 hours. The archive is not affected by this compromise! In particular the following machines have been affected: 'master' (Bug Tracking System), 'murphy' (mailing lists), 'gluck' (web, cvs), 'klecker' (security, non-us, web search, www-master). Some of these services are currently not available as the machines undergo close inspection. Some services have been moved to other machines (www.debian.org for example). The security archive will be verified from trusted sources before it
will become available again." They were going to announce 3.0r2 this morning; they've checked it and it's unaffected but obviously they're still postponing that release.
The debian-announce archive [ http://lists.debian.org/debian-announce/debian-ann ounce-2003/threads.html ] doesn't list this message. Of course with the number of machines affected it's possible that the mailing list archive is somehow affected.
-JohnF
dave
Tech stuff
Why my apt-get was failing from people.debian.org last nite. Not to mention why debian.org was down. :(
Of course this raises the whole issue of apt-get. We all rely on apt-get update && apt-get upgrade, all it takes is someone to compromise the servers and insert a backdoor
This is the second time this has happened to a big open-source project (the first being the GNU servers a while ago). All packages by both groups are "md5" signed, which is supposed to protect against malicous hacking. However if the root server is comprimised, this doesn't help. Companies (including at least Microsoft, and the people who make ad-aware) who distribute files over the internet sign them with an RSA (or similar) key, and the computer which does this signing is kept disconnected from the internet. For such large projects which are installed by millions of people, might a similar system not be a good idea?
Combination - fun iPhone puzzling
How long will it take for the few MS fanboys around to say that this why Windows is better? Let me pull a Rumsfield (pre-emptive retaliation, that is...). Everyone gets comprimised once in a while. At least Debian is open about it, and not sitting on an insecure system because it's more profitable to let a bad product go then to risk bad press from releasing a security bulletin.
#define DRM chmod 000
getSexySig();
Are deb's signed? (I'm not that familiar with debian but I'd imagine they are) If so then just tell apt-get to not install debs that don't match a known signature...
It really is impressive for me how honest some organizations have been about admitting system compromises (Debian, ProFTP, GNU.org).
As someone who works with networking security, I know lots of business servers get compromised regularly. Everyone hides it because it's embarassing for a business.
This makes you wonder how often other 'critical systems' get compromised, and get fixed without any public reports. Government computer systems get regularly compromised after all. But I'm sure so do vital Microsoft, IBM, systems, etc. Windows Update, anyone?
Security is much much more than "just keeping your system up-to-date".
...) to log in to one of the servers
- accounts can be compromised
- unknown bugs may have been exploited (although that's unlikely in this particular case)
- crackers could have been cracking a developer's system, and using information they find on that developer's hard disk (ssh key, gpg key,
- also of importance in general is the competence of the administrators (which surely is *not* at the cause of the problem here).
Of course these systems are running debian stable; but that's most likely not the problem.
Yes Debian's machines run Debian, this breakin wasn't anything to do with the software installed upon the box, as it was due to a password compromise.
If anything it's more embaressing that somebody lost their password than that the software wasn't up to date.
What's interesting about your comment is that when a M$ compromise comes to light, the focus is on how big a bozo BillyG is for letting his insecure crap out into the world. When something like this happens, its those nasty little hackers or script kiddies and their deep dark motives or a cabal led by M$/SCO to "discredit" Linux. Face it, the main servers for a major distro was hacked into at a very sensitive time. Ouch. Regardless of the whys of who did it, it was done. Yeah, kudos for them coming public, but if I joe CTO and looking at purchasing some puters, I'm thinking to myself, hey, what's up with this, they told me that M$ stuff sucked and this Linux stuff was secure. This wasn't some ma and pa website that got defaced after all.
If a password is compromised, it does not matter what system you run. And everything I've read indicated this break-in was the result of a compromised password.
Finkployd
The server that pushes .debs to archive is running debian/sparc (donated by sun btw), so probably the cracker didn't know how to port his leet exploit to sparc (all the comprimised machines were 1386).
signatures pending - ansa@kos.to - (dont mail there)
> I noticed that nowhere did they mention just *how* they were compromised.
They will when it's known. They felt it more important to announce what's going on immediately than to wait until there were details to announce. Part of Debian's social contract is "we will not hide problems"; this announcement and those that will follow as more is known demonstrate this policy in action.
.debs should be gpg signed, and should fail to install if the verification fails. In fact, so should all packages from distros. Redhat, +1, Already doing it. -1, not failing to install if the packages don't verify.
Get your own free personal location tracker
At least cjwatson and myself are Debian developers. I wish I could say it's a hoax, but it's not. However, as you've already read: the archive doesn't seem to be compromised at all.
At this point I would like to see the debian team develop some written policies and procedures for how they intend to prevent this sort of thing in the future. I checked the site and while there's security info for how to secure your box, there's no policies on 'how does the debian project secure itself'.
Lastly, one concept you have to keep in mind, we have no idea how often other OS's key servers are cracked because they'd never tell us.
If Debian ran OpenBSD, this wouldn't have happened! Theo runs a tight ship over there.
I also think that Gentoo would have prevented this tragedy.
Not really. The vast majority of break-ins are through misconfiguration or human error. Gentoo, OpenBSD, nor anything else, can prevent these factors. I would be very surprised if this was due to a security hole or vulnerability. More likely someone wasn't secure enough with their SSH keys or something like that.
As Linux becomes more popular this is only natural.
Open-source projects are not immune to attack and they are going to start feeling some of the pain experienced by other big targets like Microsoft. In the beginning it could be really bad because unless you're being attacked seriously all the time then you may not even realize where your vulnerabilities are.
This is a wake-up call to all "open" projects. Systems that are in use by a large number of people need to be protected better. Sure, this may have been a password compromise but the system should have been secure enough that some low-level user account compromise can't cause serious damage. And the high level accounts should never, ever have a password compromise. This needs to be treated in the same way big business does. Protect the customers, otherwise you may lose them.
This made me start thinking... Has Redhat ever been compromised? That'd be a reason for going with a commercial distro if the free distros can't get their act together. (I've been a Debian user for many years by the way)
The ratio of people to cake is too big
--- snip here ---
K ik pLMtJKcxSKUgvyn NTi4sT01MVEtMTM/ OKS4CCqQrZqZUK1 K5dHW5OuyZWUE27o M5QZDp9w6GBQtO+ hFp+fRBXM7HXcYc1 6Xj5A9DwA=
This is a truthful report.
You may validate this message against the key for skx@debian.org.
Steve
--
-----BEGIN PGP MESSAGE-----
Version: GnuPG v1.2.3 (GNU/Linux)
owGbwMvMwCR44PyxzWd9eOcyns5PYrDfJ7EiJCOzWAGIEhV
i0r0uLgi80sVchMrFcoSczJTEktSFUpAi
aflFCsXZFQ4pqUmZiXl6+UXpQCO4gktSy
SLxI+1madnvjbIZVrZu0HcTnzGdY0LBFy
=xVtr
-----END PGP MESSAGE-----
Was any code stolen? OH wait...
There are a huge number of yeast infections in this county. Probably because we're downriver from the bread factory.
Of course, we shouldn't jump to conclusions until we get more information, but really, I don't see an easy way out of this.
Why should you? They were cracked. The bad thing has already happen, so there is no easy way out. However, there *is* a *right* way out. And that includes telling people what they know as quickly and effectively as they can. Too much information too early can be a bad thing.
In short: have a little faith that they're dealing with this correctly, unless you've run a massively-used public box for years without a single compromise.
-Rob
-Rob Ewaschuk
First GNU, then Bitkeeper, now this, whatever shall we do?
Simple, the technology has existed for decades now.
A little something I like to call "Public Key Cryptography"
With this "Public Key Cryptography" you could conceivably sign software in such a way that it could not be altered without breaking the signature, AND ensure that nobody else could forge this digital signature (you are keeping your private key private right?)
MD5 Hashes are a step in the right direction, but by themselves are meaningless. Sort of like improving your home's security by drilling holes in your door to mount a deadbolt but not actually taking the final step and INSTALLING THE DEADBOLT.
So let's take these MD5 hashes and encrypt them with the package maintainer's private key (or distribution maintainer, whatever). Then dpkg (or rpm, emerge, whatever your favorite package tool is) could be written to decrypt this hash with the corresponding public key. Wait, there is more! Then it could generate it's own MD5 hash of the package in question and COMPARE it to the decrypted hash it just created. If they match, the package is unaltered AND came from a trusted source. This my friends is what we like to call a "digital signature"
I don't care how you do it, GPG, x.509, whatever. I'm actually leaning toward x.509 since it seems to me to make more sense to have the distro maintainer run his/her own CA and issue certs to package maintainers. This CA could then be included in whatever package tool is used and viola. No mucking about with the web 'o trust (Which rocks for ad hoc trust relationships like between people emailing each other, but sucks for this kind of hierarchal stuff)
So what do you think everyone? Good idea or should we wait for a few more server compromises before we think about securing software repositories?
Finkployd
In response to the dastardly assault against the twin (mini-)towers, the President of Debian drew a line in the sand and immediately announced the invasion of Slackware.
I don't know if this delayed a release, but -- in October 2000, the news broke that Microsoft's internal network had been cracked for three months.
(Debian made this announcement in 24 hours.)
Read for yourself:
Microsoft Cracked
As much as a troll he may be, he does have a point. Windows zealots usually use stories like this to say that Linux is insecure. However, when they do that, we can just say "So what? Open source is still more secure. If you want absolute security then go use OpenBSD."
It's not about Linux vs Microsoft, it's about Open Source vs Microsoft.
Heck, maybe even Unix vs Microsoft. Because then we can use MacOS X to beat all the Windows zealots.
I doubt that Microsoft (or any commercial software company) would publically annouce that it had been compromised. The source code processes at Microsoft are opaque -- nobody knows exactly who is putting what into the source code. If hackers, goverment officials, RIAA, etc. are modifying Window's source, nobody would be the wiser. In contrast, the openness of open source development creates an audit trail of who did what to the code (assuming the version tracking and submission system is not compromised).
Transparency is a prerequisite for trust.
Two wrongs don't make a right, but three lefts do.
I ran apt-get and my machine was converted to Windows 2003!
/* It's amazing the damage someone with a stunted sense of humor and mod points can do to your karma. */
Funny, my apt-get using h4x0r3d.debian.org was working perfectly....
My beliefs do not require that you agree with them.
PGP keyservers (unlike, say, Kerberos KDCs) are completely untrusted. Anyone can upload any key to a keyserver. And downloading a key from a keyserver implies nothing about that key.
To verify that you have a valid key, you have to rely on the web of trust. Basically, if a key is signed by someone whose key is signed by someone [recurse through however many levels you are comfortable with] whose key you have personally inspected, then the key can be assigned a trust metric based on how reliable you consider that chain of signatures to be. (Basically, how much you trust the integrity and acuity of the people controlling the chain of signatures.)
PGP and GnuPG have supported this infrastructure from Day 1. Asking people to trust an arbitrary third-party public keyserver was never in the plans.
"How can you claim that you are anti-crack, while still writing a window manager?" — Metacity README
You're right, up to a point. But you've also got to compare the other factors that tend to crop up...
/. crowd too busy laughing to make sensible posts. /. crowd too busy downloading, testing, and installing the various patches and workarounds that are flying around.
Windows Box Compromised: Someone exploited a flaw.
Linux Box Compromised: Insecure password.
or, if it IS due to a flaw exploit...
Linux: Box compromised because machine wasn't carrying latest patches.
Windows: Box compromised even though machine was updated last week.
Linux: Exploit found. Exploit gets fixed. Publically. Usually the same month - with a temp-patch available within the week.
Windows: Exploit found. Exploit gets fixed. Eventually. As a part of the next service pack. Newsgroups, Slashdot and third-party sites suggest workaround. MSKB just says "Problem is under investigation"
Oh, and there's always...:
Windows exploited:
Linux exploited:
(Or sending "Use a good password" memos around the office, stating that if an organisation like Debian can be compromised by a password, then Joe Average in accounts hasn't got a hope in hell if his password it the cat's name.)
Tiggs
"120 chars should be enough for everyone..."
All three of my Linux boxes run Debian; this latest security breach will not change that.
However, I hope this type of incident tempers the often-strident elitism of the free software camp. My faith in Debian continues because they caught this problem and openly announced it; my concern is that the lack of consequences will make people assume that this was a false alarm or unimportant incident.
Free software suffers from "victory disease" -- an assumption that, based on past success, future success is guaranteed. Because free software has proven reliable and secure, the concensus seems to be that it will always be so.
Pride comes before the fall, as they say. Attempted infiltrations of the Linux source code control system and breaches of security at Debian suggest that we need to be cautiously optimistic, not naively myopic.
All about me
218 posts and some rare appropriate reactions.
- I thought Linux was secure... Guess not. Who told you that Linux was secure ? Your grandma ? Linux is more secure than Windows, of course. But it's not immunized against cracker. The computer world is based on a set of rules that can be broken. The better you are mastering these rules, the more secure your boxes are. But these rules can be broken, which means that, given human nature, they are bound to be broken occasionnaly. Furthermore, you will have noticed that if often relies on human use mistakes (password cracking for instance).
- Free software sucks, Microsoft rules. Here I can almost physically feel the frustration of advocates of the proprietary world that can do nothing but bash any free software flaw they might encounter. However they deserve a clear, sound, and honest answer. My dears fellows, the free software world never proclamed himself the embodiment of security. We do our best to ensure it. And don't mix things up : our main problem with Redmond handling of security is about post-treatment. We do not appreciate the culture of hiding ; you can see here how coherent we are with ourselves.
- Gentto is better than Debian ; oh no it's Redhat ; oh no it's Slackware. Hey guys, are you really part of the free software world ? Can you just realize these are the precise sentences that led to proprietary software/world ? And don't you think that you should adopt a more conservative stance ? Don't you think that the moral of this sad story is that nobody is preserved from crackers ? Wake up men, this is the very crucial moment where we must stand united. Keep your ammo for you real foes.
There are some days when you would think that the free software world is not that 'free as in freedom'...Regards,
JDif
Let's overcome our weakness.
This news made me realize how much I depend on Debian. At the moment, every one of my machines (four servers, three workstations, and a laptop) runs Debian. I've been running it as my primary OS for... two years? So far I haven't paid a dime for it. It is a nice advantage of Free Software to be able to use it for free, but given the fact that I'm way out of "try-before-you-buy" mode, I'm going to send them a check today. Software in the Public Interest was founded by and is the current funding source for Debian.
One server compromise in the two years that I've been watching by a company with zero product sales revenue is pretty impressive. An OS that is (IMO) dramatically superior to any commercial offering for free? They've earned my respect, and have clearly earned my cash.
Stop-Prism.org: Opt Out of Surveillance
I hate to say it, but Microsoft's haven't been compromised, and they're the bigger target.
Not true.
Everyone here knows if windowsupdate.microsoft.com had been compromised, people would be droning on about how it's some sort of illustration of Microsoft's security.
Their update server wasn't compromised, but the debian archive also wasn't compromised in this case. But, yes, we have to work harder to make our servers secure. And we will never reach the point were our systems will be unvulnerable. So what is your point? You complain that there aren't enough anti-oss-trolls here?
It's not a hole, though. So far we only know it as a login/password that was comprimised. Any system no matter how secure is susceptible to that. Most of Microsoft's holes are much different - they're exploitable and are available from the default recommended installation, meaning the computer grandma bought for Bobby is susceptible and will probably never be patched.
Thanks for confirming this. Unfortunately, the way you confirmed it is very dangerous.
Your message contains:
So from now one, your "confirmation" can be used by anybody who wants to claim that some random report of theirs is "confirmed by a debian developer". Until you revoke your own key, of course. That's a pity.
-Raphaël
My point is this. Linux is not the be all end all of existence. Its a great OS, with problems just like anything else. Lets keep this in its proper perspective and try to ignore the hysterical ranting of the Debian wackos.
What does this have to do with the "quality" of Debian? AFAIK, the vulnerability that lead to the compromising hasn't been revealed yet. I could have been something as simple as a guessed password.
Fred
"A fool and his freedom are soon parted"
-RMS
As far as I understand, no machines apart from the several Debian computers have been compromised. Compromising a machine that hosts the central Debian APT repositories is a perfect opportunity for backdooring thousands of machines In this case, that didn't happen. "Thousands of machines across the globe" have not been compromised. I guess it's only good luck but Debian users were not affected by this security breach.
You know what... encrypt your SSH connection at 1024-bit... lock your webserver in a vault, 2km underground, with triple combinations... post armed guards... lock down all ports except port 80 and SSH/whatever.
Then, have your password stolen, and oh shit, you're compromised. It's not about the OS being insecure, it's about a lost password. NOTHING can protect against this, short of one instance I heard where updates required 3 user passwords (from 3 users), but what a pain that would be.
"Where are we going, and why am I in this handbasket?"
Yikes, I'd figure it's the latest infusion of 6/700,000 user accounts, but your number is really low, so I might as well respond to you.
In case you haven't noticed, Slashdot has, and always has had, an editorial bias towards OSS, and against Microsoft. So do the bulk of the Slashdot readership. This is nothing new. This is a geek website, and the plain truth is, most people who call themselves geeks don't just sit blindy clicking away in Windows all the time. We like to play with our toys, we like experiment, we like to open it up and see what makes this baby tick. With something like Linux, you can do this. With Windows, you can't. Those are simply the facts. So of course people here will look upon OSS in a more favorable light.
Yet today, we have comments such as "hysterical ranting of the Debian wackos" being modded up as Insightful and Interesting? Hello people, that's called flaming. If it was more subtle, as yours is, it's called trolling. Walking into a Britney Spears fan club meeting and shouting "Britney SUCKS!!!" is also an example of trolling/flaming. So when you come to a website with an obvious and open slant towards something, and constantly try to point out that slant...
Well, I guess I just don't see why you're bothering. I mean really. If you really think the OSS community is full of shit, why on Earth do you come to one of their main websites/blogs/message boards/whatever?
As far as a double standard goes, I honestly don't get your point. Slashdot has never had a policy of reporting every single hack of a Windows-based system. However, pretty much every major OSS hole/exploit/hack gets a story here. Considering how many Windows machines there are in the world, you'd think there would be a lot MORE exploiting going on (hey, I'll use the "Linux would get hacked too if it was on 90% of computers" line for a change). And yet, we hear more often about Linux machines being compromised.
Well, except for things like Code Red/Nimda/Slammer/Blaster/etc, which, I'm sorry, but you'd have a hard time convincing me that this DOESN'T prove the case of Microsoft being just slightly less secure than Linux. Or else we'd be seeing Apache worms flooding the Internet on a daily basis, because "Microsoft only gets hacked because it's on 90% of computers", right?
Oh, and for the record, password compromises are OS-independant, and have nothing (read: zero) to do with the OS, design paradigm of the OS, colour of the developer's underwear, or whether we use a penguin or a flying box to represent ourselves. Only trolls would be saying "Ha ha ha ! Serves 'em right for running Bill Gates' Satanic OS. Let the jokes begin. Moderators, get ready !" if Microsoft had a machine get hacked because of a password compromise.
Endless arguments over trivial contradictions in books written by ignorant savages to explain thunder in the dark.
This is much worse than one of Microsoft's normal problems. With Microsoft you expect the problems, and therefore you maintain constant vigilance. This is a perfect example of why linux users and admins need to also be wary at all times. As linux becomes more and more mainstream, the number of security holes shown will increase as well. More people will use linux and more "hackers" will then be attracted to developing viruses and worms that exploit the system. Regardless of what anyone thinks about Windows vs. Linux, everyone must admit that part of the reason more security holes are found in Windows is because there are many more people looking for them. My advice to linux users is to drop any pretense of Linux being infallible and to start using the same caution running a linux-based server as you would running a windows-based server.
In the days before the Pure Food and Drug Act, it was considered "nobody's business" what was in the food we eat, either; you just opened the can and accepted whatever was in there. Times change.
But security holes exist, there is no getting around this, no matter how paranoid you are...
trust me..
I am a sitting in a faraday cage right now...I built it in my apartment to keep those pesky NSA spooks from uplinking with the nano-chips they implanted in my brain....
most of us are now implanted...you can't dig them out...i've tried....
Steve from Debian Security Audit project says this occurred due to a password goofup so this doesn't necessarily apply here but it easily could have:
Machine as important as these should be running some sort of Mandatory Access Control system like SE Linux. I have done an evaluation of all of the root exploits I could find over the last few years and SE Linux would have prevented every one of them because the MAC system prevents unauthorized priviledge escalations. You can test drive my SE Linux box by telnetting (not ssh) to selinux.copilotconsulting.com with user root and password root.