Slashdot Mirror
Gentoo rsync Server Compromised [updated]
costela writes "LWN points out that the Gentoo project
fired out an alert about one compromised rsync server." From the message itself: "However, the compromised system had both an IDS and a file integrity checker installed and we have a very detailed forensic trail of what happened once the box was breached, so we are reasonably confident that the portage tree stored on that box was unaffected." Update: 12/03 22:54 GMT by T : One more damage report: gibson writes "The Free Software Foundation recently discovered that its software host site was compromised a month ago. The compromise appears to be the same as the recent attacks on the Debian servers. The site is shut down until Friday while they install replacement hardware and verify the authenticity of the hosted source code."
who didn't see this coming? I use gentoo and i figured it was a matter of time before someone did this. I mean haveing a central tree is cool but it does make it more of a target for attacks. I am however glad to see that they took precautions.
Either hackers have decided they *hate* OSS (not likely) or someone is putting up a purse trying to damage the OSS communities security image.
Quack, quack.
Any bets on which major distro will be next? Better yet, instead of point spreads on professional sporting events - Vegas should be taking bets on which distro (or well established free software org) gets rooted next...
First Debian, now Gentoo... Slackware perhaps? Maybe install a spam-bot on a knoppix image?
An intrusion detection system, e.g. snort, can send the logs offsite, so compromising a machine does not always allow you to wipe the logs. The people who administer these machines probably know what they are talking about, and they have little incentive wrt prop. software guys to lie about their system safety. Debian guys came forward with all details, I believe Gentoo people would not tell lies about security breaches either.
ato
Yes, I can. Both give Linux a negative image to people that aren't as clued in about this sort of thing, which is were Linux needs the most support.
break in to Debian, it was notices within 24 hours. Break into Gentoo, noticed in 1 hour. Break in to Microsoft, not noticed for MONTHS.
I'd like to see at least one credible report of a breakin at Windows Update. If someone can provide one you will forever earn my eternal respect.
no, its news because a very popular linux dist has been hacked which could effect a lot of people. that = news
damn microsoft bashing wannabee
I'd just set up a remote box specifically for logging and connect it to a cheapo dot-matrix line printer and have the logs printed to paper. Yeah, you might use a bunch of paper, but it also might come in real handy if you ever need to figure out what really happened to your box.
Someone you trust is one of us.
What baffles me is why crackers go after targets like this.
Because some individuals are asshats, that's why. You could create the cure for cancer and some asshole would try to shoot it down just because it's there. After all, we are the same species that nailed some poor bastard to a cross just because he said we should all get along for a change.
To celebrate the occasion of my 1000th post, I will post no more forever on Slashdot. Goodbye.
PS, full props for the Lebowski quote!
Luck favors the prepared, darling.
Why not?
You take the keys of the developers [or even a cvs key] and then sign all the emerge files. There are only like 2000 new ones a day so at about 50ms a signature [for a really slow box] that's only 100 seconds of time [two minutes not much].
That way if the end user downloads compromised emerge files they could detect them.
Damn... I'm like a genius.
Someday, I'll have a real sig.
A conspiracy theorist could have a field day..
Uh....Ok. I'll bite. Top three theories about why all the Linux bad news.
Number 3: Some companies that got in early on are outgrowing their business models and thus adapting.
Number 2: Some companies with REALLY flaky software and business models are trying to figure out how to use other peoples superior software to increase their own revenue.
The number 1 reason....: How much fun can it possibly be to say "I did a google search on Windows Exploits and owned 1000 boxen in just under an hour" as opposed to " I heard about an SSH2 compromise and searched for 2 weeks and found an affected system, gained access. Found another program with an exploit kit, eventually gained root. All in all it took a week."
facts are tricky like that:
"We don't know how many total servers the numbers were gathered from or what percentage of those servers is Linux vs. Windows, etc. It is safe to say that these results are true for the servers they monitor, but the percentages may not be true for all servers across the globe."
while there certainly exist a large number of linux machines that have been compromised, i can't imagine the number of infected linux machines is anywhere near that of the win32 systems infected by blaster/welchia/code red/nimda/sql slammer/klez/dumaru/sobig/etc. in the same time frame. i suppose the counting in this case depends quite a bit on the counter's definition of "compromised."
IDS is placed on a system to follow an attack. Audit trails on sensitive machines reveal all commands executed, to the detail you desire.
Here is the point. Bruce Schneier says that the important part of security is not that you were compromised, but rather that you can react within a time frame to keep the damage to acceptable levels. If you can tolerate having your system compromised for weeks, don't invest in a lot of security. The short response time (2 hours at 11pmEST) here indicates that the Gentoo administrators care about responsiveness enough to check on it frequently.
When the CVS gateway to Bitkeeper on the Linux Kernel was compromised, the developers of Bitkeeper were able to show that they care enough about security that they invested in many checks and balances that caught the error immediately. Since then, Bitkeeper developers, interested in protecting their good reputation (which is VERY difficult to replace), are considering even more drastic measures.
As a bonus, some cracker spent a good few days or weeks writing this exploit. We get to keep it and deploy the solution with little hassle. And the compromised system, because good security practices are in place, was mitigated to minimize damage.
Read Schneier's book Secret and Lies to find out how security is really a process. Yes, I know it's a plug, but I just thought the book hit-home to the real point - "When, not if" you get compromised.
Several other posts here hint that the world will think less of Linux for this. False. True CIOs should see that Linux has the tools to completely identify and contain attacks. Every CIO knows attacks cannot be stopped, but rather they must be contained to acceptable levels.
Let's face it, no OS is 100% secure. Operating Systems that are more secure than others still need to be on their toes. One security exploitation on a Linux box can still be as dangerous as a thousand (an underestimated ratio I'm sure) exploitations on a Windows box. However, I will take the body of security knowledge surrounding an OS to be as valuable as the initial security design principles in the OS in the first place; with that in mind, many Open Source OS's come out looking pretty good. I trust the Linux community to grind down and fix security problems and not sit around and emphasize the numerous security in a Microsoft product. If you're concerned, then help out developers by testing the software and reporting bugs. You could even code a few patches yourself, that being the whole point of community-based development.
Whether or not there is a deep and dark plot to root big Linux boxes is irrelevant. This is another opportunity to demonstrate the Open Source community's response to security issues to the rest of the computing community. If the heat is really on and this is not just another artifact of news gatekeepers getting over-zealous on a trend, then so be it. It is an opportunity to review and evolve Linux's security as well as the security processes that surround it.
One of the things I admire most about Linus Torvalds is his steadfast commitment to the quality of his product. It is a commitment that is focused on constant improvement, not PR damage control. I'm sure the real security guru's are sitting with a bit more comfort knowing their servers are running Linux.
Disclaimer: This post contains no constructive content whatsoever, swallow two tablespoons of salt and call me in the morning.
Since Gentoo doesn't have a "THE" source code repository, I'm afraid you've got some facts to get straight, Herr Coward.
The mirror had read-only rsync access to Gentoo's primary (US) mirror. Even if the tree were compromised, the changes could not propagate into the main tree. For that, one would require CVS access to the CVS repository, against which the primary rsync server is synchronized.
This was only posted as a matter of keeping our user community, and the OSS community as a whole informed.
Also, I believe the announcement gave mention of it, but the Portage tree on the primary mirror was re-created from the CVS repository immediately upon being notified that a mirror was compromised. Within 30 minutes, every Gentoo rsync mirror had a fresh copy of the tree automatically (as stated by Gentoo rsync mirror policy, mirrors are updated every 30 minutes in order to remain on the official rotation).
Sorry for the confusion, all, but there's really nothing to see here. But it was good clamouring practise for when/if a real Gentoo server is compromised. ;)
BD Phone Home!
Shameless plug. Like you weren't expecting it.
I love it.
I've pointed out before that Windows is way more widespread than Linux, and so is more attacked and vulnerable, but then zealots come on and say Apache is the most-used on the net and yet not the most breached. But to this, it's already the most-breached operating system.
Hoot and holler about the reasons all you want, but them's the facts.
We REALLY, REALLY need to stop with the "Linux is invincible, Windows sucks" attitude. It's flat-out not true, and it's severely holding the community image back in the minds of the rest of the rational computing world who just uses what they use to get the job done and don't treat operating systems like religious belief systems.
"Sufferin' succotash."
Two compromises, both cought within an hour and with no (absolutely none) adverse effects on the users - there is just not much room for improvement here, this is what good security is.
sic transit gloria mundi
Cars are built out of steel, not glass. Glass is a very strong material. But hit it with a hammer and it shatters. Steel just gets dented.
Gentoo had "ductile" security. They were able to limit the damage because they had some kind of Tripwire/mtree-like program running on the inside. Given the speed of the response, my guess is that they had a response plan ready to go.
The lesson is that measures to limit the damage from a break are as vital as measures to prevent breaks in the first place. Fire prevention doesn't substitute for sprinkler systems, and intrusion prevention doesn't substitute for backups. You've got to have both.
Linux/Unix is fundamentally secure, windows is fundamentally open designed as a disconnected workstation and slowly being secured. This is NOT Microsofts fault for marketting reasons they have to move the code base slowly or there are too many problems.
I'm going to disagree with the absolute statement that this isnt Microsoft's fault. I agree that the design of Windows not taking into account network security issues at it's inception is not their fault. it wasn't on the radar as an issue facing personal computers when windows was originally written.
However, building products you are going to market as a server that don't take into account network security is absolutely their fault.
Building applications that are designed to be used across a network (like IE and Outlook) and not seriously considering the security threat to the system that they create is their fault. Actively adding features to those applications that hamstring any attempt to secure the machine is their fault.
Claiming your stuff is secure while trying to crush anyone who exposes that it isnt; that's their fault too.
So there's plenty of security related issues with Microsoft that absolutely are their fault.
Gandma and gradpa will not compile the kernel. They will use the standard upgrade path of binary packages. They will trust the source computer has not been compromised as Microsoft users trust the Microsoft site is not compromised.
This is a great reason why security issues with computers used in the upgrade path should be disclosed quickly and the clean up process should be transparent.
The honesty of OSS groups to disclose information about vulnerabilities is one of it's strengths.
Darth --
Nil Mortifi, Sine Lucre
Really? How does a windows machine being compromised effect me?
Ever looked at the amount of incomming traffic when you're online? Ever considered where the amount of you are getting is mainly comming from?
Unless they get windows.update, I am not concerned at all.
Well, start worring right now. How big do you consider the chance that your vendor tells you about that? They don't even tell you about problem in your OS they know about for months before some exploit is published in the wild.
I do share your concern about trusting the source of your software, but even with these compromises i'd trust Debian and Gentoo more that a big company that has a huge interest in hiding problems like that.
There is no solution to this problem, other then writing all your software yourself. The thing that comes the closed to that, while being still practical, is and open development model where a lot of people are reviewing the source for mistakes and/or malware.
From the Gentoo Altert:
Gentoo realized that they got hacked after one day.
GNU Savannah realized that they got hacked after one month.
It's time to propagate the use of file integrity checkers! They can detect the effects of any new exploit and can't be circumvented (when properly used!).
AIDE
Tripwire