Slashdot Mirror
Gentoo rsync Server Compromised [updated]
costela writes "LWN points out that the Gentoo project
fired out an alert about one compromised rsync server." From the message itself: "However, the compromised system had both an IDS and a file integrity checker installed and we have a very detailed forensic trail of what happened once the box was breached, so we are reasonably confident that the portage tree stored on that box was unaffected." Update: 12/03 22:54 GMT by T : One more damage report: gibson writes "The Free Software Foundation recently discovered that its software host site was compromised a month ago. The compromise appears to be the same as the recent attacks on the Debian servers. The site is shut down until Friday while they install replacement hardware and verify the authenticity of the hosted source code."
A conspiracy theorist could have a field day..
Now where did I put my tin-foil hat?
Is it even a linux box? Just because it's part of the gentoo rsync respository network doesn't mean it's running gentoo, or indeed linux.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
I just threw away my tinfoil hat and made a new one out of steel. With a spike on top.
Once is happenstance, twice is coincidence, three times is some one playing silly buggers.
(Kernel.org, debian.org, gentoo.org - all in the same two months?)
Beep beep.
... they DO have records of what was done and were able to isolate it pretty quickly. IMHO, that's probably saved them a lot of trouble.
Whether it's because the cracker was sloppy or inexperienced, or because the Gentoo team have good server security, I can't say - but it seems they were pretty lucky compared to Debian.
What baffles me is why crackers go after targets like this. I can understand anticapitalist stuff, but my intuition says someone trying to crack a *nix server and damage a distro must have detailed knowledge of *nix systems - and is therefore likely a user of an OpenSource operating system.
Is that guess a little too far off base? If so, what's your take?
"It is dark. You are likely to be eaten by a grue." -- Zork
To those who aren't intentionally trying to troll.. and computer journalists;
Yes, Linux servers can be compramised.
No, the sky is not falling.
No, it's not the end of Linux or open source.
Luck favors the prepared, darling.
Infrequency?
Get your facts right:
"Linux is successfully compromised more than any other operating system". Mostly due to people setting it up straight out of the red box without adequately Reading The Fine Manual.
Leading to the hacking machine? Fixing the compromises on major linux servers is one thing, but why has nobody mentioned finding the perpetrators?
Anything in these logs on the source of the hacks? Probably another hacked machine, but perhaps it can be traced to a source.
Also, in any package that were compromised or attempted at, what is being inserted? Perhaps we can use it as a honeypot to catch a hacker?
Perhaps 2.4.23 should have a kernel allowance for a log that tells when somebody was trying to use the =2.4.22 exploit (or does it)?
Does anyone have an old, cached copy of the DNS record for rsync.gentoo.org?
.
Diff it against what's out there now and we're only a quick trip to http://arin.net/whois from knowing who it was . .
-Peter
lets see, the big distros are
Red Hat
SuSE
Debian
Gentoo
Slackware
(sort)
Debian
Gentoo
Red Hat
Slackware
SuSE
Red Hat (if they still count since that fedora thing) followed by Slackware, then SuSE
however, i probably missed a couple of other big ones, but then there is the whole debate about what is a major and minor distro.
Someone seems to be trying to get a trojan in Linux, probably to give it a bad name. Maybe it would be a good idea to try to do the same to the possible perpretrator(s) before they succeed, so we can point to them when (not if, when) they manage to do it?
(Note: this is only an hypotetical question. I'm not saying anyone should do it except as a thought experiment.)
I can agree to this in a way. Just because someone like an Alternate OS does not mean that they are ignorant about Linux, after all it's well documented!
This SIG pulled due to lack of funding. (This damn war is costing too much!)
And what if syncing to the server installed a compromised "emerge" program?
This would be fine assuming no software was emerged, if one of those 20 happened to 'emerge -u system' and there WAS packages amiss, that would be bad and not cleaned up by an emerge sync.
Would be a good thing to see if notifying those 20 people was possible.
Anything is possible given time and money.
One of the servers that makes up the rsync.gentoo.org rotation was compromised. This box is not an official Gentoo infrastructure box and is instead donated by a sponsor. The box provides other services as well and the sponsor has requested that we not publicly identify the box at this time.
While it may run Gentoo, it is not stated as such, and could be very well be something else.
Get a free ipod.
you could also RTFA that you posted a link to
We don't know how many total servers the numbers were gathered from or what percentage of those servers is Linux vs. Windows, etc. It is safe to say that these results are true for the servers they monitor, but the percentages may not be true for all servers across the globe. For more statistics and the rest of the story you can see this article at globetechnology.com: Linux is favourite hacker target: Study.
as the article states the statistics are not conclusive because they cannot be confirmed because the data of the actual study is not presented.
they also do not define was is meant by compromising, as far as i can tell it could be anything from a remote root exploit, or a setup in a mail server that creates an open spam relay
So I've been lurking around here long enough to spot certain trends. (Warning: generalizations ahead)
OSS advocates love to hate Windows
OSS advocates gloat when a new hole turns up in Windows
OSS advocates point to the number of worms, virus, etc in Windows and say, "Never us"
Then several OSS distros have a security breach in a short space of time.
OSS advocates respond with "Must be a conspiracy against us by some evil entity", "Hey, look how quick we caught it", "It would have been much worse with Windows".
Time to face facts gents. Windows is attacked FAR more than OSS. Why? Well, yes, it is full of holes. But downtown Philly is riddled with abandoned houses with no locks on the doors but they never get broken into. Why? No value in doing so. Not enough damage, headlines, misplaced glory, etc. But the main reason is that it is the dominant OS out there. I fear that we will see more and more attacks against OSS with it's growing popularity. If we all get our wish and 'nix takes over Windows dominant market position and is running on 90% of desktops, you will most likely find it a target for constant attacks like Windows has now.
We all know in order for 'nix to make it to the desktop, it has to become WAY more user friendly. Can't have Grandma trying to recompile the kernel now can we? User friendly unfortunately translates into users being able to do things that comprise security. Like opening attachments, downloading Trojans, etc. Then the great security built into the OS goes right out the window. no pun intended).
So before you all start crying about conspiracies, et al, just remember that we all may be victims of our own push to make the 'nix stuff more popular. By bragging about how secure it is, we just may be attracting the type of attack that is more sophisticated then the script kiddies attacking Windows. I imagine it's cool to brag to your friends that you broke into a Windows box. I imagine it's much cooler to brag that your rooted a Linux distro. Badge of honor and all that.
The opinions expressed here are not mine, but those of these dang voices in my head.
Comparing WindowsUpdate to the portage tree isn't quite an accurate analogy. Portage is distributed to a number of 3rd party donors/volunteers who look after the servers. It's not like the Gentoo team looks after them.
A better analogy would be to ask how many times the update sites for RedHat, Mandrake, etc. etc. has been broken into, since the main update locations are kept up by their respective corporations. I have no idea what the answer to this is.
Even if you didn't RTFA, at least RTFP.
However, the compromised system had both an IDS and a file integrity checker installed
The file integrity checker will have provided a list of the files that changed and if emerge was compromised then Gentoo would have let you know. After all, they haven't kept the compromise quiet so presumably they are informing users to let them know the Gentoo are on top of things.
The Romans didn't find algebra very challenging, because X was always 10
And who doesn't see this coming again? All you need is a box with some bandwidth, and you can become a gentoo mirror.
Now, you want to compromise every gentoo box on the planet? Edit any ebuild you want to add your compromise. Make it break out of the gentoo sandbox and erase that system straight from the ebuild. Or make it install a tainted binary. Whatever, just be sure to re-hash your ebuild in the Manifest, and wait for some poor suckers to download it. Given the frequency with which gentooer's rsync, this should happen very quickly.
There is abosolutely _ZERO_TRUST_ in the gentoo system, and it is frightening how easily a rsync mirror could abuse whatever clout it has to taint a significant number of hosts.
The solution for this is signed digests and shared trusted gentoo keys, but this is still a ways off.
The death of one is a tradegy, but the death of millions is a statistic
For all of you that are curious, this isn't a BSD troll (although it could be...).
My point here is that whenever a larger *NIX server is broken in to, there are ALWAYS people that comlain about "the insecurity of *NIX". Well, when ONE large *nix server is broken in to, it makes it to the front page of slashdot, whereas blaster/sobig/etc usually get a story or two.
This is where the quote above comes into play.
Linux might look insecure, but that's because we usually hear about breakins on a 1 server basis. When we here about Windows, it's usually in the HUNDREDS OF THOUSANDS (if not more). If there was a slashdot story for every one of THOSE servers, then it would appear the way it actually is.
Not Free(as in beer). Free(as in "I'm free to beat you over the head for being a dumbass")
You'd think but www.openbsd.org doesn't run OpenBSD. Here's a link that explains why.
In fact, just last year ftp.openbsd.org did get compromised!
A radio maverick jumps to internet only. The Future of Rock n Roll
If you buy the idea that spammers are behind many of the recent worm/viruses, designed to turn machines into spam zombies, it's also probably reasonable to conclude that Windows isn't exactly a reliable platform to trojan; I'd bet a lot of trojans fail to infect properly simply due to Windows problems. And then there's the problem AV software, many of the machines being behind firewalls/NAT and being unreachable. And then there's people turning their PC's off when they don't use them.
OSS machines, however, are a much more reliable computing environment, meaning that any trojans are actually like to work, and work well. And I'd also wager that many OSS machines are used AS firewalls or bastion machines, and if compromised are easily accessable for spamming or use as stepping stones to other machines. And many of these machines are always on -- you don't have to worry about lack of reliability from disabled machines.
This makes more sense to me than any other conspiracy.
http://savannah.gnu.org/statement.html
On December 1st, 2003, we discovered that the "Savannah" system, which is maintained by the Free Software Foundation and provides CVS and development services to the GNU project and other Free Software projects, was compromised at circa November 2nd, 2003.
>> There is abosolutely _ZERO_TRUST_ in the gentoo
>> system
> Don't you mean "total trust" in the system, in that the
> users "trust" the rsync servers not to be r00t3d,
> somewhat optimistically?
Doh! But yes, that's what I meant in my somewhat hasty response. (I have been thinking about gentoo and this for a while.)
Gentoo would be ideal with a web of trust, such that a gentoo deployment maintained keys that it trusted -- ie, from gentoo, from developers, from friends, etc, and accepted and rejected ebuilds accordingly. Considering the development distribution of gentoo, it is almost a case study in how to build and use a mini public key infrastructure.
The whole gentoo tree could be viewed as as many 'virtual' trees as you desire, by multiple signatures on single or multiple ebuilds: for example, instead of setting "~x86" (experimental x86 ebuild) from within the ebuild, why not sign the ebuild with the gentoo_experimental_x86 key? Then, sign it with a "testing" or "stable" key as it fits.
With keys from developers (gpg message signing is already a big habit on gentoo-dev), you can accept patches from developers and other people you trust, even if you receive the ebuild out of band (eg, on bugzilla). All of this also removes the posibility of, say, an errant CVS commit. Now all you trust is those public keys (and however many signatures you require on an ebuild to believe it wasn't porrly signed.)
(Of course, verifying signatures adds more complexity to the build time, but... this is gentoo! You love the build time!)
cheers.. (a gentoo user)
A Netcraft search for rsync.gentoo.org shows more than one server. Two of them run Gentoo, two run Red Hat, one runs Debian, three run unknown Linux, and one runs FreeBSD (some of the servers are listed twice). There are more servers (14, if one is to believe 'host rsync.gentoo.org|wc -l'), but Netcraft is only interested in those with web-servers.
You make it sound so easy. Just "break out" of the sandbox and erase the system. No system is ever going to be 100% secure, but things like sandboxing make it safer. I'll take my chances with Gentoo. The RPC exploits alone have convinced me to never even look at a Windows box again. It's just too much hassle.
Time makes more converts than reason
But I'm glad that there has been so many attacks against linux and other oss projects.
Kernel.org, debian.org, gentoo.org Gnu.org All of them had security holes and now those holes are plugged.
I used to run a few servers. Mostly web-servers, but I had a few for mail and other things. Almost every single one was hacked all in the same 2 month period. I had kept up with updates and I figured I was secure. If I wasn't hacked I would have never known that I wasn't secure and I could have been seriously screwed down the line. It was a much needed eye opener.
I'm just this guy, you know?
...and don't treat operating systems like religious belief systems.
I really don't want to be a smartass here but could this be a case of the pot calling the kettle black? You don't seem at all Overly Critical when something bad happens to Windows. Indeed, your posting history is largely criticisms of Linux. I could exchange every instance of Windows and Linux in a typical posting of yours and you would come off exactly like one of the "Linux religious fanatics" you claim to be above.
You also seem to think the most vocal and rabid Linux users are typical users. Every community has extra obnoxious members and Windows is not exempt from the vocal religious fanatic problem. And yet, no one speaks of obnoxious Windows users being the biggest problem dragging Windows down. Could it be that telling amorphous groups like "Windows users" or "Linux users" how to behave is a largely useless activity? Could it even be that "the way members of foo act" is in no way a valid criterion for assessing a technology?