Slashdot Mirror
SourceForge Server Compromised
justrob writes: "Looks like there was a massive security breach at Source Forge.
I wonder if this what caused the 'unscheduled maintenance event'
that has left the shell servers unavailable for a week. Here is part of an email I recieved:" (Read more below.)
"Dear SourceForge User,The SourceForge team takes security very seriously. This week, one of our systems was compromised. We have promptly taken the necessary steps to correct this situation.
You have been contacted, because according to our log files, you have used SourceForge during the past week and may have used the system that was compromised. In order to complete the security fix, we are asking all users who used the system to change their password immediately. We've reset your password to a randomly generated string. "
Here's where to go if you think your account might have been compromised.
But I suspect since sourceforge hosts MANY CVS based projects, that open-source software could be injected with outside code...
A developer with any sense is going to do a cvs diff and verify any changes before continuing on like usual. No big deal really.
How secure are the CVS trees? I assume sourceforge's CVS trees are backed up regularly. It would be horrendous if not, because any of the cvs trees on SF could have had backdoors added to them.
Perhaps the authors of the programs should check out their software to a different directory and do a diff.
I am happy to say I don't rely on sourceforge. It is a mistake for a large part of the community to rely on one site for this sort of thing. I run my own CVS server, and am about to pay a small amount ($10/m) for an ISP to host my hobbyist code. Granted, it doesn't have all the features of sourceforge, but I don't mind.
Full usernames do need to be known, because you're dealing with a multiuser system where users like to interact with each other. What happens if you run 'w' on a system like the one you're describing? Using uids for most things would work, but remember that humans use this stuff, and people like to have names, not numbers. If I do an 'ls -l', I don't want to see that a file is owned by user 11203, especially if for the sake of security I don't get to look up who the hell user 11203 is.
Your system could work if applied to the passwords, but it would gain you nothing. Any sane system already uses a hidden password scheme, most commonly shadow. The system never stores the actual password anyway. What it stores is a hash (MD5 usually, these days) that it can verify another (dynamically generated) hash against. Your system does not remove this dependancy on having the hash accessible on each authentication server. You have to have the sum there to verify against. How else would your systems verify that there is a string with "those two letters in that particular position... associated with that particular cryptographic sum"? In fact, your system would have to store the two letters in question and their position to do so.
Bam. You've created a less secure system than the currently existing one, because you've just fed a brute-force password cracker two of the characters it needed for each password, if that file is compromised.
Now that my favorite project, ALSA, is also moving to Sourceforge I'm more concerned about it becoming a single point of failure. Just think about it. If Sourceforge goes down for an extended period of time imagine the chaos, not to mention the loss in 'productivity' in the OSS community. I will probably be one of the last developers moving to it...on the other hand, they provide an excellent service, one that would cost an OSS developer cash otherwise. Anyway, my only real gripe with Sourceforge is their usage of Geocrawler for mailing list archiving. Damn, that's one crappy interface!! :( Sorry for ranting...
-adnans
"In short: just say NO TO DRUGS, and maybe you won't end up like the Hurd people." --Linus Torvalds
What I find particularly interesting about this whole deal is how their distributed network setup has worked against them. Reading the wording of that email closely, you notice that you are being asked to change your password because you MAY have used the system that was compromised. They don't actually know.
And even if they did know, would that help, I wonder? Is user information distributed within the SourceForge network, or is there a more central login server that warehouses all of that information? Either case, when you think about it, is somehow not exactly what you want to have happen. If all user login information is in a central location, cracking that one location would instantly compromise everyone's account information. However, distributing user information around the network (aside from making login management more difficult) makes it more likely that if a random server is cracked at least *somebody*'s account information will be compromised.
It occurs to me that a distributed, and cryptographically fragmented form is the most desireable - because then cracking any one machine would give you exactly nothing in the way of user account information.
Now, the rest of the problem is people who use the server that has been compromised. They send their usernames and passwords to this compromised server (unawares, of course) and thus their information is compromised, obviously. But does it have to be this way?
I propose that it is conceivable to build a login system where no one server receives an entire piece of a login. The login name is split into two character pieces, for example, and sent to as many servers as necessary along with an MD5/SHA1 sum of the full username - each server verifying that there exists a login name with those two letters in that particular position, and they are associated with that particular cryptographic sum, and nothing more. Notice that the controlling server (the one that you're logging in to) never sees any piece of the login name - but is merely informed (by the client) which machines (in no particular order) the login name pieces were sent to. A similar trick could then be played with the password - sending pieces of the password to some password verification servers, with a cryptographic sum of the verified login name. So, each server never has a record of full login names, and no server ever is sent the full login name.
To bad no such system exists, eh?
Queso is very inaccurate. I have seen it give answers that can best be described as "wild-ass guesses". You are better off using nmap for OS detection. At least it has been updated on a regular basis.
"Trademarks are the heraldry of the new feudalism."
Well most obviously they could subtly alter the source code of one or several programs to include a trojan horse.
Then, when you download and install something, you would also install a back door for the cracker to use to easily gain access to dozens, hundreds, or even thousands of machines.
Of course, GPG signed tarballs are what you would normally use to verify the files and protect against this, but how many people actually bother with that?
Ewan
No, you're wrong.
Access to the authorized keys means that the intruder(s) could have added their own public keys, thus ensuring future access. The problem is not public key hashes being stolen.
-Kevin
Sure code could be modified, but not without being detected. I assume that they are going to restore the data from a pre-breakin backup, then diff the backup with the current data and ask the various software maintainers to verify the diffs. Alternately they could require the various software maintainers to checkin their local CVS tree with the master repository, and verify any discrepancies. Any time you have a properly implemented change control system like CVS it is scads easier to recover from a comprimise like this and ensure that your source code has not been comprimised. This isn't scary at all.
-- Remember: Wherever you go, there you are!
Your right about the advocacy facter..
Linux advocacy rises to match that of Microsoft FUD (in otherwords no better than)
Two wrongs do NOT make a right.. three make a uturn and four make a left..
Ok why should Slashdot ever post a Windows defect anyway?
On the other hand.. Go read the stuff on Windows websites.. Every Linux defect is published as proof Linux is poor defective etc etc etc.
Getting the facts straight..
Windows can be secure Linux can be cracked..
It's the SysAdmin that make it happen...
Software is just tools...
But never tell an auto repair profesional he must use the brand of tools you picked..
Same for SysAdmin...
If the SysAdmin prefers Dos he shouldn't be forbiden from using it if it can do the job.
(Some tasks are insainly low overhead and can run from a XT running Dos... very few...)
I don't actually exist.
Until we know, there is just as good a possibility that the site was "cracked" by someone leaking an important password, somebody walking up to one of the servers, an upset employee, the brother (or brother's friend) of someone who works admining the SourceForge project ("Hey, can I check my email?" "Sure"), or even a laptop swiped at E3 that had an admin SSH account preconfigured.
Don't assume that it's a technical problem... I've only had one and a half major successful breaks in security, and both were social, rather than technical problems. Plus the *very* common problem of cell phones, ISP and AOL accounts and such that are used for a few months past termination because HR didn't inform IT. Hell, we even had someone using our contact database (that's the half) because HR never told us, and he was running around with a company laptop and an (easily turned off) key disk.
--
Evan
"$30 for the One True Ring. $10 each additional ring!" -- JRR "Bob" Tolkien
Xix.
"Everything is adjustable, provided you have the right tools"
Running queso on sourceforge.net reports :
216.136.171.196:80 * Novell Netware TCP/IP
I think 'queso' is wrong.
I don't know what PHBs you're talking about, but all the PHBs I've ever met care only about reducing their dept's spending and increasing their dept's output. They most certainly are 'interested in why not to use Microsoft', and it's called money. But they don't think there is anything else out there that they could possibly use.
In the recent past, some of them have learned of this 'Lee-nooks' thing that they might be able to use instead of the oh-so-expensive Microsoft stuff! However, they're very cautious since 'no one was ever fired for buying Microsoft', and if Linux fails (many of them worry Linux and OSS may 'fail' as a concept, but in reality it's their deployment that might fail) then it's their ass for using it. Reports like this one do make them nervous about Linux, but only because they are worried that there's no comapny to blame when this happens (they think).
PHBs may be stupid but they can add and subtract. Unless it's relevant to their business, they care very little about 'uptime' (usually) or 'reliability' (usually), but stuff like 'price' and 'liability' they are very well aware of. Linux intrigues them with the price, but they are very worried about the lack of centralized liability. That's why Redhat is doing so well, the PHBs have to have someone to
Whoever compromised the system should post their exploit on SourceForge.
Feed the need: Digitaladdiction.net
But I suspect since sourceforge hosts MANY CVS based projects, that open-source software could be injected with outside code...
Will they be changing their name to ForgedSource, then?
-- Insert witty one-liner here. --
Actually, I'm waiting to find out what the cause was.
Stupid admin errors happen both on NT and UNIX - they're not a feature of the OS.
Software problems are a different story. And we don't know which yet, so I'm waiting. Mostly. :)
I've heard people decry the fact that most Open-Source projects are now hosted on Source Forge (primarily I think because they were worried about the company suddenly going out of business or doing something crummy and underhanded). I guess this is one of the other consequences of having a centralised repository.
I agree you can't "steal" open-source software. But I suspect since sourceforge hosts MANY CVS based projects, that open-source software could be injected with outside code... correct me if I'm wrong, but that seems pretty damn serious.
cheers,
metric
Sourceforge is probably not withholding information because they don't want people to try it again. Whenever there is a complicated compromise, it can take weeks to figure out what has been compromised and how. Generally the priority is to get the system running again, which typically requires a completely fresh start for the operating system. When a system is compromised, it is easy to tell for some thing what probably has been compromised (such as the password files in this case). But most hackers hide their presence from logs as much as possible - even script kiddies do that. Otherwise they would be detected before they got very far, and it would be easier for the victims to figure out how they were compromised.
How about this one ?
Those graphs are very misleading, because they lump NT4 in with Windows 2000. It is widely known that NT4 had serious issues, which isn't surprising, since it was designed prior to the real internet explosion -- IIS was originally an add-on.
That's a ridiculous argument. Regardless of what it was originally designed for, Microsoft ended up selling NT4 as an Internet server OS. If, as you readily admit, it had serious issues, then why were they promoting it as a premium Internet-capable OS in the first place? Further, when the server technology changes again in the future (and believe me, it will), how can you possibly trust Microsoft to get it right given the mess they made of transitioning NT4 to the Internet and webserving?
Second, I should point out that Linux, and indeed every other Unix and Unix-alike, was designed before the 'real internet explosion' too. Indeed, if you trace back the lineage of 'proper' Unix, it was around when the internet was just being born, and many years before the arrival of TCP/IP, let alone HTTP. A webserver is still an add-on for most Unices and yet they seem to be able to cope quite adequately and securely with it.
Third, all the other OS stats combine current and previous versions of the OS together. Given that Windows 2000 is merely NT5, why should it get any different treatment?
Finally, go have a look at Attrition's website defacement stats for May 2001 so far (although Attrition are no longer mirroring defaced websites, they are still compiling statistics on defacements). Here NT and Windows 2000 are treated separately. You will notice that although NT is by far the most defaced, Windows 2000 comes second with some 29.55% of all defacings (all this information correct at time of writing). This compares to a total of 8.99% of all defacings for combined versions of Linux. This is a quite remarkable achievement for Windows 2000, to achieve this in just 18 months since its release - over 3 times the defacement rate of Linux. Well done Redmond!
Oh yes, for those of you who need a reality check about market share in the webserver market, this is the latest Netcraft survey. Sadly, the statistics by OS are not available without paying Netcraft (come on, we know it's the SSL survey that you make money from, please give us some hard OS information for non-SSL sites). However, it would be conservative to assume that approximately 60% of all Apache sites run Linux, and that figure still gives Linux twice the market share of NT and Windows 2000 combined. If we make another very conservative assumption, that Windows 2000 is half of that combined Microsoft figure (the following figures get worse for Windows 2000 the lower that share is), then we get this rather amazing figure:
Taking even very conservative estimates, a Windows 2000 webserver is currently at least 12 times more likely to be defaced than a Linux webserver.
I think that says it all.
This could be extremely BAD, think about if someone were to add, modify just a couple of lines of code. As long as people have been keeping original, master copies offsite it shouldn't be too dificult to get things back to normal.
But could you imaging though, if someone were to add/modify 4-5 lines off code and you didn't have another copy offsite? You would have to remember why every single line of code was there, you couldn't trust the comments telling you why this procedure is there.
For an example: It would be very easy to add code to add, "allow-anyone with this password to get a root shell" lines to a compromised box. Now lets say it's an ISP running an IMAP server they get off of Sourceforge, gets a new version. The program appears to run just like it's supposed to, they never would know that there is a 3 line, fork shell process code added into it. Being an ISP they have to have this program accessible to anyone anywhere, so they can't have this program behind a firewall. Along comes the wiley hacker who compromised the code to begin with... *poof* root shell on box.
That is the scary part, without being able to go back to a true "golden" state, EVERY single line of code has to be checked, without relying on possibly forged comments. Comparing MD5 hashes could tell you if the program has been modified, but some programs are modified hourly, most of these don't have checksum information.
Well you set it up and tell it what you are doing. It installs all the software etc and sets everything up at the WORST possable setting."
I think someone beat you to it.
-atrowe: Card-carrying Mensa member. I have no toleranse for stupidity.
As a side note, you're absolutely right. Security breaches are a fact of life on the Internet and no site is 100% secure. Just remember this the next time there's a story about a MS site getting cracked and try not to get all high and mighty about it because Linux, just like every other piece of software is NOT 100% secure.
-atrowe: Card-carrying Mensa member. I have no toleranse for stupidity.
If they didn't provide full source for it, we could bust 'em for violating the license ;).
+5:offtopic,but anti-American
When we yell about this site or that site getting cracked while using MS products, it is typically tied with a report explicitly stating that it was, infact, a MS product that had the security hole.
But right now, we don't know anything about this attack. We don't even know if software is to blame. It could be a bad admin, it could be a buggy kernal patch, it could be someone sniffed an admin's root password while they unknowingly used a cracked copy of MS Terminal on a Windows box...
Until we know what actually happend we can spin it any which way.
My
Signed distributions don't really protect against this - you pretty much have to audit all the changes since the last distribution before security was compromised, assuming you know when security was compromised. If you don't know when, then it's probably wise to audit the entire source for your project.
The solution to this kind of problem does involve digital signing, though - digital signing of each code check-in by the author. This way you have accountability for each check-in, and you can just prune out all the "anonymous" check-ins. As long as you keep your private key safe on your dev box, compromises to the repository can't be undetectable (though they can go undetected if you're not looking for them).
I wonder how hard it would be to modify CVS to add a little delta-signing? Might not be possible if CVS uses reverse deltas (like most repositories do).
Basically you want to be able to run a utility on the source tree that verifies the identity of each check-in by cryptographically verifying an actual file hash signature of some sort. It would need to maintain a collection of the public keys used in signing to do this, so you need some key exchange protocol. In order to verify the signature of each check-in, we need to know what the set of data is for each check in. Typically it is a "delta" (basically the output of diff), or a set of deltas for a merge.
The problem with signing the deltas is, most modern systems don't keep the deltas. Instead, they update the "tip" with your changes (this is an optimization, since the tip is the most requested version (with the exception of our waitress the other day, who said "I don't want the tip, baby, I want the whole thing!")), and then generate "reverse deltas" which are applied to the tip version if you need to go back in time.
On the other hand, the forward deltas can probably be computed from the reverse deltas, so signatures on normal deltas could be verified with some computation. Alternately, the reverse deltas could be computed by the client (this is probably bad performance-wise) and signed.
Better yet, since we're client-server-ish with CVS, you could submit your changes, the server could hash the set of reverse deltas after it's computed them, send the hash back to you, and you could sign (encrypt with your private key) the hash and send it back.
That way, there is a quick way to verify who made each set of changes, without too much network bandwidth lost. To verify, the server can take the hash of the existing set of reverse deltas from a particular merge, then use your public key to decrypt the "signature" hash, and compare. If they are the same, then the delta files have not been tampered with.
The question with this approach is, how do you protect the tip? In a reverse delta system, protecting the deltas is only half the battle (well a lot of things are half the battle! Losing is half the battle!) If you can't verify the authenticity of the tip, then an intruder can modify the tip and not touch the deltas, and you couldn't detect it. That's bad. I think in order to protect the tip, in addition to signing the hashes of the deltas, you should also sign a hash generated from the full and updated versions of all the files you touched with your merge. This way we should be able to verify both the tip versions and all the deltas, and have some accountability (we can associate 'nyms with each public key we have).
Sounds like you would need to alter the pserver protocol, add some hashing to the post-merge processing, and update the clients with some GPG code so they could sign hashes.
Any volunteers? ;)
Phil
Sourceforge really should provide more details about how security was breached. They probably are reasoning that if they give details, other people will do the same thing. However, anyone who's interested in this sort of nonsense probably already knows.
So, why not let the legitimate Sourceforge users have a bit more information about what happened? Perhaps some of the users might have an idea for a fix, or at least a way to protect their own work. Heck, we don't even know the nature of the breach, do we? Was data stolen? Corrupted? What? Inquiring minds have a legitimate need to know.
I'm the stranger...posting to