FSF FTP Site Cracked, Looking for MD5 Sums

landley writes "The Free Software Foundation's FTP site at ftp.gnu.org has been "compromised", and they don't seem to have full backups. They've yanked a bunch of recent packages (and their whole alpha.gnu.org ftp site), and when I asked about it they responded 'Our FTP server was compromised, yes. We are beginning to find good MD5sums for files which have not yet been restored, and they will be available again Real Soon Now. If you can provide MD5sums for any of the files listed in MISSING-FILES, it would be very much appreciated.' " Update the FSF has a statement on the FTP site explaining the matter.

Comments

Correct MD5s

[Henry V .009]

Sure, I've got the "correct" MD5s right here. You trust me, don't you?

Re:Correct MD5s

[brechmos]

Yeah, but if enough people send in the same MD5 sums for each file, then it "should be" easy to confirm it is correct or not.

Surely, there aren't that many dishonest people, and if there were, then it would be hard for them all to get together and come up with the same MD5.

Re:Correct MD5s

[Henry V .009]

The man of the million email addresses replies: Are they confirming MD5s in person, or over the phone, or by other electronic means? You have yet to master the art of paranoia, grasshopper.

Finnishing move

[palad1]

After getting their FTP server rammed in the sockets, I bet the maintainers of ftp.gnu.org will be just more than happy to go through a good ol' slashdotting because someone _has_ to convert urls into hyperlinks for his /. submission.

I know, I clicked on the link :)

SCO

[Amon Re]

Hmm odd...one day they speak of taking sco support out of gcc, the next their ftp server gets comprised, interesting.

Obg.

[Rosonowski]

"Real men don't use backups, they post their stuff on a public ftp server and let the rest of the world make copies." - Linus Torvalds

Re:Obg.

[nolife]

My thoughts exactly, recently I've been using P2P to backup my music files.

I have the files

[Zabu]

But do to some sort of wierd computer problem my machine keeps on restarting...


I will get around to fixing it sometime next week.

BSD Ports trees should have them

[lactose99]

Taking a brief glance over my FreeBSD server, all of the entries in the Ports tree have the MD5SUMs in the "files" file. The Ports tree includes many many FSF software package installs.

Re:BSD Ports trees should have them

[lactose99]

Oops... its the "distinfo" file that contains the MD5SUMs, not "files".

Re:BSD Ports trees should have them

[mph]

As a port maintainer and committer, I can confirm what you say. The recorded md5 signatures are for the distributed source archive (e.g. from ftp.gnu.org, or Sourceforge, or whatever). They are there to ensure that the source has not been tampered with.

BSD-specific patches are then applied to the downloaded source, but have no implications for the md5 signature that's on file.

This is a conspiracy

[palad1]

When looking at the missing files: gnu/windows/emacs/21.2/leim-21.2-src.tar.gz gnu/windows/emacs/21.2/emacs-21.2-barebin-i386.tar .gz gnu/windows/emacs/21.2/emacs-21.2-bin-i386.tar.gz gnu/windows/emacs/21.2/emacs-21.2-fullbin-i386.tar .gz gnu/windows/emacs/21.2/emacs-21.2-leim.tar.gz gnu/windows/emacs/21.2/emacs-21.2-lisp.tar.gz gnu/windows/emacs/21.2/emacs-21.2-src.tar.gz gnu/windows/emacs/21.2/emacs-21.2-undumped-i386.ta r.gz

the list goes on abd on and...
now, grep for 'vi' : nothing, nada, null.

Of course, what do you think? This is a conspiracy orchestrated by VI lovers, to wipe out EMACS from the face of earth!

Re:Well that's good and all, but

[rkz]

Crackers exploited this vunerability, there was even a patch available!!

headline

[Lxy]

if you understand the headline

FSF FTP Site Cracked, Looking for MD5 Sums

You just might be a geek.

Re:headline

[wfberg]

if you understand the headline

FSF FTP Site Cracked, Looking for MD5 Sums

You just might be a geek.

The headline should have been simply

FSF ftp 0wn3d IM RMS teh md5sum's

Then the mainstream media would be all "OMFG WTF?! STFU /. I'm writing another MS Blaster story, bi0tch!"

This pisses me off more than it should.

[Deadbolt]

Okay, this kind of shit makes me want to start throwing bricks. Cracking the GNU FTP server? Is nothing sacred anymore? I feel like someone burned down a church.

They've done so much for humanity and some utter twit decides to compensate for his bad childhood by taking their server down.

*goes off to dock another point from his faith in humanity*

You're Kidding?

[System Control]

The Free Software Foundation's FTP site at ftp.gnu.org has been "compromised", and they don't seem to have full backups.

Unbelievable. And I'm supposed to trust their methods and products with my enterprise?

Re:You're Kidding?

[Lxy]

While your post is somewhat trollish, I have to agree that this is an interesting prediciment for the FSF. To save face, I hope they post a detailed account of how they were cracked, and own up to their mistakes so they can all teach us what not to do. That's the power of openness :-)

Re:You're Kidding?

[pongo000]

You mean, an accounting like this? Seems pretty detailed to me...

Re:You're Kidding?

[NoOneInParticular]

As some other posters in other threads noticed, the FSF does not have full backups because all backups made after early 2003 can be compromised. The crack happened in March, and what they miss is all the stuff that was uploaded after the crack. Backups from before March are available. In this situation no backup strategy at all would leave you with total security after March. The fact that the site was cracked five months ago is a bit scary though.

Re:ouch, saw this yesterday

[gearheadsmp]

Look no further than across the pond, my friend! Faster downloads than iBiblio, and it's run by this guy. So dig in!

If this had been an open source ftp server

[Stalemate]

We would already be flooded with posts about how if this were a Microsoft server we would already be flooded with posts bashing Microsoft and talking about....oh, right, my bad.

Re:Mirrors?

[gearheadsmp]

Mirror, mirror on the wall, who is the fastest of them all?

Re:the $64,000 question:

[hawkestein]

Or maybe, JUST FUCKING MAYBE , Linux isn't some sort of magical bug free OS where every buffer is checked, every race condition averted, and every service that runs on it is guaranteed bug free.

That would be OpenBSD. ;)

Complete md5sum

[Penguin]

$ md5sum complete-gnu.tgz
deadbeefdeadbeefdeadbeefdeadbeef complete-gnu.tgz

Re:the $64,000 question:

[saskwach]

Actually, this vulnerability had already been patched, just not on this particular server.

iSEC Security Research reports that wu-ftpd contains an off-by-one bug in the fb_realpath function which could be exploited by a logged-in user (local or anonymous) to gain root privileges. A demonstration exploit is reportedly available.

and patched August 31, 2003

Re:And in other news...

[iapetus]

Well, it will be as soon as they can remember the key combination for 'hack into VI web site' is. Now I know it's in here somewhere - is it M-~ h C-V...?

Re:the $64,000 question:

[iii_rjm]

No. The real $64,000 question is why didn't they have reliable backups and a disaster recovery plan

Re:the $64,000 question:

[Wuffle]

and patched August 31, 2003

I knew the open source community worked fast but that's just scary.

Re:Well that's good and all, but

[Uruk]

I'll bet that 90% (or more) of all break-ins are the result of problems that could have been patched. Yeah, it sucks that this happened to GNU, but they're only human. Last I heard, they only have one system administrator to handle all of their machines, including Savannah. I can understand that this happens from time to time. GNU has to be a relatively high profile target (such as for disgruntled BSD h4x0rs and so on) so cut them some slack. If you patch 40 machines 99.9% of the time, nobody remembers that, what they remember is that you got cracked on one tiny detail you missed.

At least they yanked the programs until they could verify that they were correct. That really was the only thing they could do. The lesson to take from this is that with computer security and auditing, nothing less than absolute perfection is necessary. And so long as human beings are doing the admin work, absolute perfection just isn't realistic. :)

Re:the $64,000 question:

[DunbarTheInept]

leaving out the profanities, this isn't flamebait

Duhhh. "If it wasn't for the flames, this wouldn't be a flame."

How Long

[jpmorgan]

How long was the server compromised and serving out possibly trojan-horse software before it was detected?

Also nicely demonstrates the pointlessness (and stupidity) of serving out your MD5sums from the same machine.

DARL! DARL!!

[pair-a-noyd]

Turn that pee-cee thing off and go to bed RIGHT NOW!

Yes mom.... /pull covers over head and laptop/

Re:Wait? I thought Linux was Secure??

[the_othergy]

the next time a virus is released that takes down 90% of Linux installs, and toasts most of the internet, let me know. Until then, your point isn't exactly valid

The next time a virus takes down 90% of Windows installs and toasts most of the internet, let ME know...

Though don't bother if it only toasts about 50% of Windows installs and bring down only a significant portion of the internet. That's becoming too common place.

Re:Well that's good and all, but

[bmj]

While I agree with the premise of the post, this is sort of thing that would get flamed to hell and back if the thread dealt with a Microsoft security breach (case in point, see yesterday's discussion about the RPC worm). According to that thread, being overworked, underpaid, or anything else is not an excuse for having an unpatched machine.

Re:the $64,000 question:

[prizog]

There are backups from before the crack.

If you want to give FSF $64,000, we could hire someone to implement a better plan. But we're not made of money.

RTFA: There *are* backups, and they *did* patch

[stewby18]

...The machine appears to have been cracked in March 2003, but we only very recently discovered the crack.
[snip]
(For the ptrace bug, an root-shell exploit available on 17 March 2003, and a working fix was not available on linux-kernel until the following week. Evidence found on the machine indicates that were cracked during that week.)
Given the nature of the compromise and the length of time the machine was compromised, we have spent the last few weeks verifying the integrity of the GNU source code stored on gnuftp. Most of this work is done, and the remaining work is primarily for files that were uploaded since early 2003, as our backups from that period could also theoretically be compromised.

(emphasis added). So in other words, they were cracked in the brief space between the exploit post and the patch, and didn't find it right away. Now, they are carefully vetting all their backups from that period to remove any possibility that a compromised backup could be redistributed.

So, to answer your poorly-researched questions:

• They have reliable backups of everything, except for those files which, due to their upload time, cannot possibly be considered secure
• They are systematically verifying the reliability of the files where there could be any doubt

Which part of this would you not consider a disaster recovery plan?

FTP (the protocol) is NOT the problem.

[MartinG]

ftp as a protocol is far simpler to implement than ssh2 for example, so if you have no authentication to do, use ftp.

Using ssl is good if you have eg. passwords to hide, but other than that it just introduces complexity. more complexity tends to mean more possibility for bugs, which means more possible exploits.

However, don't use bloated, over-complicated stuff like wuftpd etc. something like vsftpd is /much/ better. its very simple and designed from scratch to be secure above all else. afaik it has never had a security bug found, and I would say is as close to secure as it is possible to be.

FSF systems

[devphil]

They do have more than one sysadmin, but none of them are full-time, I believe.

There are also some "interesting" schools of thought regarding security over in gnu.org land, and I'm sure there's tension between them as well. For example, savannah has to have some level of security, but their shell machine (not savannha) has almost zero "sysadmin-added" security: important configuration files are world-writable[*], because RMS doesn't believe in restricting individual actions of users on that machine. The only security is what's provided by the default installation, minus the world-writabilities.

So it should come as no suprise that the shell machine has been compromised multiple times. All from local users exploiting holes. The most recent was done in April, but they didn't find out about it until a few weeks ago. They're still recreating accounts.

I don't know about the ftp machine; I assume it's neither the same system as savannah nor the shell box. But it wouldn't surprise me to find the same situation: some important people gnu.org don't believe in locking down machines, some important people do, but (gripping hand) it almost doesn't matter because none of them have the time to do so.

(If you wonder why the GCC manuals, web pages, etc, on {savannha,www,ftp}.gnu.org are occasionally out of date, it's because gcc.gnu.org (the master) is not admin'd by the same group. Events like this are why it's not admin'd by the same group.)

[*] Backups are done by having little Emacs hooks in comments in the files. When you edit the file -- and of COURSE you're using GNU/FSF Emacs, not XEmacs or any other editor in the world, cuz it's a gnu.org machine -- Emacs knows to make backup copies. I have no idea whether real backups are done, or how.

Re:Status update from FSF on GNU FTP site crack

[bkuhn]

Yes, the crack was carried out by a local user. We don't know if it was a social engineer or someone who compromised an existing account.

Re:the $64,000 question:

[vadim_t]

They shouldn't be.

If a bug in IIS causes a remote exploit then that's a bug in IIS, and that's it. Now, if there's a bug in the Windows TCP/IP stack, networking components, some kernel call, etc, which causes an exploit then that *is* a bug in Windows.

A bug in wu-ftpd doesn't just affect Linux. It will also affect the other supported platforms: BSD/OS 1.1, and 3.1, FreeBSD 2.2.6, SCO OpenServer 5.x, SCO UnixWare 2.1, Solaris 2.4, 2.5.1 and 2.6, Sun Sparc Platforms, Solaris 2.6, Solaris 2.5.1, SunOS 4.1.4

The only real security vulnerabilities in Linux are the ones that affect only the kernel and Linux specific tools. Everything else is just a vulnerability in some other program.

Easy to point out someone else's mistakes

[ThePyro]

It's very easy to point out other people's "mistakes" like this, but I wonder how many people actually take all these various precautions that they're so quick to accuse others of not implementing?

The fools! They forgot to install a firewall!
The fools! They didn't purge all the old user accounts!
The fools! They didn't install the latest security patch! On all the boxes in the office!
The fools! They didn't require 10 character passwords, to be changed every 15 days!
The fools! They didn't update their virus definition files! Within the last 24 hours!
The fools! They didn't make triple-redundant off site backups!
The fools! They didn't have a plan C!
The fools! They don't know where their towel is!

Now granted, if you're being paid the big bucks to think about nothing but information security all day then all of these things should probably cross your mind... but I would be willing to bet that most people who are so quick and proud to show off their shiny, impenetrable suit of dragon scales have a soft vulnerable spot on their bellies.

WTF?

[MasTRE]

Neither the OP _nor_ the moderator think it important to note in front-page post that the box was compromised in _March_ 2003? Jeez, is this /. or -.?

Go easy on 'em...

[chuckw]

Yeesh guys, go easy on these people. They bust their asses every day for us. Their GPL enforcement queue is usually about 50 cases deep. They're on the phones and on capital hill every day educating and lobbying industry groups and politicians. Say what you will about the GPL, you don't even have to like it or agree with it and perhaps you even think RMS is a narrow minded prick (for the most part RMS isn't even involved in the day to day operations at the fsf). They are making life easier for all of us.

Rather than boast about all of the work they do, they quietly work behind the scenes just so you can play Monday morning quarterback. They have one fulltime systems administrator who is *INCREDIBLY* overworked. They are doing everything they can to keep the boat together. Last year they were over $315,000 in the red. Thanks to the FSF associate program and some skillful fundraising they're back in the black.

Want to help? Go get your FSF associate membership. It's not that expensive and it goes a long way towards helping to protect your freedoms.

Incidentally, this is also old news. They had MD5 sums verified, and the servers were patched up and back online almost two full weeks ago. None of the software was trojaned.

Who am I? Just another hacker who bothered to pay for an associate membership (#1142)...