Slashdot Mirror

← Back to Stories (view on slashdot.org)

Two FreeBSD Project Servers Hacked

Posted by samzenpus on from the protect-ya-neck dept.

hypnosec writes "The FreeBSD project has suffered a security breach. Hackers have successfully compromised servers that were part of the infrastructure used to build third-party software packages. The Security team over at the FreeBSD project is of the opinion that hackers were able to gain access to the servers using legitimate SSH keys and not by exploiting any operating system vulnerabilities. Instances of intrusion were first detected on November 11. FreeBSD project, through a message on public announcements mailing list said that the security breach hasn't affected the project's core components like kernel or system libraries but, has affected third-party software packages being distributed by the project."

46 comments

  1. Yes, I read /. on Saturday by alphatel · · Score: 5, Informative

    This was already submitted two days ago.
    New article link merely references the material already posted by freebsd on Nov 17th.

    --
    When the foot seeks the place of the head, the line is crossed. Know your place. Keep your place. Be a shoe.
    1. Re:Yes, I read /. on Saturday by Jeremiah+Cornelius · · Score: 5, Funny

      Dupe, dupe, dupe,
      Dupe of URL
      Dupe, dupe,
      Dupe of URL
      Yes, oh, I, I'm gonna link you
      Nothing can stop me now
      'Cause I'm the Dupe of URL...

      --
      "Flyin' in just a sweet place,
      Never been known to fail..."
    2. Re:Yes, I read /. on Saturday by kwerle · · Score: 2

      Sigh. I was actually hoping for new information. Instead we're left with "/. editors can't scrub for dupes." Which we all knew already.

    3. Re:Yes, I read /. on Saturday by LurkerXXX · · Score: 1

      Yes, but it was posted bt Timothy, who I and many others block all stories from (because he's an idiot that doesn't bother to look at the pages he links to, or in general give any other thought to posting), so at least it's new to some of us.

    4. Re:Yes, I read /. on Saturday by Anonymous Coward · · Score: 0

      FreeBSD has still been hacked.

  2. at least its 36 hours since the original posting by Lawrence_Bird · · Score: 3, Informative

    Posted by timothy on Saturday November 17, @09:22AM
    from the happy-transparency dept.

  3. Has anyone found out how they got the keys yet? by Viol8 · · Score: 1

    They're not something you can guess. Someone with access to those systems either was careless with them, let someone else use their account and they were stolen or its an inside job and they're simply trying to make it look like it was external hackers.

    1. Re:Has anyone found out how they got the keys yet? by zeroryoko1974 · · Score: 1

      Probably tricked someone into giving them up. Your security chain is only as strong as the weakest link

    2. Re:Has anyone found out how they got the keys yet? by Idbar · · Score: 3, Funny

      Probably someone left the keys in a bar in San Francisco. Isn't that the way it works these days?

    3. Re:Has anyone found out how they got the keys yet? by dkleinsc · · Score: 1

      My guess:
      1. Somebody who legitimately has the keys put them on a cell phone or laptop.
      2. Somebody else pwns that device (because it's not running a super-secure OS), sees the keys.
      3. The person with access doesn't know he's been hacked, or doesn't want to admit it, so the rest of the organization doesn't get notified and can't change the keys.
      4. Voila, easy access to FreeBSD's servers.

      That's one of the standard techniques in getting around security: You target the relatively insecure partner with legitimate access to your real target.

      --
      I am officially gone from /. Long live http://www.soylentnews.com/
    4. Re:Has anyone found out how they got the keys yet? by Anonymous Coward · · Score: 1

      It concerns me that so many people (lots of people on forums, Slashdot, and a few of my own peers) are focused on how the person's SSH private keys were obtained. It doesn't matter how the keys were obtained -- truly it doesn't. You have to assume those keys are going to be obtainable. Most people keep their private keys stored on their workstation or laptop, or on a USB flash drive; laptops get stolen, USB flash drives get stolen or lost, workstations get compromised, and so on. Given this, there's no real way to guarantee someone won't get your SSH private key, and this is exactly why a private key should be password-protected. So unless a keypress logger was installed on the person's machine whose keys were stolen, I have to assume the SSH key in question was passwordless, in which case the fault here is with 1) sshd being set up to allow passwordless keys, and 2) the user using a passwordless key to begin with. (Also, for those considering a rebuttal along the lines of "passwords are annoying with SSH keys, you have to enter them every time" -- look into this amazing thing called an SSH agent).

      The things that concern me most about this breach:

      1. What I mentioned above,

      2. Why two package-build cluster machines (which may or may not allow non-root users to write to the official package repository -- see #3) are connected directly to the Internet without (presumably) any firewall rules permitting SSH access from specific IP addresses (I can guess this is because most of the FreeBSD devs tend to roam internationally, but that's no excuse when it comes to security -- there are ways to solve this, both technical and social),

      3. Why the account's access level was not stated in the disclosure. Did he/she have root or not? It matters -- if they didn't have root, why would there be any concern over the integrity of the packages on the official package FTP server? Unless, of course, somehow that non-root-level account had the ability to remove/overwrite/replace those packages on the official package server directly, which to me indicates a much bigger problem than the breach itself,

      4. Why the FreeBSD Project members said nothing officially or in an official capacity until November 17th. Someone (I'm assuming Peter Wemm) made the decision to down most (but not all) of the related services on the 11th. Someone on the PR side (in this case, probably FreeBSD Core, or possibly Ken Smith) should have sent an Email to -announce or -stable or -questions, posted something on the freebsd.org website, RSS feed, or even Twitter, as well as the official FreeBSD forums, stating something to the effect of "we've shut off the SVN-to-CVS gateway for an investigative matter; you won't see ports/src/etc. updates until we turn it back on. We've also shut off the portsnap master. csup/cvsup, portsnap, freebsd-update, native CVS, and GNATS are all impacted. We'll provide more detail when we have it, rest assured".

      I appreciate public disclosure of the breach (even though it leaves many questions), but WRT #4, a 5-day outage without any official announcement is completely unacceptable. There are already FreeBSD users on the mailing lists complaining about this fact, and so far not a single Project member has responded in official capacity to those concerns.

      I'm also choosing to exclude a 5th item from my list, which is why 2 or 3 key FreeBSD Project members tried to sweep this under the rug by stating on mailing lists it was "maintenance". That indicates people were either in the know but were mum about it, or didn't know and were spouting off speculative nonsense (both of which are unacceptable in this situation).

      What Timothy's news item, as well as yesterday's news item, lacked was what FreeBSD services were impacted and for how long. I submit a Slashdot story (which never made it past the firehose approval process, not sure why) which disclosed what all services were impacted:

    5. Re:Has anyone found out how they got the keys yet? by icebike · · Score: 1

      Look hard enough and you will find a conspiracy.

      It seems just as likely to me that forensics required a certain period of silence while packages were checked against backup sources.

      What could they announce on the 11th that would have made you happy? WE'VE BEEN BREACHED!! (perhaps add two or three more exclamation marks). Then what. 10 thousand questions, phone calls, and emails, distracting them from the task at hand?
      You know that even you would be demanding more answers if they posted exactly what you asked for in #4.

      The 5 day outage was TOTALLY acceptable and appropriate. If your systems can't run for 5 days without contact with the mother ship, then you need a different system. The Silence was acceptable too.

      Mountain / Molehill.

      --
      Sig Battery depleted. Reverting to safe mode.
    6. Re:Has anyone found out how they got the keys yet? by icebike · · Score: 1

      Well even having found a cell phone with ssh keys on it doesn't gain you access unless the ssh keys themselves have no passphrase.

      This use to be a fairly common practice (unfortunately) when key caching agents were not available and every single transfer over ssh required yet another entry of your ssh passphrase.

      If no passphrase was used on the keys, simply walking away from your workstation for two minutes allows an untrustworth co-worker to email your entire .ssh directory to himself at some obscure mail site, or copy them to usb.

      By far the likeliest situation is a ssh key with no passphrase.

      --
      Sig Battery depleted. Reverting to safe mode.
    7. Re:Has anyone found out how they got the keys yet? by Anonymous Coward · · Score: 0

      There is no conspiracy theory in my statement. There is a hinted conspiracy theory in the OP's statement, but I do not share that sentiment in the least.

      I already stated what could have been announced (to mailing lists, RSS feed, Twitter, and the forums) on the 11th that was appropriate:

      "We've shut off the SVN-to-CVS gateway for an investigative matter; you won't see ports/src/etc. updates until we turn it back on. We've also shut off the portsnap master. csup/cvsup, portsnap, freebsd-update, native CVS, and GNATS are all impacted. We'll provide more detail when we have it, rest assured."

      The outage duration is only a problem when nothing official was stated anywhere and the community was being kept in the dark. The fact that nothing official was stated about said services being taken down (i.e. what was going on), and what *was* stated by key FreeBSD Project members was in response to user inquiries (vs. something official, i.e. an announcement) was either flat out wrong or vague as hell, is something that needs to be addressed for the future (see very last paragraph for a mirrored opinion). I can provide references to every one of those Project members' statements if you want to review them.

      Rephrased: users had no idea why commits weren't being picked up by csup/cvsup or by portsnap, and at least one user was frightened to find the freebsd-update servers were completely unreachable.

      The concern items I presented are factual and based on what actually transpired. Please cease the "what if" and "predictive" straw man arguments.

    8. Re:Has anyone found out how they got the keys yet? by Anonymous Coward · · Score: 0

      So unless a keypress logger was installed on the person's machine whose keys were stolen, I have to assume the SSH key in question was passwordless

      Or they simply connected to ssh-agent and had it do the authentication as usual. If the attack is well-planned, even just logging in once into the remote machine can be enough. Or they got root on the client and got everything needed out of the ssh-agent process. Also, getting a keylogger on a system that runs XWindows is not as hard as it may sound. You don't even need root to get the keylogger running and you can get everything that user types in any X application, including terminals and the unlock screen where the user's password is typed.

      sshd being set up to allow passwordless keys

      Can the ssh server distinguish between the client's private key being password-protected or not?

    9. Re:Has anyone found out how they got the keys yet? by Anonymous Coward · · Score: 0

      So unless a keypress logger was installed on the person's machine whose keys were stolen, I have to assume the SSH key in question was passwordless, in which case the fault here is with 1) sshd being set up to allow passwordless keys, and 2) the user using a passwordless key to begin with.

      How does the sshd (on the server side) check if your private key (only available on the client side) have a password or not?

  4. Should have run on OpenBSD by HaeMaker · · Score: 1

    "Only two remote holes in the default install, in a heck of a long time!"

    1. Re:Should have run on OpenBSD by Zemplar · · Score: 3, Insightful

      "Only two remote holes in the default install, in a heck of a long time!"

      A security breech using legitimate authentication credentials is not a remote hole.

    2. Re:Should have run on OpenBSD by Anonymous Coward · · Score: 0

      Yeah, it is something much worse -- it does clearly describe the lack of fundamental security procedures being applied within the infrastructure.

      This is why security is a process that does require constant evaluation and strong base.

      Luke Piatek

  5. Sooo... by Anonymous Coward · · Score: 0

    Hackers did this, not crackers?

    1. Re:Sooo... by Anonymous Coward · · Score: 0

      Bush did it.

  6. What Is The Systematic Fix ? by Anonymous Coward · · Score: 0

    This event and the kernel.org breach raise the question "how can this type of events be avoided in the future ?".

    There are some very tough questions:

    A) Where does the code signing take place ? On a random developer's machine ? Or, the random developer machine uploads "to a safe machine" where the signing is done ? That still leaves the issue of an attacker modifying code on the developer machine without the developer knowing of that.

    B) What about attacks on "secure, centralized" machines ? They are a very juicy target for the $government, for the Russki Mafia, the Chicoms and for Israel Ltd. Don't expect those parties to hesitate breaking into buildings, including secured buildings.

    These questions lead to:

    C) Shouldn't all new pieces of code and all patches be signed by multiple competent persons after a proper review ? Similar to the GPG Web Of Trust ?

    I am not saying I full understand the problem or have a "silver bullet" solution - just a start of discussion about systematic fixes.

  7. Still suspect this could have something to do with by RocketRabbit · · Score: 1

    Still suspect this could have something to do with the SSL backdoor allegations made a while back. http://www.mail-archive.com/full-disclosure@lists.grok.org.uk/msg47029.html

    Yes I know the allegations have largely just petered out over time, but this doesn't allay my suspicion.

  8. So ?? by Anonymous Coward · · Score: 0

    Adobe has been penetrated through all of its bodily openings multiple times. Just Last month. There are lots of Mitnick-style stories about DEC and SUN, both of whom were once "leaders" in the minicomputer/mainframe (read: Unix, VMS) business.

    Humanity learns from mistakes and you bet the free software community won't sleep until we have found processes which make manipulations of source code much, much harder. To the point of not being practically doable.

    Meanwhile $corporation will rely on "shut up about our broken build/signing process or we will fire you $developer !"

  9. "Passphrases" by Anonymous Coward · · Score: 0

    ..are a shitty concept, as attackers have enormous compute resources these days, combined with sophisticated dictionary attack software. $100 from a creditcard can buy you serious compute power on AWS.

    How do you know people chose "good" pass phrases ?

    1. Re:"Passphrases" by icebike · · Score: 2

      Its as easy as simply running a dictionary attack.
      You can't tell a pasphrase protected private key from an unprotected one. Both are gibberish. You would never know when you
      decoded it correctly unless you try to use it.

      Each dictionary attack attempt will have to be tried via an attempted log in to either the target site or a replicate there of.

      But, hey, we are all ears if you have a better method. People have only been looking for one for something like 20 years. You can be a hero.

      --
      Sig Battery depleted. Reverting to safe mode.
    2. Re:"Passphrases" by fnj · · Score: 1

      ..are a shitty concept, as attackers have enormous compute resources these days, combined with sophisticated dictionary attack software. $100 from a creditcard can buy you serious compute power on AWS.

      Perhaps you're not very good at statistical analysis, or you just haven't gotten the message. Yeah, they're not a magic bullet. You have to pick a decent one. But you can remember a HELL OF A LOT more entropy in the form of a phrase than you can in the form of a nonsense character string.

      Of course you don't KNOW they are using a "good" passphrase. You don't know they haven't written down a password, either. Like icebike says, maybe you have a magic solution that works with no brain or discipline required. Let's hear it, genius. Yeah, I guess I just insulted an anonymous idiot.

    3. Re:"Passphrases" by smash · · Score: 1

      A passphrase is better than NO passphrase.

      --
      I run: Windows, OS X, Linux, FreeBSD. Just because you have a hammer, doesn't mean everything is a nail.
  10. OpenBSD would have been worse by Onymous+Coward · · Score: 1

    Indeed.

    And this is why there's an essay on how OpenBSD is insecure. Because security breaches do happen. You need to lock your systems down against internal attackers as well. Defense in depth and all that. Not just hermetically sealing your system by abstaining from running any services at all.

  11. **** netcraft is wrong! by Anonymous Coward · · Score: 0

    It is not dead, it just smells that way!
    But seriously, both remaining users of FreeToRunBinaryBlobsInTheKernelAndAdoptCrazyChilliNamedPermissionSchemesBSD were very upset by the breach. They only noticed something was wrong because of actual disk activity (usually there is none for this project).

    1. Re:**** netcraft is wrong! by Anonymous Coward · · Score: 0

      Stupid asshole.

    2. Re:**** netcraft is wrong! by Anonymous Coward · · Score: 0

      Now now, that's no way to talk with mom.

  12. Commie hackers hate genuinely free software by Anonymous Coward · · Score: 0

    niggers did it.

    Naughty Ignorant Gnu/Gpl-Evangelist Retard Shitheads?

    Nincompoops Inspired by Grotesque Grossness of Evil Richard Stallman?

    Nefarious International Government Gatekeepers Eradicating Restrictionless Software?

    Possibly...

    --libman

  13. Malice vs Incompetence by Anonymous Coward · · Score: 0

    From what I read out of your references (and one level deeper), it appears that *BSD security practices aren't that good actually. Once greek guy develops some "special" code to some degree of completeness and other people "just integrate" it into the main codebase. What kind of shit is that ? They accept code into their code base which they don't properly understand themselves. That would be typical of Adobe Inc. but not of a "super secure OS".

    Maybe *BSD isn't so secure after all. Because of incompetence. Read from that Mr Napoleon of what he thought about malice and incompetence.

    1. Re:Malice vs Incompetence by RocketRabbit · · Score: 2

      Linux, Windows, OS X, and Solaris all use the BSD SSL code, or very close derivations of it. If the BSD coders are lazy, then the coders responsible for the above-mentioned OSs are even worse, right?

  14. You Can't by Anonymous Coward · · Score: 0

    If a single SSH or GPG key is critical for the security of the source code of an OS, then that OS is doomed. $government and $mafia will pwn that system - very physically if required.

  15. Re:at least its 36 hours since the original postin by Anonymous Coward · · Score: 0

    Apparently slashdot employees don't talk to one another.

  16. Damn pirates by bursch-X · · Score: 2

    And the worst: They stole all the source code and pirated BSD!!!!

    --
    There are two rules for success:
    1. Never tell everything you know.
    1. Re:Damn pirates by Anonymous Coward · · Score: 0

      The worst would be if they injected code into the code base for some nefarious purpose.. hidden spyware for example.

  17. "hacked" by smash · · Score: 1

    It's not really a hack if you log in with legitimate credentials. Compromised, yes. Hacked? No.

    --
    I run: Windows, OS X, Linux, FreeBSD. Just because you have a hammer, doesn't mean everything is a nail.
  18. Indeed by Anonymous Coward · · Score: 0

    Properly inferred.

  19. More Like by Anonymous Coward · · Score: 0

    ...$government BACKDOOR. Along the lines of "when that bit is set in that IPSEC header, we can inject our Surveillanceware as a binary into the *BSD kernel".

  20. NOT True by Anonymous Coward · · Score: 0

    If I have the ability to intercept an encrypted session of a particular SSH key (say, via $government collection facility, SATCOM terminal, by a router I control, or WLAN) and I can obtain the passphrase-protected SSH key (PPSK), I can indeed run Massively Parallel dictionary attacks on PPSK:

    I WILL be alble to decided whether a certain passphrase is correct, simply as a function of {ciphertext, PPSK}.

    I can do this on as many CPUs as I my credit card buys on AWS. Or as many as my TLA has running in some remote bunker.

    That implies that a *single* Passphrase should not be enough to compromise the security of an OS' source code. See my other post "What Is The Systematic Fix ?"

  21. Please Refer To by Anonymous Coward · · Score: 0

    "NOT True" on this page.