Slashdot Mirror
FreeBSD Project Discloses Security Breach Via Stolen SSH Key
An anonymous reader writes "Following recent compromises of the Linux kernel.org and Sourceforge, the FreeBSD Project is now reporting that several machines have been broken into. After a brief outage, ftp.FreeBSD.org and other services appear to be back. The project announcement states that some deprecated services (e.g., cvsup) may be removed rather than restored. Users are advised to check for packages downloaded between certain dates and replace them, although not because known trojans have been found, but rather because the project has not yet been able to confirm that they could not exist. Apparently initial access was via a stolen SSH key, but fortunately the project's clusters were partitioned so that the effects were limited. The announcement contains more detailed information — and we are left wondering, would proprietary companies that get broken into so forthcoming? Should they be?"
If you run on freebsd, examine your tar and tar.gz
Access via ssh key, someone may have changed the tree
If you only use base release, power down and anti-freeze
For package add post 9/16, SVN and confirm you're clean
When the foot seeks the place of the head, the line is crossed. Know your place. Keep your place. Be a shoe.
and we are left wondering, would proprietary companies that get broken into so forthcoming?
I suspect most would not be so forthcoming.
Should they be?"
Yes.
Really do seem to know what they're doing, and are very proactive with their security.
I'm glad they openly announced this, how to deal with the breach for end-users, and also how they're dealing with it. (This coming from a proud FreeBSD server and desktop user)
(yes I use the Oxford comma.)
"...and we are left wondering, would proprietary companies that get broken into so forthcoming? Should they be?"
Short answer:
No, they do not want to scare the stockholders.
and... Yes, they should be because openness allows people to recover or protect themselves faster.
Silence is a state of mime.
...that any company which holds personally identifiable information (so that's all of them - it goes from CRM databases to employee records and payroll) has a Statutory obligation to register Company details with the Information Commissioner's Office and to report any breaches to the Information Commissioner.
For the definition of "breach", read: lost or stolen mobile phone, laptop, notepad, application or registration document, tablet, audio recording, video capture, or any other method, known or unknown, of recording personally identifiable information.
Operation Guillotine is in effect.
You don't seem to be aware that SSH keys are typically encrypted, and still require a password to unlock. Yes, some people foolishly enable passwordless use of SSH keys, but that does not reflect on the principle of SSH key login in general.
I think you mean unencrypted SSH keys...
Uhh, people use OpenSSH because it's free, it's everywhere, and they don't need any goddamn hardware token generators and other nonsense like that.
You security hardware guys have been pushing this crap since at least the 1980s. Seriously, it hasn't taken off in 25+ years because it's not practical and it's not what the public in general wants to deal with.
So give it a rest already! Aside from a few niche users, the public at large is not going to pay good money for a hardware token generator, and they sure as hell aren't going to waste their time using it!
You can password protect SSH keys. Furthermore, you can store them on an encrypted volume. Passwords can be bruteforced rather easily, because most people tend to use weak passwords. Bruteforcing an SSH server server that enforces PKI however... I guess the only way to get in is... to steal a user's key, which means you need physical access to it or the user has been really careless.
http://www.howtogeek.com/121650/how-to-secure-ssh-with-google-authenticators-two-factor-authentication/
Amen. Not to mention, if you lose your token, your screwed. To create a strong password that is easy for you to remember, follow these simple steps: Do not use personal information. You should never use personal information as a part of your password. It is very easy for someone to guess things like your last name, pet's name, child's birth date and other similar details. Do not use real words. There are tools available to help attackers guess your password. With today's computing power, it doesn't take long to try every word in the dictionary and find your password. Mix different character types. You can make a password much more secure by mixing different types of characters. Use some uppercase letters along with lowercase letters, numbers and even special characters such as '&' or '%'. Use a passphrase. Rather than trying to remember a password created using various character types which is also not a word from the dictionary, you can use a passphrase. Think up a sentence or a line from a song or poem that you like and create a password using the first letter from each word.
No matter how secure your system is (and SSH is very secure), if the individual using it is careless, the system will end up getting compromized.
Why would you use a stolen SSH key to announce a security breach?
From a recent security audit I participated in, you are mistaken. The number of SSH keys that were _not_ passphrase encrypted, in a typical multi-user environment, vastly exceeded the number that were encrypted. These keys were stored on an unsecured NFSv3 environment, and on poorly secured backup tapes. This configuration is common, and we even found several github and Sourceforge SSH keys for known participants in open source projects there.
While the number of security errors in those environments were quite large, they're quite commonplace. They are partly the result of the fact that SSH servers have no way of restricting users to the use of passphrase protected keys, and SSH key generators, especially those in the OpenSSH codebase, do not enforce the use of passphrase protected keys. (They issue a warning, but do not enforce the use of passphrase.) There are certainly tools available to help manage passphrase protected SSH keys. but even where available, they remain rarely used.
This is compounded by the lack of effective centralized management tools for SSH key access, and the nonexisting or recently implemented and rarely used expiration or revocation technologies for SSH. SSH should only be considered robust for protecting individual sessions from decryption. Its "key" technology should not be considered a robust authentication technology due to these commonplace flaws.
There are better general authentication approaches: SSH, both OpenSSH and SecureCRT's tool suites, now offer Kerberos authentication. This is a much safer technology, not vulnerable to the various "stolen passphrase free key" issues of normal SSH. Unfortunately, I've not seen any way for it to mesh well with SSH configurations that rely on the "ForceCommand" option being tuned for individual users and their SSH keys, especially source control technologies such as the "git" and "Subversion" and "CVS" access at Sourceforge.
Also, do _not_ rely on the commonly enforced use of excessively "Mix3d!Cas3". The reasons not to use this take some thought, but are actually well described in XKCD:
http://xkcd.com/936/
But the real reason is not there: it's that people frequently store such passwords in unencrypted, easily accessible locations such as a file called "passwords.doc" on their desktop, or send them via unencrypted email because they're too hard to remember or explain on a voice transmission.
or the user has been really careless.
Which is always the weakest link in any security system.
---- Booth was a patriot ----
Although this is a troll, there still is an unanswered question: how did the ssh key get stolen? While its nice to see that FreeBSD wasn't breached due to a vulnerability in *its* systems, someone obviously had a vulnerability in their system. To all the sysadmins out there, I think that's what keeps you up at night: How do you ensure that your users safeguard their secrets? Other than a "corporate policy" document imploring them to use "good judgement"?
Especially with BYOD coming into vogue, I think the security community needs to come up with a solution that is cross platform and easy to implement, verify and enforce.
Well.. maybe. Or Maybe not. But Definitely not sort of.
And so, when Microsoft gets raped by a bunch of hackers you think they are going to let the public know?
No, they are going to keep it under wraps.
No, they are *not* going to keep it under wraps, at least not if the break-in puts its users or customers at risk.
The reason is simple: Microsoft is required by law to disclose any such breach. The penalties for "keeping it under wraps" are severe and could include paying restitution/punitive damages to each individual customers/user.
But don't let such minor detail stand in the way of spewing your MHD all over slashdot.
Reading slashdot one-liner: (irm http://rss.slashdot.org/Slashdot/slashdot).rdf.item | fl title,desc*
Seriously? You are suggesting we leave passwords laying around in plain text in batch files, and go back to telnet???
Your method is only better in one single way. There is no worrying that you are the 0.0001% that did it wrong and are vulnerable. With your way, you KNOW you are already exploited!
would proprietary companies that get broken into so forthcoming? Should they be?
Yes, they are already required to
BTW, have we ever seen a satisfying explanation for what happened at kernel.org and linuxfoundation.org? We were initially told that it was something similar (stolen password/compromised user system), but AFAICT they have never explained how that could lead to the servers being root'ed. A rootkit *was* installed. That requires careless use of root privileges or an exploit of a privilege escalation vulnerability. Which was it?
Reading slashdot one-liner: (irm http://rss.slashdot.org/Slashdot/slashdot).rdf.item | fl title,desc*
No, we are not left wondering (unless one thinks that FreeBSD has a patent on especially leaky SSH developer keys) so instead we pretend that we are left wondering to justify hanging around and scribbling on the bathroom wall.
If Apple can't keep their mitts on an iPhone prototype and Google can't keep their mitts on a Nexus prototype, do you really think these butter-finger organizations have any better control over their developer's SSH keys?
This is compounded by the lack of effective centralized management tools for SSH key access ...
It works both ways. Precisely because there *is no* centralized control of SSH keys, my workplace cannot implement crazy password aging schemes, or demand at least one digit in each passphrase. End result: I take much better care of my ssh keys than of my plain login passwords.
I agree, everybody should get paid for their IPs. It's only fair damnit.
[t]here still is an unanswered question: how did the ssh key get stolen? While its nice to see that FreeBSD wasn't breached due to a vulnerability in *its* systems, someone obviously had a vulnerability in their system.
The explanation is simple enough, and provided on the compromise notice:
The compromise is believed to have occurred due to the leak of an SSH key from a developer who legitimately had access to the machines in question, and was not due to any vulnerability or code exploit within FreeBSD.
It only takes one instance of walking away from your workstation leaving it running to have a co-worker slip into your chair and email your .ssh directory to some obscure off shore email address, then remove the outgoing email from the "sent" list. A stolen phone, a purloined laptop, the possibilities are endless, although in the latter two instances you would expect revocations to be issued (assuming you had a backup copy somewhere)..
Once someone has your private key they ARE you, and it it was done without being immediately discovered, the key could linger in the wild for months or years.
Sig Battery depleted. Reverting to safe mode.
Yes! These things have finally gotten cheap enough (around $20) that those of us with access to a lot of servers ought to have one.
For those not in the know, these things look like a USB flash drive, but have more number-crunching power than storage. You load your SSH private key onto the USB fob and the key never leaves the device. Plug the fob into a USB port and ssh offloads the private-key RSA operations to the fob, which won't do anything unless you enter a PIN. As the private key never leaves the device, it can't be stolen without physically stealing the device or somehow hacking into the firmware. Using the fob requires software (opensc) on the computer you plug the fob in to, but to the server it's just a plain old SSH connection.
Obviously if you lose the fob you're screwed unless you have a backup copy of the key somewhere. The best option seems to be to generate the key on a PC with no network connectivity, save the key to the fob, and also save to removable media as an offline backup.
You probably should have read this...
"requiring notice to individuals when the security of their personal information has been compromised"
Those laws have nothing to do with a security breach of this sort if their own personal information isn't stored on the machine as well and in this context, the only people who would be notified **might** be the people writing the code. .
Shit happens. One intrusion through OpenSSH into two out of an entire cluster FreeBSD servers doesn't mean jack shit as to the overall security of using SSH as your authentication method. I'll continue to use SSH, and I'm sure pretty much anyone else who uses it now will see no reason to stop just because an encryption key was used as an entry point to a high-profile server. According to TFA, steps are being taken to prevent this from happening again by deprecating legacy services... and SSH doesn't look to be one of them. Besides... it appears that the SSH key wasn't "cracked" anyway, it just somehow made its way to someone it wasn't supposed to.
The compromise is believed to have occurred due to the leak of an SSH key from a developer who legitimately had access to the machines in question, and was not due to any vulnerability or code exploit within FreeBSD.
Good job FreeBSD, for not only disclosing this but also taking steps to prevent similar security breaches in the future.
Could you possibly state plainly and precisely what you mean. For Christ's sake, man, you are posting anonymously. Why be coy? Give us an allegation we can check.
I had an account on a machine in the same rack as kernel.org, and that machine was taken away for forensic analysis and still isn't back. Apparently (I don't do security research, but I work on a team that does) kernel.org contained the world's best collection of rootkits found to date, which was incredibly useful to people doing work in this area.
I am TheRaven on Soylent News
Whats the point encrypting private key, it would add hassle of typing password every time without real security benefit - if attacker got access to your account installing keylogger is not a problem, right? Just use common sense - do not store unencrypted backups of your keys.
and you makes 6.
Birds are not dinosaur descendants;birds are dinosaurs, for all useful meanings of "birds", "are" and "dinosaurs"
Main difference between the BSD license and the GPL license: one is from California and the other is from Massachusetts
I'm afraid that you are mistaken: your ignorance of the technology is widespread, and leads to precisely the behavior I described of leaving the SSH keys unencrypted and widely available.
Look into the "ssh-agent" tool, the wrappers for it, and the various system keychaiins on different operating systems. It may take thought to handle it for your particular environment, but the simplest approach from a common shell environment is below:
eval `ssh-agent1
echo $SSH_AUTH_SOCK
ssh-add -l
ssh-add $HOME/.ssh/id_dsa
ssh-add -l
You'll see that the key has been unlocked for any shells or system commands on that host that have the "SSH_AUTH_SOCK" set to access the running ssh-agent.
This is a very good point. One dumbass user who doesn't keep a passphrase on his private key, doesn't encrypt his hard drive, etc. and bam, you get hosed.
If you're on a current OpenSSH (as available in Red Hat 6.3 at least, or its rebuilds like Scientific Linux or CentOS), you can require both key and password auth. From the release notes at https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/6/html-single/6.3_Release_Notes/index.html#id3199604: /etc/ssh/sshd_config file to specify authentications that are required for a successful log in."
"SSH can now be set up to require multiple ways of authentication (whereas previously SSH allowed multiple ways of authentication of which only one was required for a successful login); for example, logging in to an SSH-enabled machine requires both a passphrase and a public key to be entered. The RequiredAuthentications1 and RequiredAuthentications2 options can be configured in the
To implement on an SSH server where only SSH protocol 2 is allowed, drop this in your /etc/ssh/sshd_config:
RequiredAuthentications2 publickey,password
You need to specify PasswordAuthentication yes as well, or you'll be told: "Invalid required authentication list"
Once you set it up, restart your sshd daemon, and you will be good to go.
Nothing's foolproof however, and I mean that in the literal sense of the word "foolproof". Some fool can store his password in plain text on the same system as his key, write his password on his computer in Magic Marker or whatever, and you're screwed again. Allowing SSH access to morons is a major security hole.
In Reason We Trust