The Low-Intensity, Brute-Force Zombies Are Back

Peter N. M. Hansteen writes "In real life, zombies feed off both weak minds and the weak passwords they choose. When the distributed brute-force attempts stopped abruptly after a couple of months of futile pounding on ssh servers, most of us thought they had seen sense and given up. Now, it seems that they have not; they are back. 'This can only mean that there were enough successful attempts at guessing people's weak passwords in the last round that our unknown perpetrators found it worthwhile to start another round. For all I know they may have been at it all along, probing other parts of the Internet ...' The article has some analysis and links to fresh log data."

Comments

why are passwords even allowed?

[rcpitt]

None of my systems allow passwords via ssh - and I run log-guardian.pl to "3 strikes - you're out" the idiots who do the brute-forces by putting them into iptables

Anyone with passwords turned on is not secure IMHO

Re:why are passwords even allowed?

[miknix]

IMHO if the passwords are strong enough there is nothing to worry about, unless you get pissed off with flooded log files and the waste of bandwidth.

None of my systems allow passwords via ssh

Exactly, using public key authentication and disabling PAM/Password authentication solves the problem.

and I run log-guardian.pl to "3 strikes - you're out" the idiots who do the brute-forces by putting them into iptables

sshguard is nice too, they will be firewalled in no time. (watch out for DoSs though)

However, it is not just ssh. Http and servers also suffer a lot from automated breakin attempts.
Anyway.. I'm glad services running on port 22 are not in the same security level of services in 23,137 and 138. Otherwise it would be the holocaust!!

Re:why are passwords even allowed?

[Wonko the Sane]

If you only allow public key authentication then there's really no need to bother blocking them: you'll never block the entire botnet.

Just let them try - they'll never guess the right private key.

Re:why are passwords even allowed?

[BitZtream]

grep -v for the win!

Re:why are passwords even allowed?

[palegray.net]

I don't allow password-based logins either (SSH keys only), allow SSH only from specific IP addresses, and I use fail2ban across all services that involve any kind of authentication (mail, ftp, http auth, etc). I've got it set to "two strikes and you're out"; every day I still get hundreds (some days thousands) of IPs banned in the logs. It's pretty sad.

Re:why are passwords even allowed?

[palegray.net]

tail -f for the bored!

Re:why are passwords even allowed?

[Thantik]

I would suspect it's going after port 22 only. If your smart enough to move the port from 22, your probably smart enough to use key pairs and then what is the point of trying to brute force you? Focusing on default 22 is a good strategy because you'll find those who have completely defaulted settings, weak passwords, etc.

Re:why are passwords even allowed?

[tagno25]

grep -v |tail -f for the smart and lazy

Re:why are passwords even allowed?

[dbIII]

Nice in the short term but giving people an easy way to add rules to your firewall may create hassles later once miscreants know that is what you are doing. Some people have scripts that implement temporary blocking so it doesn't hurt much on the day that some script kiddie decides to have fun with them by forging attacks from different addresses.

Re:why are passwords even allowed?

[palegray.net]

grep -v | tail -f running in a screen session for the smart and lazy who frequently suffer from denial of service attacks from botnets.

Re:why are passwords even allowed?

[dbIII]

From a quick look at fail2ban it looks like one of it's features is that the blocking only lasts until the next log rotation. Considering that attacks are temporary and automatic firewall rule generation could become a script kiddies playpen this is actually a good thing. If they work out you have this and spoof a few adresses as a denial of service attack the system will recover over time without needing someone to go through all the firewall rules.
I'm still a bit nervous about allowing malicious third parties to effectively write firewall rules for me. The ideal is of course to only allow a certain address range in, but some of us don't really know where the next legitimate connection is going to come from.

Re:why are passwords even allowed?

[The Redster!]

"aplay -t raw" for the truly over-the-edge!

Re:why are passwords even allowed?

[timmarhy]

passwords are still the most portable lightweigth authentication method out there, that's why. I had a system that got caught by this when a user with shell access set a weakish password. the user was sandboxed with only a limited whitelist of applications they could execute, so no harm done, but i did log all of the bot's attempts and the IRC channel it connected to along with passwords to other bots. it was very interesting to take the attack a part, at it's core it runs a simple dictionary attack and once it's in attempts to launches the same attack on the root password (no chance mine is 9+ random characters), plants a trojan in /temp (which failed in my case) and then connects to an IRC channel and awaits instructions which were pharsed using a simple perl script (which also failed). it then starts scanning for other victims using another perl script.

the activity on the channel indicated brazilian hackers. the thing that suprised me the most though was the amaturish and fragile nature of the process, instead of insulating the whole package so it could operate in a sandbox it atcually relied heavily on installed dependancies which i suspect means it was targeting the default install of one linux distro.

Re:why are passwords even allowed?

[al0ha]

>> Anyone with passwords turned on is not secure IMHO.

I disagree. I both cases, password auth or key auth, barring any security problem with the protocol, the weakest link is the user. A passphraseless ssh key is tempting to the user for many reasons and unless you audit the passphrases selected by your users, key auth is no more secure than password auth. In fact password auth may actually be more secure if you enforce complexity on the system(s). You have no control over the passphrase on a user's SSH key. You can try to control it, but the user can strip the password at any time and on any remote system.

Not allowing passwords may actually lull you into feeling secure. Multi-user and personal user systems are compromised all the time, and in my and my collegues experiences, break-ins via SSH are as likely to happen via either method. Lame botnets try weak passwords, crafty exploiters try everything and thanks to Phalanx2, their craft became much easier.

Poor Odds

[Nerdfest]

The odds of them getting into a system like this must be quite low, but I guess they're after the low-hanging fruit. Running your services on a high port rather than the default reduces this, as does disabling password login and using 2-factor authentication. Quite easy to do, and very, very secure.

Not seeing it yet

[MichaelSmith]

...unless they are only attacking from my existing list of blocked IP addresses.

How-to

[Nerdfest]

Sorry, should have posted this with the original. Instructions for Linux 2 factor authentication

Oh...

[Perseid]

...you mean zombie PROGRAMS. Damn.

[puts shotgun down]

Re:Oh...

[MichaelSmith]

[puts shotgun down]

Don't be so hasty. There are real people herding the zombies.

Protect yourself

[Matt Perry]

Use SSH keys in addition to passwords. Disable ssh root logins. Use the AllowUsers command in sshd_config to restrict what accounts can log in with ssh. Edit /etc/hosts.deny and add IP ranges for where you are unlikely to login from. Use iptables rules to block people who are hammering your ssh server from the same address. Use tools like Fail2ban and DenyHosts to block other abusers and share abuser information with other victims.

Re:Protect yourself

[anothy]

mostly good advice. you might consider using ssh keys instead of passwords, depending on your environment. the only thing i'd outright disagree with is pre-denying IP ranges based on a guess of where you're likely to log in from. i've had to leave the country on business unexpectedly on very short notice; it'd suck to have been locked out when i landed.

Re:Protect yourself

[GaryOlson]

Use VPN. Although it may seem redundant, SSH thru a VPN tunnel does provide a secondary access method which is secure.

SPA / PORT KNOCKING - Bye Bye Brute

[myspace-cn]

Roll out SPA / Port knocking, their IP shouldn't be touching your sensitive ports without a rule, table, or chain specifically allowing access. FORGET THE PASSWORD!

Another solution

[IceCreamGuy]

Use a script like denyhosts, and I'm sure there are a ton of others out there that are just as good if not better. Unless your password is weak enough to be guessed in five attempts and the attacker isn't already in the denyhosts list, you shouldn't have to worry about too much. And, most importantly, just peruse your auth logs every now and then, it's not really that big of a chore.

Not So Shameless Plug

[value_added]

For those already familiar with Peter Hansteen's website, I'll offer a Thumbs Up recommendation for his Book of PF.

There's already been several stories on Slashdot either submitted by or about him, and I don't recall any mention of his book. I'd say his efforts if not his humility deserve some kind of reward, and the reduced sale price of $19.77 is a bargain.

Denial

[the eric conspiracy]

I've used a script to block servers that failed a certain number of attempts along with AllowUsers. That worked well for a couple of years, but was annoying in that you could see the attempts being made and knowing that if you made a config error you could be vulnerable. It seems to me that even after I got several hundred systems in my block list it wasn't making a difference since the pool of zombies was so large.

Now I just use key only access and AuthUsers and feel a lot more secure. I'm thinking I may add a white list of IP addresses as well. That would really lock things down pretty well.

In all seriousness

[actionbastard]

This has been going on for years. Really. I've been seeing this crap in my logs since we started running an Internet-facing SSH host nearly ten years ago. It's always the same password based login attempts with the same dictionary/script used in the attacks. This is probably just some training exercise for Chinese hackers at some state-run school to see who can break into the running-dog Yankee Imperialist's computers the fastest.

I'm safe...

[hoytak]

I've now changed my password from Thomas to ThomasX, where X is a digit that I'm not telling.

Re:I'm safe...

[Main Gauche]

pinkie? C'mon, gimme a hint.

Re:I'm safe...

[spartacus_prime]

I tried to make my password "penis," but it said it wasn't long enough. :(

Re:I'm safe...

[Phroggy]

reference

Don't just run it on a higher port.

[Anonymous Coward]

Be proactive on port 22 as well. At the advice of another comment I saw on /. a year or so ago I'm running a honeypot, with three static ports (one of them 22) and 4 roving ports. Establishing a TCP connection to any of them causes your IP to be instantly added to an iptables blacklist. It's worked pretty well; I've got about 2-3 unique addresses trying per day, and about 294 have been blocked since mid-December 2008. It takes care of both port scanners and bots deliberately connecting in order to try and get root on my server.

Of course, the only way to get root on the server anyway is through a Yubikey OTP or the 64-character randomly generated password I have on an encrypted partition somewhere in case my Yubikey is ever fried/stolen/lost.

Re:My server got attacked last Thursday

[mellon]

Um. You realize, of course, that remote desktop is a lot less secure than ssh, right?

It doesn't matter if people are trying to pick the lock on your door. What matters is whether they can pick the lock. Use RSA-based authentication, and no amount of brute force is going to improve the odds of their breaking in to the point where it's worth bothering.

Remote desktop, on the other hand, is completely brute-forceable. If you're not seeing brute force attacks, it's because nobody's bothering, not because you're not vulnerable.

quickest fix

[capn_nemo]

While it's true there are a variety of techniques that can increase security, I've found simply moving to a new high-numbered port eliminates all random login attempts. Yes, security through obscurity is all it's cracked up to be, but for now, I've eliminated the problem with a pretty quick fix.

Goodness.

[geekboy642]

There sure are a lot of people who didn't bother to read the article.
The point of these attacks are that it's a coordinated botnet attack. Meaning if you block any single IP, or even a large subnet, you've cost the attacker nothing. Fail2ban, denyhosts, all of these won't even slow these attacks down.

iptables goodness

[grasshoppa]

Once again, we have a built in linux goody which helps us out;
-A INPUT -p tcp -m tcp --dport 22 --tcp-flags SYN,RST,ACK SYN -m recent --set --name sshattack --rsource
-A INPUT -p tcp -m tcp --dport 22 --tcp-flags SYN,RST,ACK SYN -m recent --rcheck --seconds 300 --hitcount 3 --name sshattack --rsource -j LOG --log-prefix "SSH Drop: "
-A INPUT -p tcp -m tcp --dport 22 --tcp-flags SYN,RST,ACK SYN -m recent --rcheck --seconds 300 --hitcount 3 --name sshattack --rsource -j REJECT --reject-with tcp-reset
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT

The above allows three connections in a 5 minute period to port 22. After that it rejects any further connection attempts until the 5 minute timer is up.

Re:iptables goodness

[wytcld]

Which means this won't work, since what I see in the logs (at least in the last go-round with this stuff) is one attempt per remote IP, yet a coordinated distributed dictionary attack. What we need is a ruleset that properly spots this attack pattern, then locks down port 22 entirely except for defined, fixed-IP administrative exceptions, just if there is any pattern of failed attempts over a longer period.

In a slightly-different recent case, I saw someone get ahold of an ftp login. The uses of that login were one-off per remote IP. The IPs were in Ukraine, Russia, China, Texas ... with hours between successive attacks. From what was attempted on the logins, it was obviously a generic attack, not well-directed at the particular server. Quite likely an infected client system divulged the login to the botnet that infected it, and the botnet then focused the distributed attack. As with ftp, so with ssh. They don't want you to even notice it in the logs. Each IP will be used for one transaction attempt with you, spaced hours apart.

We need tools to spot these slow-moving distributed attacks. The single bad login attempt needs to be correlated with others from completely different IPs, over a span of days, not just minutes.

Re:iptables goodness

[Qzukk]

then locks down port 22 entirely except for defined, fixed-IP administrative exceptions

This is my default configuration, with a port knock pattern for emergency access.

Re:iptables goodness

[drcesteffen]

Wouldn't it be better to TARPIT them rather than --reject-with tcp-reset? That said, if they are generating one query from each IP address and trying to log in more than 300 seconds apart this may have no effect. Perhaps all firewalls without ssh enabled should TARPIT attempts to connect to slow this down. Perhaps one could create a script on all computers to change the ssh port, assumes you are using a nonstandard high port, as a function of time and to tarpit all other ports.

Post your banlist

[UnRDJ]

I get about 5 of these a day, on a relatively small site. I wrote a small shell script out of sheer boredom that parses hosts.deny and gives me country and hostname info. Here's the output from the past week or so. It seems to confirm that most of these are from public isps overseas.

117.21.249.75 CN, China
121.166.163.142 KR, Korea, Republic of
122.128.36.3 KR, Korea, Republic of
189.19.245.182 BR, Brazil 189-19-245-182.dsl.telesp.net.br.
200.111.157.187 CL, Chile
201.6.124.155 BR, Brazil c9067c9b.static.spo.virtua.com.br.
202.229.120.74 JP, Japan
203.125.51.23 SG, Singapore
207.5.149.188 US, United States 149-188.suscom-maine.net.
211.87.224.190 CN, China
211.99.203.168 CN, China
216.164.162.155 US, United States 216-164-162-155.pa.subnet.cable.rcn.com.
217.219.67.86 IR, Iran, Islamic Republic of
218.241.177.241 CN, China
219.121.10.35 JP, Japan m010035.ppp.asahi-net.or.jp.
221.3.131.110 CN, China
61.184.136.164 CN, China
61.191.53.99 CN, China 99.53.191.61.broad.static.hf.ah.cndata.com.
77.79.88.247 TR, Turkey reverse-77-79-88-247.grid.com.tr.
80.179.149.122 IL, Israel 122.sharatim.co.il.
82.165.27.220 DE, Germany p15173261.pureserver.info.
83.19.20.66 PL, Poland byq66.internetdsl.tpnet.pl.
84.53.78.183 NL, Netherlands 84-53-78-183.wxdsl.nl.
87.197.110.47 SK, Slovakia static-dsl-47.87-197-110.telecom.sk.
99.53.191.61 --, N/A adsl-99-53-191-61.dsl.mtry01.sbcglobal.net.
200.219.194.74 BR, Brazil static.200.219.194.74.datacenter1.com.br.
202.190.177.144 MY, Malaysia
164.77.195.60 CL, Chile
89.119.5.106 IT, Italy 89-119-5-106-static.albacom.net.
221.154.206.233 KR, Korea, Republic of
218.208.91.111 MY, Malaysia
65.120.74.22 US, United States
210.185.64.195 AU, Australia 210-185-64-195.intrapower.net.au.
202.168.58.111 AU, Australia 111.58.168.202.static.comindico.com.au.
85.25.71.36 DE, Germany loft1551.serverloft.de.
193.136.39.26 PT, Portugal genid.dcc.fc.up.pt.
150.146.40.173 IT, Italy mspasiano.cedrc.cnr.it.
221.194.128.66 CN, China
82.135.192.72 LT, Lithuania 82-135-192-72.static.zebra.lt.
80.10.250.17 FR, France
207.28.220.1 US, United States voyager.iavalley.cc.ia.us.
164.77.208.198 CL, Chile
38.107.141.131 US, United States host-38-107-141-131.mtl.net.vexxhost.com.
222.218.156.41 CN, China

My Solution

[ironicsky]

I was having similar brute force attacks.
I've made some alterations to protect my server from brute force SSH attempts.

1) Moved SSH to another random port
2) Bound the SSHD to an IP address that is not used for Web/Mail/FTP, etc.. So the IP should generally see less traffic
3) Disable Password Authentication, Users who are given SSH access must use a password protected key file
4) Disabled Root SSH Login
5) Setup the system that 3 failed logins add the entire IP Subnet(X.X.X.0-X.X.X.255) for 15 minutes, 5 failed attempts 1 week, anything else is a never ending ban. (iptables and hosts.deny, just in case)

Re:My server got attacked last Thursday

[RzUpAnmsCwrds]

Um. You realize, of course, that remote desktop is a lot less secure than ssh, right?

Remote Desktop uses TLS and X.509 certificates, so it's not exactly trivial to crack. It's easily as secure as SSH using password-based authentication. It's definitely *more* secure if your users never bother to actually check unknown server keys.

Now, compared to SSH using only key-based authentication, Remote Desktop isn't as secure. Unless you use smart cards for authentication with Remote Desktop, which are probably more secure than having your private key stored on your computer. Unless it's encrypted with a strong passphrase. Unless your computer has been rooted and has a keylogger.

Bottom line? Both systems can be plenty secure. It's all in how you configure and use them.

Passwords are a terrible idea

[betterunixthanunix]

"IMHO if the passwords are strong enough there is nothing to worry about"

The problem is that most users are not capable of choosing a strong password. Even when you try to enforce policies about minimum password strength, users will manage to choose weak passwords; aside from the world's most common password (password1), there are plenty of people who use their own username as a password -- and requiring non-alphanumeric symbols won't stop them: jane123 will just becomes j@ne123. Minimum password lengths won't do it either, since the users will either write their password down (not an issue for botnets, but certainly an issue for high profile targets) or just come up with something that is easily guessed (abc123abc123).

Public key authentication is better but not by much. The real issue with it is that users do things like not having a passphrase for their private key (which is even more convenient than a weak password) and making copies of that key everywhere that they want to log in from (some might even carry the key around on a thumb drive). Public key authentication will solve the problem of distributed brute force attacks, but it does not really solve the problem of users having their accounts compromised by a determined adversary.

Assuming that your system is high profile enough to be worth the expense, you would really want to use a one-time password device. If the device is stolen, you have a problem, but beyond that there is not much an adversary can do. You can even mitigate the problem by requiring check-ins -- a user must login at a certain time, so if their device was stolen they will be forced to call helpdesk and report the problem.

Of course, it really depends on your security needs. None of the systems I administrate are high profile enough that I am worried about anything other than a distributed brute force attack, so public key authentication is good enough (an attacker who wanted to take over some Linux servers for his nefarious deeds could find an easier target with equal computing power, and the data on the disks is just academic research which will be published anyway).