Slashdot Mirror
Cracked Linux Boxes Used to Wield Windows Botnets
m-stone writes "Online auction house eBay recently did a threat assessment to better understand the forces ranging against them. The company is keeping the fine details under wraps, but the biggest source of danger for the company is apparently botnets. You're never going to guess who was running them. '[Dave Cullinane, eBay's chief information and security officer] noticed an unusual trend when taking down phishing sites. 'The vast majority of the threats we saw were rootkitted Linux boxes, which was rather startling. We expected Microsoft boxes,' he said. Rootkit software covers the tracks of the attackers and can be extremely difficult to detect. According to Cullinane, none of the Linux operators whose machines had been compromised were even aware they'd been infected. Because Linux is highly reliable and a great platform for running server software, Linux machines are desired by phishers, who set up fake websites, hoping to lure victims into disclosing their passwords."
I've noticed a large increase in attempts to crack my co-lo Linux servers recently, and it must be said that two got through (shared site, some customers running old content management apps and the kits hit). When we watched the behaviour of the cracked box, it was connecting back to...I think undernet.org or similar?...and sending controls via IRC. Plus doing a spot of spamming of its own bat.
Our set-up is that we have a host OS install doing nothing but running VMware Server and then any real stuff gets done in a VM, so this was easy for us to recover from quickly via VM snapshotting. But still, it's a trend that's noticeably on the increase.
Cheers,
Ian
So many people have that mentality or were converted by hearing sayings like that.
They don't realize, like any other operating system, if you want it secure, you have to work to make it secure. Everything from using good passwords, to not running unecessary services, to getting behind a firewall or two.
And, as usually, the biggest security hole is between the keyboard and the chair.
A fair amount of it, I'm sorry to say, is due to the perception that Linux boxes are much more secure than Windows and therefore don't need (a) up-to-date patches (b) proper security reviews of any app code (which these days usually means web apps) (c) defence in depth (block outbound connections from your web server, except for a hole poked in tcp|udp/53 to/from your DNS server if needed (d) proper security monitoring. Review your firewall logs! Run an external syslogNG box! use netflow, nagios, ntop etc -- can you account for all the packet flows from the machine? If you have time to spare, look into Snort.
Get the facts.
This really doesn't suprise me. With tools like ssh and shells installed by default, Linux is just plain easier to use remotely. Linux machines would also tend to stay up and online, whereas (predominantly Windows) desktops are often shut off when not in use. So, Linux makes the best "control console" for a botnet. The "army" should still be made up of Windows desktop machines, due to their large numbers.
It's the double edged sword of software popularity.
... perhaps I should.
Linux is becoming so respected and desired as an operating system for servers that phishers & hackers are slowly turning their attention towards it being profitable.
I think this will be the true test for Linux to prove that it can beat Windows in all departments.
I actually see this as good news although I must confess that when I get home I'm going to check & double check the configurations on the ports on my router and all my Linux boxes. When toying with app servers & apache, I have noticed tons of port scanners probing my Linux boxes. I paid them no mind although now
My work here is dung.
I work for a fairly well known dedicated server provider. If I had to give a rough estimate, I'd say we're 40% Windows and 60% Linux environments. Not surprisingly, the number of boxes that get hacked (rooted entirely or not) is about equal between the two, however the purpose for which they're hacked is generally quite different. 80% of the hacked Linux boxes are used for UDP floods, things like that. Also IRC bots. Interestingly enough, in my 6 months working there, I don't believe I've ever seen a Windows box used for phishing. They're always used for FTP servers hosting movies/music/programs and/or IRC servers doing the same thing.
Nothing like getting a stupidly high bandwidth bill to find out your hosting server has been hacked. Its even better when you have to fight them to prove its their fault for being hacked and not yours for being cohosted by them!
and yes they are running Linux... they apparently didn't cover all their bases and were caught by more than one known exploit and some default settings.
Just because its Linux does not make it secure, you actually have to use it correctly.
* Winners compare their achievements to their goals, losers compare theirs to that of others.
I think it's interesting to note that while we get submerged in a barrage of Windows trolls, that the hackers hack one or a few Linux boxes and use them to control the hundred or more Windows boxes they've hacked.
Still looks bad for Windows. Plus, here's betting they're servers, and not home computers behind a plain old linksys router.
-Nathan
Please stop stalking me, bro.
I'm sure pretty much everybody who is running a Linux server (or any server as a matter of fact), especially with services like SSH enabled, is currently subject to brute force attacks.
When I looked at my auth log I noticed a huge amount of brute force attacks for all my servers, so I installed denyhosts, which seems to work fine.
I guess the problem is also that in many distributions SSH servers are configured to allow root logins, and if nobody looks at the log files these go totally unnoticed.
Iftach Amit says "Since Linux machines can be used to more easily create specially crafted networking packets, they can be used in highly sophisticated online attacks". If you root-kit a machine then regardless of OS you can create whatever packets you want. Bypassing the IP protocol stack and sending raw data on the wire can't be particularly difficult if you are trying to conceal processes from the equivalent of "ps" and avoid other methods of detecting your code.
While I agree that Linux is a reliable OS, I doubt that is a reason for attackers to target it for running phishing web servers either. A good reason for targeting an OS is that you know it well and can easily write code for it. Given that many insecure machines can be obtained running any OS you please it makes sense that attackers will target their attack on machines that they know well. Maybe the criminals in question just enjoy Linux programming!
http://survey.netcraft.com/Reports/200708/
Then there's the issue of where servers are located, if you want reliable servers on the net then often the location of the server (in terms of a server room with UPS etc) is more important than the OS. What's the server market share for Linux? The above URL shows Apache leading the field for web servers and most Apache installations run on Linux...
It seems that if you want to own some web servers then aiming at Apache on Linux gives the largest number of potential targets - whether that gives the largest number of vulnerable targets is another matter.
See http://etbe.coker.com.au/ for my blog.
Maybe the slashdot host had to pause to update a botnet;-)
From tfa:
Cullinane: "The vast majority of the threats we saw were rootkitted Linux boxes, which was rather startling. We expected Microsoft boxes,"
Alfred Huger: "We see a lot of Linux machines used in phishing, We see them as part of the command and control networks for botnets, but we rarely see them be the actual bots. Botnets are almost uniformly Windows-based."
Seems like people are jumping on this as "linux bad!" where in fact the article is fairly neutral, Colinane has one opinion, Huger has another (and generally more accepted) opinion. Haydn.
Time is an illusion. Lunchtime doubly so. - Douglas Adams
The company I work for performs emergency Linux support services. We get a lot of calls from peoples boxes who are attacked. I've seen at least two eBay/PayPal phishing sites recently. In both cases, it had nothing at all to do with Linux itself.
/var/www/html, and stick some php code in there.
Case #1: Customer running a web server had vulnerable PHP applications (I believe it was an outdated WordPress). Someone was able to use this vulnerability to wget a few php scripts and bury them under some subfolders.
Case #2: Customer had a non-root account with a weak password. This account was in the "root" group, giving it write access to a number of system files. Cracker was able to brute force the password quite easily, make a directory called eBay under
In both cases, the php scripts were logging username and password guesses into a text file. The text file was within the same web root, allowing the cracker to easily grab the latest passwords over http instead of needing to re-crack. Also, in both cases, there were at least a dozen usernames and passwords in the text files.
The lesson: Keep your web apps up to date, use strong passwords, and don't add anyone to the root group.
Because Linux is highly reliable and a great platform for running server software, Linux machines are desired by phishers
So when phishers target windows servers, it's because windows has horrible security, but when they target linux servers, it's because linux is just awesome?
I am a supporting system administrator for Linux/UNIX servers at a large hosting
/var/www/html
company. I have come across many Linux servers that are compromised and being
used to host phishing scams, spamware, IRC servers, etc. Rarely, however, do I
see a "root'ed" server -- that is, a server on which an unauthorized
person or program has gained root privileges illicitly. In fact, having root
access is not necessary to host web content, send mail or provide other
Internet-facing services.
All that is needed is the privilege to put content served by the web server in
place. That could be a script for server-side execution, page or fragment for
browser- (client-) side execution, etc. If you can upload to the web content
(DocumentRoot or include) directories and the web server automatically servers
that content, you, too, can host a phishing scam or illicit media for download.
If a directory in the DocumentRoot tree on a web server can be written to by the
web server (the apache or nobody system account) then it is easy to inject one's
illicit content on that server. OS is irrelevant at that point. In fact, if a
web server has world- or apache-writable directories in the web content area the
OS *must* allow any web client to upload whatever they desire to that server.
It is the responsibility of the owner of the server to restrict who gets to
upload what content to his/her server.
I try to explain to web designers that granting write access to the
apache/nobody user is BAD, but often I hear back: "Ya, but, I can't make
the script work without opening the permissions." Usually, this is done on
PHP Content Management System portal sites that allow content to be uploaded
directly from the web browser by arbitrary users. There is a little bit of
effort required to make doing this difficult -- and it can be tricky to get
right -- but forcing the script to work by removing world/apache write
privileges is EASY:
$ sudo chmod -R 777
Ugh. Then, when that same customer is complaining that, "Hey! I've been
hacked!" I respond, "no, you haven't. You been compromised. You
allowed *anyone* to upload *anything* to your server and set apache to
automatically server that content. You were trusting *everyone* on the Internet
to behave. Your trust was broken and now your server is distributing phishing
scams/malware/kidde porn/spam."
If you ever think you need to "open up" permissions so your PHP script
will "run right" you either need a different PHP script or help making
the script run "safely." It's harder than chmod'ing 777 but it's
definitely worth doing.
One server I worked on had a lazy owner who allowed apache full write and
execute access to his web content directories. He would not upgrade his PHP
scripts to patched versions that plugged well-publicized holes. After repeated
warnings I received a frantic call from him that his server was
"hacked" and running a banking phishing scam. I checked the weblogs
and found that 20,000 people had clicked the phishing scam links from their
webmail inbox and retrieved the malware-ladden web pages with Internet Explorer
-- meaning many of these people were sending their data right to the
Russian/terrorist criminals for funding their illicit operations. The customer
asked that I call the FBI to "find out who is responsible" and I said
I didn't need to make that call to find out: he was responsible.
That customer is now fully-turned around and is complying with the necessary
steps to ensure that his server is not used for illicit purposes any longer.
Root was never required for these compromises. Just poor administration.
-- @rjamestaylor on Ello
One of the problems are dedicated server hosts. I picked up a dedicated box a while back and I was startled to find that I was put in a position to scramble to secure the box immediately upon receiving my ssh password.
Of course, I could have paid extra to get a more secure box, but budget was an issue, and my plans were pretty simple for the machine.
Another problem is that a lot of webmasters with dedicated boxes and virtual servers end up running older and insecure versions of software - from mail servers to web servers, etc. because the software is all wrapped as part of Plesk or something similar. When security patches come out, the turnaround time for updates from the software providers is far from instantaneous.
A third problem is efficiency. If your system has been rooted, it's easy to not notice as long as the person who rooted you isn't abusing your system resources.
Recovering a rooted system is a problem as well - sys admins in general could stand to take a lesson from rootkits to protect their own system. I've seen two instances myself where overwritten binaries like ps and ls could not be reverted without a great deal of effort.
Further - people who get "Managed" servers expect that they have a secure system and that their system is being monitored for security issues regularly. From what I've seen, "Managed" means that vendor provided packages get updated automatically and uptime may be monitored, but that's a far cry from someone actually managing a system.
Linux can be secure, but I think the vast majority of web servers out there are wide open targets, much like all those windows ME boxes attached directly to cable modems.
With windows of course those poor hard working crackers and continually having to rebuild their botnet as other crackers pilfer their bots as readily as they orginally gained, 24/7 no rest for the wicked.
So winbots while easy to gain are nearly impossible to keep because of course they are just so slutty, they are anybodies ;).
Chaos - everything, everywhere, everywhen
There's a particularly nasty rootkit out there which overwrites certain system programs (such as ls, ps, netstat, md5sum and a few others) with modified versions, then does a chattr to stop you overwriting them (though lsattr is left alone). And while attempting to clean up a machine so infected, I've seen Perl scripts changing the value of $0. This means even if you've got a "clean" ps around (like a copy of busybox in your own non-root home directory ..... you do have a non-root login, don't you?), it will report the "wrong" thing. Another clue that this rootkit is installed, is that (at least on Debian and Slackware) coloured directory listings don't work properly, and invoking ls generates a non-fatal error message. (The "special" ls must be based on an older version.)
..... install a script in a user's home directory, then persuade it to run. Beware of badly-written PHP scripts which don't chmod uploaded files to make them non-executable (turning off short open tags is also surprisingly effective). And what you think might be a DDoS (repeated attempts to retrieve mail on nonexistent accounts via POP3) might actually be a password-guesser. Block the /24 with an iptables rule at once. Note, if you aren't within walking distance of your co-lo, make your first firewall rule /32 because my IP is static) and never, ever use -I INPUT 1; use -I INPUT 2 or -A INPUT instead. It's too easy to block yourself out with an injudiciously-applied rule (and I do live within walking distance of my co-lo). If you see a process running that looks suspicious, leave it running long enough to examine its /proc entry before applying kill -9. Give users who don't need shell access a "shell" of /bin/true or /usr/games/fortune -o; but be sure to include whatever "shell" you gave them in /etc/shells -- otherwise they will not be able to use FTP. (If they don't have any web space on your server, just e-mail, then use /bin/false and don't put that in /etc/shells. That will make it harder to use an ftpd-based exploit.)
..... so running 64-bit Debian (which has *no* 32-bit libraries) will break them. Personally, I'd like to see a patch that will make Perl give a segmentation fault if any script tries to alter $0. In fact, I'd like to see a kernel patch that will break any binary that was not compiled locally.
The www-data (Debian / Ubuntu) or apache (Fedora) user should not be running any process other than apache2 or httpd. If you see something like "accepting connections", that's a sign that someone could be running something nasty.
In general, watch for world-writable directories (they list with a green background in Debian) because that's one of the first steps in cracking a box
iptables -I INPUT 1 -s 10.20.30.40/32 -j ACCEPT
(replace 10.20.30.40/32 by a subnet specifier which will always contain your own IP address -- get this from your broadband company -- and just to make you all jealous, my one ends in
Note that the binaries in this rootkit are 32-bit
Je fume. Tu fumes. Nous fûmes!
A friend emailed me about this just this morning. Here is what he wrote and my reply:
> I'm going to chalk this up (tentatively) to the increasing popularity of
> Linux, which means that a subset of users will be those who don't actually
> know what they're doing, and how to protect a box-- something long the norm
> in the Windows world:
>
> http://computerworld.co.nz/news.nsf/scrt/CD0B9D97EE6FE411CC25736A000E4723
>
> While there, he noticed an unusual trend when taking down phishing sites.
>> "The vast majority of the threats we saw were rootkitted Linux boxes,
>> which was rather startling. We expected Microsoft boxes," he said.
I am not surprised in the least that this was their conclusion. I don't chalk it up to the increasing popularity of Linux at all. I have never (not once) run across a Linux box operating in a botnet. Nor can anyone name a botnet software that infects Linux boxes. In the last 5 years I have found only one Linux box that had a security issue and that was because of PHP (*spit*) which had an XML-RPC exploit a while back and allowed someone to make the box host a fishing website that looked like some bank website. It seems very rare that a Linux desktop (not a webserver) would fall victim to this. I have never seen a security incident such as a botnet on a Linux desktop. I have seen that phishing page on the Linux server that hosted the bogus PHP install. That's it.
And I suspect that they are using terminology incorrectly. A Linux box hosting a fishing site is not part of a botnet. I can understand how Linux boxes would be more popular for fishing websites. PHP is popular and is a pox on Linux as PHP released a bunch of absolute garbage which only happens to run on Linux. It can run on Windows also but that is the expensive and less reliable way to do it so few people do. If people make a conscious decision to install software on Linux that lets just about anyone use the box for whatever they want such as PHP often does I don't think counts against Linux security.
Glancing over the article I immediately spotted this:
"eBay recently did an in-depth analysis of its threat situation, and while the company is not releasing the results of this analysis, it did uncover a huge number of hacked, botnet computers, said Dave Cullinane, eBay's chief information and security officer, speaking at a Microsoft-sponsored security symposium at Santa Clara University."
I challenge anyone to find a single MS sponsored paper or symposium which DOESN'T come to a conclusion favorable to MS and unfavorable to Linux. Just one. And they won't release the raw data. How much is a large botnet? 10? 100? Among millions of infected MS machines. I would also like to know what this alleged Linux botnet software is called.
I am positive that Linux will not be nearly so adversely affected by users who do not know what they are doing. Linux is very different from Windows and is architected for performance, security, and utility instead of being architected to make someone a boatload of money and maintaining monopoly lock-in. (See the fine the EU just imposed on MS.)
Some technical features which help ensure that even if Linux becomes popular on the desktop it won't suffer the same fate as Windows:
* Linux users don't run as admin/root.
* Email programs do not automatically execute attachments.
* Does not depend on filename extensions for anything.
* Does not auto-run anything from inserted media (Worth a laugh: http://www.foxnews.com/story/0,2933,299155,00.html )
* System of mandatory access controls (SE Linux) which really locks things down (some people still turn that off but it is improving rapidly, I use it on my desktop).
* Linux also takes advantage of NX (non-executable memory) which is a recent feature of x86 cpu's