Slashdot Mirror

On July 1st I found that one of my less important websites, hosted on a low-cost shared Web hosting service, had been broken into. A friend emailed me to say that the site was showing up in Google's search results with the Google "This site may harm your computer" warning listed next to it. I found that on one of the pages, about 1,500 HTML script tags had been inserted, loading JavaScript files from pseudo-random Russian hostnames like "www.chk06.ru" and "www.errghr.ru", none of which are currently resolving. Usually, when such script tags are maliciously inserted into a page on a website, the script tags attempt to install spyware on the machines of people who visit the site.

I immediately replaced the infected file on the website with the backed-up clean copy from my machine, and changed the password on the website in case the attacker had gotten in by using the old one. (The original file with the script tags inserted is here if you want to examine it, but use with caution -- if the .ru hostnames in the script tags start resolving again, then opening the file could cause the JavaScript on the pages to be loaded, which might infect your machine.) Then I started investigating (a) how this probably happened; (b) whether future similar attacks could be prevented, by changing some defaults in the way that hosting accounts are set up; and (c) whether the incentives for hosting providers are such that these changes are likely to happen by themselves, or whether it will require some third-party advocacy to change what we think of as "best practices".

Denis Sinegubko, the webmaster of Unmask Parasites, a free service that scans websites on demand for signs of break-ins, says:

The majority of web site compromises happen because of:

1. Stolen FTP credentials. Spyware on webmasters' computers: key-loggers, traffic sniffers (FTP protocol sends username/password as plain text), trojans that steal credentials from various programs' configuration files (FTP clients, DreamWeaver, etc).
2. Security holes in popular web software: CMS (Joomla, Drupal, etc), Forums (phpBB, vBulletin, Simple Machines, etc), Blogs (WordPress). Once a vulnerability discovered, hackers configure their automated tools to search the web for websites running vulnerable versions of the software and exploit them. This can be done easily and at almost no cost when they have an army of zombie computers.
3. Security hole in "in-house" web software. Many novice (and even many experienced) web developers don't properly sanitize user input making various attacks possible (SQL injections, XSS, etc)
4. Poor security practices (Something that should be manually configured by site/server admins and cannot be fixed with automated security updates): Weak passwords, open ports, insufficiently strict permissions for limited accounts, files and directories with world write permissions, etc.

I didn't have any third-party web software or custom-made software installed on the PublicEditorMyAss.com site, the password was a seven-letter meaningless mix of letters and numbers, and I didn't have permission to change most of the things like open ports and file permissions. That left the possibility of stolen FTP credentials. This is in fact what Sinegubko says is the most common cause of such break-ins:

I guess 90% of attacks use stolen FTP credentials this year. Check this Google's graph that shows the top 10 malware sites as counted by the number of compromised web sites that referenced it:
http://googleonlinesecurity.blogspot.com/2009/06/top-10-malware-sites.html

I reviewed 4 most widespread of them (Gumblar, Martuz, Goooogleadsense, Googleanalytlcs). All four used stolen FTP credential to penetrate web sites and upload malicious content. The chances are the rest used this vector too.

When the PublicEditorMyAss.com site was set up, the default setting was for pages to be edited over FTP. Even though FTP sends and receives passwords without encrypting them (in contrast with alternatives like SFTP or "secure FTP", which encrypts passwords), for a long time I had assumed that this was not a major security problem, because in order for an attacker to intercept the passwords in transit, they would have to control a machine somewhere on the path between my home computer and the PublicEditorMyAss.com server. I figured this wasn't worth worrying about, because it was much more likely that an attacker would attempt to steal the password by installing spyware on my home computer. And if an attacker managed to do that, then I assumed that the risk of passwords being stolen by spyware was about the same whether I used FTP or SFTP -- because either way, the spyware could just steal my password by reading it out of a configuration file where the password was stored. (Even though FTP and SFTP programs both store passwords in an encrypted format, the programs have to be able to decrypt the passwords in order to use them whenever the user wants to open a connection. So the spyware could just mimic whatever steps the client programs use to decrypt the stored passwords, in order to steal one of my passwords stored in a file.) So, I assumed it made no difference whether I used FTP or SFTP.

But according to what Sinegubko told me, this reasoning was probably wrong. The problem is that even though spyware installed on your machine could read passwords that are stored in configuration files, it would be a lot of work to write a spyware program that could do this, because every FTP program and SFTP program stores passwords according to a different algorithm. It's much simpler for spyware to simply watch the traffic sent and received from your machine, so that any unencrypted passwords will be spotted:

[Passwords can be stolen by] sniffers that read all TCP traffic on local computers. Like personal firewalls but malicious. They can easily intercept FTP credentials since they are sent as a plain text.

Sinegubko describes how one of his contacts obtained evidence that a common spyware program was doing exactly this:

One of them even infected a spare WinXP computer (with Gumblar) to test the consequences. On the infected computer he created a new account in a popular FTP client and saved it. The server address was correct (his server) and the username/password pair was not valid. A few hours later in FTP logs, he discovered login attempts that used that invalid username/password pair from a Singapore IP, then from a Florida IP, the some other country's IP. Apparently the FTP credentials were somehow stolen from that infected computer.

I know of only two instances where I've ever definitely been infected with spyware. I don't do stupid things like downloading and running strange programs from third-party sites, so I think both infections were probably caused by a site exploiting a security hole in Internet Explorer, or in a plug-in like Adobe Acrobat or the Flash player. Both times, once I noticed I was infected, I got rid of the infection with Malwarebytes, but I don't know how much damage the spyware did in the meantime.

So this was a case where a little knowledge can be a dangerous thing. If I had known nothing about Internet architecture, and someone told me "FTP is less secure than SFTP," I would have found a way to switch to administering the site via SFTP. But because I knew that the main reason FTP was considered "insecure" was because it transmitted passwords unencrypted, but I also knew that most of of the machines relaying those passwords in transit were secure and trustworthy, I thought it didn't matter. Now it seems that is probably how my password got compromised after all.

In that case, why don't more people switch to administering their sites via SFTP instead of FTP? Here are the steps it took me to enable SFTP on my GoDaddy hosting account. Feel free to use this as a reference, but the obvious point is that as long as this many steps are required, it's safe to say that most users won't be switching:

1. Go to the "Hosting" menu and pick "My Hosting Account."
2. Next to the name of your website, pick "Manage Account." This will open the Hosting Control Center.
3. In Hosting Control Center, click to expand the "Settings" options.
4. In the "Settings" control panel, click the "SSH" icon.
5. You will see a page saying "SSH is not set up", and prompting you to enter a phone number so that their automated service can call you with a PIN number. After you enter your phone number, the phone rings a second later, and you enter the PIN in a form on the GoDaddy website.
6. You will then see a page which says:

   Current Hosting Account Status: Pending Account Change
   Your request to enable SSH is being processed. This upgrade may take up to 24 hours.

In fact, even if only one step were required to switch, most users probably wouldn't change from the default setting to use FTP, due to the eternal, unchangeable fact that most people do not change their default settings, ever. (What percent of users ever change the default set of toolbars that are displayed at the top of their Web browser window?)

If more Web hosting companies made SFTP the default, then the number of websites that were compromised by stolen login credentials, would probably go down. Spyware authors might start to make their programs smarter at that point, enabling them to read the passwords stored by popular FTP and SFTP programs, so that it would make no difference whether the passwords were transmitted in the clear or not. However, this would be harder for spyware authors to do correctly, so it would at least raise the bar for a successful malware attack, and the number of compromised websites would be reduced.

Unfortunately, Web hosting companies don't have much incentive to make users switch to the more secure SFTP protocol. This isn't necessarily true of all security risks; sometimes the hosting company has a strong incentive to pass on the right wisdom (and select the right default settings) for their customers. From the hosting company's point of view, you could divide risks into three categories:

• Risks where the hosting company pays a large part of the price for a customer's machine being compromised. For example, if a cyber-criminal takes over a customer's machine and uses it to launch a denial-of-service attack by sending it a flood of traffic, the hosting company will see that traffic spike on their network. The hosting company has the most incentive to help prevent these types of attacks.

• Risks where the hosting company doesn't directly pay a price for the customer's machine being compromised, but they may have to deal with complaints sent in by third parties. For example, a customer's website could get broken into, and script tags could be inserted into the pages that cause visitors' machines to be infected with spyware. Those visitors might complain to the webmaster of the infected site, or they might complain to the hosting company, which then forwards the complaint to the webmaster. The hosting company may have to provide a few minutes of tech support to the customer, advising them to change their password and scan their own machine for spyware, but they probably won't incur any other material costs.

• Risks where neither the hosting company nor the customer pays a price for the machine being infected, but the price is paid by "Internet users as a whole." The only attack that I can think of in this category, is an attack where a cyber-criminal inserts key words into your web page and links them to his site, in order to increase his Google ranking for searches for those key words. Neither the website owner, nor any visitors to the website, are victimized directly; the harm being done is that the quality of Google search results is reduced for everybody. The only reports of the attack would probably come from "good Samaritan" Web surfers, who tell the hosting company or the webmaster that one of their pages has been vandalized.

When a customer's FTP credentials are stolen, the price paid by the hosting company lies somewhere in the middle. An attacker who stole my current PublicEditorMyAss.com credentials would only be able to deface the content on the site, but they wouldn't be able to launch an attack against a third-party network (my PublicEditorMyAss.com hosting account doesn't have the ability to initiate an outgoing connection to a third-party site).

Weighing in the other direction are the costs of switching to SFTP. If existing customers are forcibly switched over, phone lines will be clogged by customers wanting to know why their old method of logging in to their site has suddenly stopped working. A better choice would be to allow existing customers to stay with FTP while making SFTP the default for new customers. But there is a time and money cost of changing anything, even a default setting.

So GoDaddy doesn't have much incentive to make SFTP their new default. Indeed, I've used many different shared hosting companies before I started running proxies exclusively on dedicated servers, and none of the shared hosting companies ever used anything but FTP as the default method for customers to administer their websites. So who can blame them? They're not making the choice that makes the most sense for their customers or for Internet security as a whole, they're making the choice that makes the most sense in terms of costs and benefits for themselves, and I'm not being judgmental about that. We shouldn't expect most companies to ever behave in any other way.

That's why I think that glib "solutions" to security problems, like "Everybody install anti-virus software", or "Everybody stop using Windows", aren't helpful, because regardless of whether these ideas would work if everybody actually followed them, the fact is that most people won't. The problems have to be addressed in terms of changing incentives for the choices people make.

What's an idea for reducing the risks of FTP credentials stolen by malware, that addresses the incentives problem? Maybe give tax breaks to Web hosting companies that set up customer accounts to use SFTP instead of FTP by default? Or ask more computer vendors to include a desktop link to pre-installed SFTP software, so that when Web hosting companies present options to their customers, it's easier for users to choose the SFTP option since they have a client already installed? (I was tempted to recommend that Microsoft include a universal SFTP client pre-installed in Windows with a prominent desktop link, but the problem with that is that if almost everybody used the same SFTP client, malware authors would have greater incentive to reverse-engineer the algorithm that the client used to store saved passwords -- and then passwords would be just as easily accessible to spyware, as if the user were using FTP all along. So a good mix of SFTP clients is safer for this purpose.)

Since the difference between SFTP and FTP usually only matters in cases where a customer's machine has been infected with malware, obviously the best solution is to avoid malware altogether, but that's much harder problem to solve, as long as malware authors can keep finding security holes in Internet Explorer and other popular programs. Making SFTP the new standard for Web hosting accounts is something that we know how to do, right now. The incentives aren't currently right for Web hosting companies to make it happen. But there may be ways to change that, and I'll bet some people can think of better ideas than the ones I've suggested. I'm just saying that the incentives problem is where attention should be focused.