Widespread Hijacking of Search Traffic In the US

Peter Eckersley writes "The Netalyzr research project from the ICSI networking group has discovered that on a number of U.S. ISPs' networks, search traffic for Bing, Yahoo! and sometimes Google is being redirected to proxy servers operated by a company called Paxfire. In addition to posing a grave privacy problem, this server impersonation is being used to redirect certain searches away from the user's chosen search engine and to affiliate marketing programs instead. Further analysis is available in a post at the EFF."

Comments

Use HTTPS

[mrogers]

Another good reason to install HTTPS Everywhere, a browser extension that will redirect your Google searches to the HTTPS version of the site. By checking the certificate presented by the server, your browser can then be sure that it's talking directly to Google. (HTTPS Everywhere also works for a lot of other popular sites.)

Or, if you don't like Google, use DuckDuckGo, which uses HTTPS by default with no need for a browser extension.

Re:Use HTTPS

[Gaygirlie]

I too have to recommend HTTPS everywhere, it's a great addon and makes it a lot safer to e.g. Surf the web over an unencrypted WIFI hotspot. And so far I haven't actually had a single glitch because of it.

Re:Use HTTPS

[arth1]

Sure, there are benefits, but as always, TANSTAAFL.

- https does incur overhead and higher CPU usage on both ends, so it will be slower.
- I will defeat most of the benefits of running local caching proxy servers (come on, this is /., surely I'm not the only one with a proxy array at home?)
- Some sites serve different content on the http and https sites.
- A few even redirects the https to http (to save themselves cycles and bandwidth, while not losing the visitor).

Re:Use HTTPS

[silanea]

- https does incur overhead and higher CPU usage on both ends, so it will be slower.

Firstly, this overhead is manageable. You do not have to be Google to run all your content over HTTPS. Secondly, apparently encrypting every single connection is a necessity of the times to prevent assholes from hijacking traffic, so that overhead is simply the necessary cost of interacting safely over the Internet.

- - I will defeat most of the benefits of running local caching proxy servers (come on, this is /., surely I'm not the only one with a proxy array at home?)

I do not know a single person who runs a proxy at home.

- - Some sites serve different content on the http and https sites. - A few even redirects the https to http (to save themselves cycles and bandwidth, while not losing the visitor).

You can disable individual rules. Over time those websites will have to stop doing those things or they will lose visitors.

Re:Use HTTPS

[cavreader]

And how long will using HTTPS pevent this? Damn near every security measure except unplugging the network cable has been defeated or made useless.

Re:Use HTTPS

[avatar4d]

For users of Chrome, you can change your default Google search to use HTTPS by following the instructions here

Re:Use HTTPS

[PNutts]

I do not know a single person who runs a proxy at home.

You should get out more, or stay in more. I'm not sure which one applies here.

Re:Use HTTPS

[Joce640k]

https does incur overhead and higher CPU usage on both ends, so it will be slower.

Yeah, my quad-core really bogs down when I use https on a connection which can transfer as much as a few hundred kbytes per second..

Re:Use HTTPS

[GameboyRMH]

come on, this is /., surely I'm not the only one with a proxy array at home?

You on dial-up or something? I just let my browser cache do the work (RAM cache only, I always disable disk caching to defeat Evercookies).

Re:Use HTTPS

[erroneus]

Either that or get these jackasses to respect Network neutrality before the law requires them to.

So now that we can see that ISPs everywhere are interested in hijacking and intercepting your traffic for their profit (and you thought you were paying them to just give you a connection to the internet) are all those people out there on Slashdot still saying we don't need any network neutrality laws?

We live in a capitalist society and their aim it to make money in every way they can. Respect for their customers takes a back seat to all other profit motives.

Currently, the phone companies are enjoined against such activity and for good reason. Why internet services are not required under the same laws to behave the same way is baffling to me.

Re:Use HTTPS

[Stellian]

Another good reason to install HTTPS Everywhere

I would also actually run a HTTPS server everywhere if I didn't have to deal with the certificate mafia, and if major browsers would silently accept self-signed without drowning the user in a storm of "RUN FOREST, RUN !!!" messages. This is currently pretty tricky to do on the browser side without opening PayPal to attack (cache the sites that use real certs ? have a hardcoded master list for first connect ?). But it would be very nice if I could publish a flag in DNSSEC that could say "This is my certificate thumbprint, use it", and leverage the secure DNS tree instead of the insecure and bogus certificate industry.

Why again should I have to fork a pile of cash to obtain a bit string that says that I actually own the domain I'm using ? Generating this bit string seems like a task that could easily be automated to the point of being free. I can understand why Microsoft would be against this (and claim tens of thousand to add you to their root zone), but for example Mozilla or Google could create such an automated certification authority, and add it to their trusted root zone since they know they can trust themselves. Such certificates would work just as the "real thing" on Mozilla or Chrome, but would of course get the usual prompts in Internet Explorer.

Re:Use HTTPS

[GameboyRMH]

One that I don't even notice on my PDA...

There's a little more lag but that's happening on the server side.

Re:Use HTTPS

[dachshund]

Or, if you're a browser that doesn't support it, just set your default search engine to https://encrypted.google.com/#q= followed by the query string.

Re:Use HTTPS

[GameboyRMH]

They could do an SSL MITM attack, I doubt their buddies in the government would mind, but to prevent that you could use Perspectives.

Re:Use HTTPS

[synapse7]

Wow I like duckduckgo, I was struck with a feeling I don't think I've had since the first time I used Google after using webcrawler, I like it.

Re:Use HTTPS

[Qzukk]

- I will defeat most of the benefits of running local caching proxy servers (come on, this is /., surely I'm not the only one with a proxy array at home?)

This is slashdot, those of us with proxies at home can make them work with https if we wanted them to.

Re:Use HTTPS

[arth1]

You on dial-up or something? I just let my browser cache do the work (RAM cache only, I always disable disk caching to defeat Evercookies).

No, load balanced Cable+DSL.
According to my local statistics, it saves around 20% bandwidth and increases page load speed around 30% (this is higher because there's a lot of tiny requests going back and forth to servers, where latency is the killer, not the bandwidth). That's significant. And it's also an average - for certain sites, the benefits are much larger.

There are some immediate benefits too, like when someone else in the household IMs me a link, and it pops up instantaneously because all the elements are already loaded, including big video files or flash.
Or when several users (or machines) have to download and install the same updates.

Got an old machine you're not using? Give it a second life as a caching proxy and caching dns forwarder. It's not hard, and if it's a frugal old PIII, the electricity costs are low too.

Re:Use HTTPS

[Larryish]

Now you know of at least one.

Privoxy blocks things that Adblock misses.

Re:Use HTTPS

[arth1]

Over time those websites will have to stop doing those things or they will lose visitors.

Like google, you mean? Their https://www.google.com/ is a redirect to a site with less functionality than http://www.google.com/
I bet they are bleeding visitors right and left over that one...

Re:Use HTTPS

[arth1]

Just because you don't notice something doesn't mean it isn't there.

http://serverfault.com/questions/43692/how-much-of-a-performance-hit-for-https-vs-http-for-apache
http://www.semicomplete.com/blog/geekery/ssl-latency.html

Re:Use HTTPS

[whoever57]

I do not know a single person who runs a proxy at home.

I do, and I use it to block both certain sites (advertising and tracking networks) and headers (referrer header is blocked, with a list of exceptions for sites that won't work without it).

Re:Use HTTPS

[Hatta]

HTTPS Everywhere only works with sites that support HTTPS. If you want to really be safe on WIFI* use a VPN, or set up a quick socks proxy with 'ssh -D'.

*notice that even encrypted WIFI isn't safe. Anyone with access to the encrypted network can eavesdrop on your packets

Re:Use HTTPS

[arth1]

The easiest is to pick up a dual wan router, which will do the load balancing for you. Something like a SYSWAN SW24 would probably work. Some firewalls, like WatchGuard and Cisco also have dual or quad WAN ports, and can be set up for load balancing, but they are going to be more expensive.

Alternatively, you can set up a linux box with multiple network cards to do the job for you.
http://lartc.org/howto/lartc.rpdb.multiple-links.html

Re:Use HTTPS

[TheLink]

Wait this is only a network neutrality problem? If I did this to someone else wouldn't it be a computer crime?

I suppose it's like when Sony rootkits everybody it's just an embarrassment that they are caught doing it, but when I rootkit even one person it's a computer crime?

Re:Use HTTPS

[BitZtream]

I tried, but when I clicked install it gave me a completely useless firefox extension. Its completely useless because I, and well, the majority of the web users, do not use firefox. Now I realize that of the 5 big ones, my prefered browser is second to last with only 2 or 3% usage according to w3schools, but nothing for Chrome or IE either? Not even a 'go get it from your built in extension source'? Seriously?

Firefox is the new IE, people went retarded and code shit for firefox rather than remembering there is more than just firefox out there. Good job Mozilla, you've official made yourself the next IE.

Now, I'll go stumble around until I find it for Safari, probably will just take a 2 second Google search I know, its just stupid that when I go there I get a damn .xpi. I wouldn't even know what to do with it if I wasn't familiar with Firefox. Truely bad form guys.

Re:Use HTTPS

[Dan667]

https://encrypted.google.com/ works if you use the url directly.

Re:Use HTTPS

[BitZtream]

- I will defeat most of the benefits of running local caching proxy servers (come on, this is /., surely I'm not the only one with a proxy array at home?)

I think you'll find the percentage pretty low, the only people who do it really are those with more time than money, as in most cases it offers very little benefit.

My browser caches pretty well on its own, there are 2 people in my house, me and my wife. We view pretty much 0 related content on the web with the exception of photos ... which are served locally anyway. A cache proxy is going to get almost 0 cache hits in normal usage, except for that one time when I happen to look at an old reference manual 3 months down the road after my browser forgot it.

Running a cache proxy at home is something you do when you don't have a real job and want to futz around in your free time, maybe learn about it so you can use it at a job and make yourself more valuable. I get that. The rest of us however, don't have time to set it up (yes, I know its almost trivial, but lets face it, you gotta tweak it for yourself and next thing you know, its 5 hours later). And then a year or two later it breaks, or you futz with the firewall or upgrade the OS and the transparent part of it breaks, so you open it up so your wife can browse and never get around to fixing it again.

Basically, running a local proxy at home is a toy for people with extra free time, and occasionally, someone doing some testing, which I admit, my home network is a functional testbed for future rollouts at the office. But its not something everyone does, its limited to a few geeks, even here on slashdot.

Re:Use HTTPS

[aztracker1]

For Chrome users, see KB SSL Enforcer.

Re:Use HTTPS

[SuricouRaven]

I use a proxy for blocking too, as it allows centralised managment of blocking rather than having to deal with software on five laptops, two mobile phones and a tablet.

Re:Use HTTPS

[BitZtream]

Do you have browser caching turned off or something or do you just browse so freaking much your browser cache is overflowing regularly.

You're right on automatic updates, but being that you can just schedule them for a time when your lines aren't busy, which seems a whole lot simpler than setting up a proxy.

I find your numbers suspect. Perhaps in a large enough household, with a bunch of facebook users or something where you guys visit the same set of sites, but in my house, which is small with only 2 perm residents and a couple that float in and out from time to time, family is like that :/ ... when I bothered with a caching proxy we saw almost 0 cache hits from it, we all browse different stuff and our browsers handle the caching just fine.

I do run a caching DNS server, but thats only because I have different DNS views internally for a few personal/work domains in order to keep that traffic flowing over the VPN connections, caching is on anyway so its not like I have to maintain it. I do have to occasionally flush it rather than wait for name updates, but again thats just due to my work environment.

Re:Use HTTPS

[steevven1]

The annoying this I find about using HTTPS Everywhere with Google Search is that it won't let you click through to an image search right from your web page search results. Google should really allow you to do so, even if it took you off of HTTPS.

Re:Use HTTPS

[erroneus]

If there were a criminal investigation, it would end with the privacy agreement between the subscriber and the provider in the clause that says "subject to change without notice."

If someone else did it, it would be a crime because it would be intrusion on someone else's network.

Re:Use HTTPS

[bedouin]

Running a cache proxy at home is something you do when you don't have a real job and want to futz around in your free time, maybe learn about it so you can use it at a job and make yourself more valuable.

If you have one machine running as a home server, it's trivial. My box handles a plethora of little tasks that can be centralized: file server, web server, torrent box, machine to run Linux X11 apps from in OS X or even iOS, web cache, home DNS and forwarding, BOINC, backup server . . .

The rest of us however, don't have time to set it up

sudo apt-get install squid

Make a few changes to squid.conf and you're done. You might not find it worthwhile, but others who're already running a little home server might as well.

Re:Use HTTPS

[BitZtream]

, and if major browsers would silently accept self-signed without drowning the user in a storm of "RUN FOREST, RUN !!!" messages.

Just a hint, every time you say that, it makes it very clear that you have absolutely no idea how SSL works. SSL with unverified certificates is absolutely useless, which means blindly accepting it and pretending its okay is a lie of omission to the user, its basically snake oil instead of something useful.

At the very minimum, the user has to be prompted to verify the unknown certificate. You must make the wording here strong enough that people GET that its a dangerous decision. You setup a site with a self signed cert, then your upstream ISP just MITMs it, and instantly your SSL means exactly DICK because no one will know the difference except for the people who get a warning that its a different certain than the previously self signed one ... which means they'll get that warning every year or so anyway (unless you're just retarded and using long term certs, again showing you completely fail to understand SSL and what makes it secure).

But it would be very nice if I could publish a flag in DNSSEC that could say "This is my certificate thumbprint, use it", and leverage the secure DNS tree instead of the insecure and bogus certificate industry.

That may happen to some extent, but your missing the point of SSL. A TRUSTED THIRD PARTY has verified the identity of the certificate holder (well, thats the theory anyway, we've seen examples of where it breaks down occasionally). DNS is unverified, I could go buy MTV.COM if no one else had done so (and made a fucking fortune selling it to them ;), and then publish my own certs as you suggest, and put up sites and make it look like I'm MTV and you would have absolutely no way what so ever to verify that I really am MTV, all you know is that I bought a domain, which has absolutely 0 verification associated with it. With SSL the way it works, someone else has verified the company information that MTV.COM uses in their certs, so when I view the information, I know with a high degree of trust that the information presented to me is accurate, so when it says whatever MTV.com's cert says for the company name, I can trust that its true and that I'm talking to a server that not only the owner says is the right one, but someone else, whom I trust to verify their information has also verified it. The third party makes it WAY harder to lie about who you are, which cuts out MITM attacks, which is the entire point, making sure no one in the middle can read or modify your data.

Why again should I have to fork a pile of cash to obtain a bit string that says that I actually own the domain I'm using ?

A pile of cash? Seriously? Their like 15 fucking bucks from godaddy. If you can't spend $15 dollars on a cert, you probably can't afford to do most everything else required to run a website. I'm sorry, this is a stupid fucking reason to not have an SSL cert, if you can't pay $15, you need to go home. A pan handler on the streets of washington DC makes $100-150/day ... and you can't spare $15 for a cert? You're priorities are fucked up.

Of course, those are shitty certs that I specifically don't trust and have removed the godaddy certs from my trusted roots, but I'm certainly a rare example of that, and go daddy isn't the only source of cheap certs.

The reason it costs money is because someone actually needs to put a little effort into verifying not that 'you own the domain' but that you ARE who you say you are, and you're not some fake company that doesn't actually exist just running a scam. Now we've seen plenty of examples of how some shitty cert provider fucks up and allows it to happen anyway, but that generally gets fixed and doesn't happen again.

Second, it weeds out a bunch of people who want to do stuff, but really don't know how and won't maintain their sit

Re:Use HTTPS

[bedouin]

Remember when people moved to Google around '98 or so. The search results were nice, but the main attraction (for me) was a truly sparse layout with minimal advertising.

Now Google, in addition to appearing very nefarious, has a cluttered shithole of a layout that makes Bing attractive in comparison.

DuckDuckGo is minimal. It's not trying to be anything more than a search engine -- just like Google was in the beginning. Seems like each time I search for anything remotely obscure on Google nowadays I get an assload of spam links.

You know DuckDuckGo has an SSL option right? Others should look into it -- they have a no logging policy.

Re:Use HTTPS

[bedouin]

I had to build my own Safari version. There are some issues that prevent the EFF from standing behind an official binary; you've probably encountered info about it by this point.

Here is a link to my build, if you're willing to trust me:

http://tinyurl.com/6dul8vr

Re:Use HTTPS

[instagib]

How do you manage blocking on these devices if they're used outside of your house?

Re:Use HTTPS

[datapharmer]

Not anymore. It functions just like the http version. Yes, it has changed recently.

Re:Use HTTPS

[arth1]

Do you have browser caching turned off or something or do you just browse so freaking much your browser cache is overflowing regularly.

I use several machines. My laptop doesn't benefit from my workstation's browser cache, or vice versa. And even on the same machine, I use multiple operating systems. With or without browser caching, there's a saving.

I find your numbers suspect. Perhaps in a large enough household, with a bunch of facebook users or something where you guys visit the same set of sites, but in my house, which is small with only 2 perm residents and a couple that float in and out from time to time, family is like that :/ ... when I bothered with a caching proxy we saw almost 0 cache hits from it, we all browse different stuff and our browsers handle the caching just fine.

That's your problem, not mine.
Even on a single-user machine with squid only serving localhost (like the one I'm on right now at work), there's a benefit. Looking at the current access.log for squid:
# wc -l access.log
25027 access.log
# egrep 'TCP_(|MEM_|IMS_)HIT' access.log | wc -l
3172
That's a substantial saving.
In addition, a lot of requests where the remote server had to be queried, but the data didn't have to be re-fetched (saves on bandwidth, not on latency):
# grep TCP_REFRESH_UNMODIFIED access.log | wc -l
2330

In a household, you get the same benefits as single users, plus, of course, hits where one user has generated or refreshed cache objects that later are used by others.

As for why you didn't see any good results, one can only speculate. Incorrect set-up, or not giving it enough time for a persistent cache to have an effect are my first guesses, but that would just be speculation.

Re:Use HTTPS

[arth1]

No, the https version is still crippled compared to the non-https search page. Compare the top bar of the two, for example.

Re:Use HTTPS

[mister_playboy]

Indeed... I ended up disabling the extension on Google's site because of this. :(

Re:Use HTTPS

[arth1]

Oh wait, search results wouldn't get cached anyway.

Correct. From a Google reply:

HTTP/1.0 200 OK
Date: Fri, 05 Aug 2011 18:50:31 GMT
Expires: -1
Cache-Control: private, max-age=0

The graphical elements and external javascripts on the search result page, though, can be cached, so it often helps even there.

Re:Use HTTPS

[Vrtigo1]

You don't even need a browser extension. It's very simple to alter the search settings to point to the HTTPS version of the site. I've been using encrypted.google.com since they made it available a year or so back with no issues. The only problem is that all of their content is not available over https. If you do a search on the https site and compare it to a search on the www site, you'll see that there are more service links (images, shopping, etc) on the www site.

And to the folks complaining that https slows down your connection - get real. Unless you live in a datacenter and are connected to the Internet vie GigE, any modern computer can encrypt and decrypt data far faster than your Internet connection can transmit it...

Re:Use HTTPS

[Vrtigo1]

You do not have to be Google to run all your content over HTTPS

Indeed you do not. The company I work for has several public facing websites that sustain > 200 Mbps during peak times. These sites are all SSL enabled and we don't have any issues keeping up with it. We use an SSL offload engine in our DC which sits in front of the web cluster and does all the SSL stuff for us. These are widely available and aren't as expensive as they used to be.

Re:Use HTTPS

[Aighearach]

Most of us quit running home caching proxies 10 years or more ago when we got broadband.

Re:Use HTTPS

[stms]

If you use HTTPS everywhere in conjunction with HTTPS finder you can add your favorite sites to your HTTPS ruleset automatically.

Re:Use HTTPS

[stms]

If you use HTTPS everywhere in conjunction with HTTPS finder you can add your favorite sites to your HTTPS ruleset automatically.

Re:Use HTTPS

[arth1]

There's a little more lag but that's happening on the server side.

No, it happens in the protocol. Even with the fastest server in the world accessed by the fastest client in the world, you still have three times as many packets (9 instead of 3) to establish an SSL connection. Say you are 150 ms away from the server; instead of the 450 ms it takes to establish each connection with http, it takes 1450 ms with https. There's not much you can do about that, except moving closer to the server.

Re:Use HTTPS

[SilentChasm]

*notice that even encrypted WIFI isn't safe. Anyone with access to the encrypted network can eavesdrop on your packets

That was somewhat true with WEP which used a shared key, but with WPA/WPA2 the attacker must capture the handshake at the beginning of the connection in order to get the session key, so just having access to the network doesn't automatically give someone access to all the data anymore. If you're that paranoid about wifi though, a VPN shouldn't hurt. A VPN also protects from attackers on the wired network too (until it gets to the endpoint).

Re:Use HTTPS

[SuricouRaven]

I don't. Only the mobiles routinely leave the house, and not much browsing is done on those.

Re:Use HTTPS

[Stellian]

Your rant is ruined by some factual inaccuracies. You seem to lack of understanding of how "domain validation only" certificates work, and how EV certificates came to be. You are presuming that all certificates do the offline identity check, when in fact only EV issuers do that nowadays. Leveraging the DNSSEC tree to do the domain validation is actualy more secure than involving a 3rd party via the classic DNS system.

I've never proposed to make SSL work like Putty, and I clearly stated in my message that accepting self-signed certificates today way will open PayPal to attack. Self signed certificates would become usable, I was saying, only when they could be authenticated via the secure DNS tree, at which point the system would become equivalent to the existing non-EV certs that validate only the domain.

I concede that 15$ is indeed a low price, last time I checked it was something like 70 less inflated dollars, and when I had multiple vhosts on multiple domains it added up quite quickly. But there's also a huge benefit with free: you can automate it as a step in a webserver setup script, making ALL sites secure by default. Since the domain is verified by Google's or Mozilla's automated interface, you would get full end-to-end encryption and authentication without DNSSEC extensions, not just protection against passive eavesdropping .

On a side note, you should understand that accepting a self-signed certificate still protects you from passive eavesdropping, a major attack scenario. A man-the-middle is forced to fake all your SSL traffic with certificates generated by him. If applied on a large scale this can easily be detected by informed users, so a democratically elected government will think twice before doing it.

That didn't take long

[Skarecrow77]

Site slashdotted in under 5 minutes.

Re:That didn't take long

[AHuxley]

http://www.usenix.org/event/leet11/tech/full_papers/Zhang.pdf paper quoted is the only real missing link.

Re:That didn't take long

[Anonymous Coward]

nah, you are being redirected.

Re:That didn't take long

[alostpacket]

Works fine for me. I just won 2 free $250 Walmart Pirce club cards and I get 20% off my next purchase of a HiPhone 5 Nano from Somy. Pretty exciting.

Re:That didn't take long

[nweaver]

Netalyzr is up for me, connecting from Washington DC starbucks, as are the EFF and New Scientist articles.

Re:That didn't take long

[Joce640k]

Hah! I installed their little app and I won a FREE iPad. It's in the mail as I write this...

ISPs

[Jaysyn]

Here is a list of the ISPs mentioned in the article:

Cavalier
Cincinnati Bell
Cogent
Frontier
Hughes
IBBS
Insight Broadband
Megapath
Paetec
RCN
Wide Open West
XO Communication

Re:ISPs

[Kreigaffe]

No Comcast? No Cox? Heck, none of the big evil corps? I am... everything I learned on /. is wrong! My world has been thrown askew!

Re:ISPs

[Anonymous Coward]

You do realize a lot of the ISPs mentioned here are simply subsidiaries of the larger ISPs, right?

XO Communication, for example, is just the end-of-the-line provider of an ATT&T backbone.

Re:ISPs

[Anonymous Coward]

Megapath

They're the ones that merged with Speakeasy. Is there no refuge for the geek any more?

Re:ISPs

[Dahamma]

Comcast and Cox already rape you openly on your monthly bill, so there's no need for them to be sneaky about their revenue...

Simple Solution

[Anonymous Coward]

Don't use your ISP's DNS. Use Google DNS: 8.8.8.8 and 8.8.4.4. No way that's hijacked.

Re:Simple Solution

[X0563511]

Then use a local resolver, ensure you set up DNSSec checking, and beat everyone with a stick who still doesn't sign their zones.

Re:Simple Solution

[fuzzyfuzzyfungus]

It would be slightly more difficult/costly than just tweaking the DNS server and getting 95% of the suckers for free; but your ISP isn't exactly technologically incapable of simply dropping traffic to/from known independent DNS servers, or rewriting responses therefrom...

Re:Simple Solution

[Skapare]

That can easily be hijacked by the ISP. They simply set up a DNS server host, add these IP addresses to an interface, and add routes to direct the traffic to that server. Done.

Re:Simple Solution

[GameboyRMH]

Not hijacked but I get a bad feeling about sending my DNS requests through an advertising company that's already nearly omnipresent and omniscient (unless you've blocked their scripts and cookies) on the web...

Re:Simple Solution

[erroneus]

Use HTTPS, use your own resolver with DNSSec, do this technical measure or that.

The fact is, you are going across a pipe controlled by another party and without laws and penalties to discourage and prohibit this behavior, this is what we can expect and will continue to get. And at the moment, they feel no guilt nor shame about this at all. They want more money (because if you're not growing, you're dying) and they will sell you and your mother to get it.

Re:Simple Solution

[bedouin]

Just run your own, unless you think Google offers that service just to be nice.

I wonder

[Bob the Super Hamste]

As I can't RTFA I do wonder if this explains some of the strangeness I see in doing searches between by work machine and my home machine. This really shouldn't surprise anyone as ISPs have been know to redirect DNS look up failures.

Re:I wonder

[Bob the Super Hamste]

After the new scientist link finally loaded it does appear that this is indeed the case as one of the listed ISPs is my home ISP (Frontier). Now if only I could vote with my dollars and switch to a different ISP that hasn't done this (Charter is my other option and they "claim" to have stopped).

Re:I wonder

[number11]

Now if only I could vote with my dollars and switch to a different ISP that hasn't done this (Charter is my other option and they "claim" to have stopped).

Why not simply plug in a different DNS instead of using their crappy one?
Google 8.8.8.8, 8.8.4.4
OpenDNS 208.67.222.222, 208.67.220.220
Verizon 4.2.2.1, 4.2.2.2, 4.2.2.3, 4.2.2.4, 4.2.2.5, 4.2.2.6 (since these are all same subnet, don't use for both primary and secondary)

You can use Google Namebench to compare DNS speeds.

Re:I wonder

[lee1]

If the strangeness is you getting different results from different computers, it could be due to this.

Re:I wonder

[BitZtream]

OpenDNS does the same thing you tool, they at least tell you about it, but none the less, they do the exact same thing.

Re:I wonder

[EsbenMoseHansen]

Not necessarily. In DK, the government censorship is at the DNS level, but anyone can just set up their own or use google's. I makes perfect sense once you realize that DK is a nanny state.

That's not a privacy concern...

[Anonymous Coward]

... that's a fucking computer crime.

Re:That's not a privacy concern...

[GameboyRMH]

No no no, big corporations did this, it's just a privacy concern ^_^

Re:That's not a privacy concern...

[MaerD]

Even better, it's trademark infringement. If I expect that when I type in google.com and/or submit a search there and I get a result back from somewhere else, you've just created confusion about who is providing the search results. Even more so if you try to disguise your page as a genuine google page.
Every search engine needs to sue not just the company providing this "service" to ISPs, but the ISPs themselves. it'll take care of itself pretty quickly.

Re:That's not a privacy concern...

[SmilingBoy]

Shouldn't Amazon and other affiliate providers be able to sue them? After all, they have paid out money for bogus referrals.

Comcast

[OzPeter]

For once Comcast does good as my local ISP. All it does is hijack the page if the DNS doesn't resolve and then puts up its own results of what it thinks the domain should be.

Re:Comcast

[Bob the Super Hamste]

Any time my ISP does that I add the returned search site to my etc/hosts file so it will never load again as Frontier seems to like to send you to crappy search pages

Re:Comcast

[X0563511]

This is available should you wish to stop even that behavior.

Re:Comcast

[Skapare]

And of course Comcast would never hijack the 8.8.8.8 and 8.8.4.4 name servers by rerouting those IPs to its own name server.

Re:Comcast

[Bucc5062]

Okay, I read through the information as went so far as to set up my laptop to use the Google public server. What's the catch? I read their write-ups about security, but frankly, I'm not a network guy and had eyes glazing fast.

If the end result is that by using 8.8.8.8 I am blocking the ability for an ISP to spoof or redirect my searches, then mission accomplished, but TANSTAAFL! What does Google get from providing this service? Better ads dollars?

Re:Comcast

[nweaver]

OpenDNS also does NXDOMAIN wildcarding.

If you want a clean public DNS, Google Public DNS is a better choice.

If you want a DNS that includes considerable filtering of known badness and other controls, at the cost of NXDOMAIN wildcarding, use OpenDNS.

Re:Comcast

[Skapare]

I just tested Comcast's DNS lookup. They are redirecting SLDs that get NXDOMAIN from the TLD server. However, for hostnames within registered and working SLDs, they are redirecting SOME of those, as well. In particular my test for a couple of my own domains shows that for .net they are not doing 3rd level name redirection, but for .us they are. IMHO, the 3rd level redirection is bad.

Re:Comcast

[X0563511]

Okay, I read through the information as went so far as to set up my laptop to use the Google public server. What's the catch? I read their write-ups about security, but frankly, I'm not a network guy and had eyes glazing fast.

If the end result is that by using 8.8.8.8 I am blocking the ability for an ISP to spoof or redirect my searches, then mission accomplished, but TANSTAAFL! What does Google get from providing this service? Better ads dollars?

These questions are answered in the FAQ. I linked them above in your quote.

Unless they are outright lying, this is one of those projects they do "For the Good of the Community"

Now, since DNS is a cleartext protocol, there's no technical reason why your ISP cannot interfere with this if they wish to. This said, doing so is more involved than simply tinkering with their own DNS servers, and this gets into a grey area legally.

Before, they were simply altering the behavior of their DNS systems, which you requested the use of (by using them). If they were to alter your requests to, say, 8.8.8.8, then they would be deliberately violating their common-carrier status and exposing themselves to all kinds of lawyer-bait.

Re:Comcast

[X0563511]

... and in doing so, invite all kinds of fun to the party!

In short, they would have to be stupid to do so.

Re:Comcast

[X0563511]

True enough. Not all ISPs that do this allow you to turn it off, however. Comcast is doing something right in that respect - at least they let you opt out cleanly.

Re:Comcast

[nabsltd]

If they were to alter your requests to, say, 8.8.8.8, then they would be deliberately violating their common-carrier status and exposing themselves to all kinds of lawyer-bait.

ISPs are not common carriers, and this sort of level of proxying happens all the time. In particular, many ISPs re-direct all outgoing connections to port 25 to their own mail server, and similarly all connections to port 53 (DNS) are sent to their own DNS server. It's not that they are "altering requests to 8.8.8.8", but rather they are altering requests to particular ports.

Also, almost every ISP blocks incoming requests to well-known "server" ports for their non-business customers. If "altering requests" was a problem, then every ISP would be in trouble for this.

Re:Comcast

[X0563511]

Redirections are one thing, but in-place modification... that's just not cool.

Re:Comcast

[mcl630]

You can turn this off... or use their DNSSEC servers which don't support it.

Re:Comcast

[nabsltd]

That's as may be, but it's not illegal. In addition, pretty much every device that your packets traverse does some sort of "in-place modification". Whether that modification is "bad" is always a point of contention, and depends on who you ask.

Akamai: good or bad? Google cache pages: good or bad? Malware stripping web proxy? "Friendly" HTTP error messages? NAT? Captive-page HTTP to give you free Wi-Fi? Take some time and think about both sides of these examples, and you'll see that it's really not black and white.

That said, business connections seem to be untouched by almost every ISP, so if you really don't want to worry about modifications to your connections, spend the money.

Re:Use https?

[fuzzyfuzzyfungus]

HTTPS will(barring CA incompetence or your ISP 'install disk' quietly adding their own root certs) assure you that you are talking to the real google.

If your ISP is fucking with DNS, though, and your attempts to talk to the real google are going to a different IP entirely, it will only warn you of that, not get you where you want to go.

If only because copyright/trademark claims for a US company serving an exact duplicate of the google homepage for monetary gain could pretty quickly hit the zillions, I'm guessing that these "Paxfire" shitbags aren't actually trying to do a 100% spoof of the site you want, just redirecting you to some horrid 'search' page of the sort normally maintained by typosquatters and similar scum.

HTTPS isn't harmful under this circumstance; but it is unlikely to tell you anything you didn't already know, and it isn't even intended to solve the problem you will want to solve...

The list of ISPs

[Bob the Super Hamste]

For those of you wondering what ISPs are doing this the New Scientist article has it:

List of ISPs that are redirecting some search queries

Cavalier
Cincinnati Bell
Cogent
Frontier
Hughes
IBBS
Insight Broadband
Megapath
Paetec
RCN
Wide Open West
XO Communication

Charter and Iowa Telecom were observed to be redirecting search terms, but have since ceased doing so. Iowa Telecom stopped its redirection between July and September 2010, and Charter stopped in March 2011.

Re:The list of ISPs

[Cornwallis]

Add One Communications (now owned by Earthlink) to the list.

Re:The list of ISPs

[nweaver]

Could you email a Netalyzr execution from One Communications to netalyzr-help@icsi.berkeley.edu, so we can verify this? It could be due to IBBS, which runs DNS for multiple ISPs.

Re:The list of ISPs

[Cornwallis]

Yes. Let me switch back and I'll email the results.

Re:The list of ISPs

[cswiger]

Add Verizon DSL in Manhattan, NY:

http://n3.netalyzr.icsi.berkeley.edu/summary/id=ae81b058-20468-26aad796-356d-4fce-806b

I was using my own nameservers before, but I'd recently swapped out my older Linksys BEFSR81 (which was becoming flaky) to an E2100L.
Its DHCP server was using Verizon-supplied nameservers by default. Fixed that, thank you ICSI team.

Re:The list of ISPs

[SmilingBoy]

Looking at your Netalyzr results, you also seem to suffer from pretty severe buffer bloat.

Re:Use https?

[X0563511]

How convenient !

Re:Use https?

[X0563511]

... or if you are feeling adventurous, you can always install your own resolver locally. Unless your ISP would hijack requests going to root servers (which is a whole other level of maliciousness)...

Re:warn visitors

[X0563511]

Probably not. You would think to try the referral URL, however that includes the DNS entry. That said, the ISP is already monkeying with the traffic, so they can always rewrite this header anyway.

Use a VPN always.

[drolli]

anyway thats not a bad idea. In that case also an hijacked machine withing you own network plays a lesser role.

Questions answered in this thread...

[nweaver]

I am one of the Netalyzr developers involved in this work. I or my colleagues will answer questions in this thread, but I may be offline for a little while so responses may be somewhat delayed at times.

Re:Questions answered in this thread...

[PineGreen]

How much does the use of neutral (for example google's) DNS services rather than default ISP's DNS help?

Re:Questions answered in this thread...

[nweaver]

They do NOT intercept DNS that's not directed to the ISP's resolvers, thus using Google Public DNS allows you to avoid this redirection completely if you are affected.

Re:Questions answered in this thread...

[nweaver]

Thats unfortunately common. Your ISP probably offers an opt-out. If it doesn't, change your DNS server to something like Google Public DNS.

Re:Questions answered in this thread...

[whoever57]

What are the gifs it tries to download? My proxy apparently blocked a gif file from being downloaded, but the only recent records of denied requests in my squid log file referred to google-analytics.com

Re:Questions answered in this thread...

[nweaver]

We don't do GIFs except for ones on the web page, there are test JPG downloads however, which check for caching.

We also fetch a .exe, a .torrent, a .mp3 file, and a "virus" (the benign EICAR test file which AV systems detect as a virus so you can check AV operation safely).

Re:Questions answered in this thread...

[andydouble07]

I've got a different ISP (Buckeye CableSystem) than the ones listed here, but they hijack google searches to their own useless branded search engine with ads and etc. Using different DNS doesn't stop it from happening, but changing the url parameters does stop it. For example reordering the &client= flag would stop it from happening, they must be using some really ineffective regex. So they're not hijacking DNS but instead doing some nasty DPI.

Re:Questions answered in this thread...

[nweaver]

We detect JUST redirection, we don't discriminate on it. True, we should probably surpress 127.0.0.1 blocking on a few sites, but we also see those sites in particular redirected to malicious sites by rogue DNS resolvers (caused by malcode infections) so we have to check for redirections on theses sites in DNS.

(The malicious resolvers change ad.doubleclick.net so the ads go through a different ad network which earns money for the malcode-authors who change people's DNS settings, and the change to www.google-analytics.com is to create context-sensitive pop-under advertisements on sites)

Re:warn visitors

[nweaver]

Google did. This is why the ISPs that were proxying Google stopped in the past couple of months: Google's abuse-detection threw up a CAPTCHA on the queries, and then Google posted about it.

Also, you can run Netalyzr to detect this condition.

Make sure to include DNS in your VPN...

[nweaver]

Make double-sure that your VPN also tunnels the DNS requests, by checking the configuration and/or by using TCPdump. EG, its pretty easy to accidentally set-up firefox through an SSH tunnel in a way where the DNS requests don't pass through the tunnel.

HTTPS/SSL is a good solution

[bigtrike]

Assuming you have a browser capable of secure renegotiation (not IE on XP or older), your ISP would have to set up a certificate authority and someone would have to add the certificates into your browsers to bypass the giant red warnings.

Re:HTTPS/SSL is a good solution

[Coren22]

You actually run those disks? I have never heard of someone who knows computers actually running them.

Re:HTTPS/SSL is a good solution

[I(rispee_I(reme]

Hah, AT&T currently redirects all HTTP traffic to a site that requires their customers to install a certificate under the guise of "configuring their DSL connection", along with some ActiveX control. Of course Firefox won't work at all until until AT&T pulls out and has a shower.

No disk required.

I do the required install in VMware, but every few months they have a screwup on their end and require the certificate to be reinstalled for some reason.

This was the case as of a month ago when I cancelled my AT&T DSL rather than support Evil with a capital E.

Re:HTTPS/SSL is a good solution

[Runaway1956]

What, your ISP didn't give you a setup CD that they wanted you to install?

WTF? Are you STILL on AOL? They're the only people who ever gave me a "setup CD" - despite the fact that I've never been a customer!

Do you have a useful tool for identifying this?

[bigtrike]

Is there some easy way we can check for this, such as with a curl or wget command line script? A great way to defeat this practice would be to notify the businesses that are needlessly paying commissions out even though they are the first result.

Re:Do you have a useful tool for identifying this?

[nweaver]

Yes. Netalyzr specifically detects this condition amongst its many other tests. We also have a Java Command Line Client.

You can also check by doing a "dig search.yahoo.com". If the authority is "jomax.net", its a Paxfire appliance changing the results.

Suddenlink redirects 404's

[BlueKitties]

I live about 30 miles from the East Texas court most of these tech patent disputes take place at. The only (see: ONLY) high-speed service in my area is Suddenlink. The alternative sold out a few years ago. Well, lo-and-behold, everytime I mistype a URL I don't get a 404 -- I get a search result (all clad in ads) with "Suddenlink" across the top of the page. This is why so many people are worried about ISP's screwing up the Internet. First, even if Suddenlink argues they're doing me a favor, why do they get to decide which search engine my 404 is sent to? Second, that makes it awfully tempting for Suddenlink to monitor my Internet activity for targeted advertising in their 404 redirect page. And third, what the buggar are the data retention policies for the site they redirect to?

Re:Suddenlink redirects 404's

[SnarfQuest]

Well, my ISP has similar rules, except they will also freuently tell me that sites like cnn.com and google.com do not exist, and redirect me to their search page.

Re:Suddenlink redirects 404's

[nweaver]

We specifically detect this condition in Netalyzr as well: we fetch three different 404 pages from our server (a blank page, a default apache page, and a custom page) and detect if they are changed in flight.

Net Neutrality

[lavagolemking]

Ok, I know this is just DNS and not some network-level hijacking, but crap like this is exactly why we need net neutrality. Capitalizing on customers' traffic by redirecting their searches (or otherwise interfering with customers' activities) is type of behavior net neutrality activists have claimed will happen for a long time, and that ISPs have claimed will never happen. Odd that the big players aren't the culprits for once (they're probably scared of regulation after the bittorrent scandal), but I'm sure if this is successful, or if a corrupt judge somewhere rules there is nothing wrong with what's going on, then we can expect to see all the big players stepping in and this will become a lot more widespread than it already is.

Re:Net Neutrality

[BitZtream]

Ok, I know this is just DNS and not some network-level hijacking

Thats irrelevant. ANY UNAUTHORIZED access to computer systems or data is illegal under federal law. You can thank Kevin Mitnick and DEC (May have my companies wrong) for that. Shortly after that whole event laws were enacted that basically made it so you need explicit permission to even VIEW someones data let alone manipulate it.

This sort of tampering, to me fits squarely as a violation of that law. I authorize them to look at the IP headers only for routing purposes, I grant no authorization for any modification of my packets. I suspect the root servers for DNS and Google would both have a similar point of view. This means they are not authorized to manipulate the data by any of the parties involved.

Seems like they should be prosecuted to me. Now, IANAL, so I'm sure theres some retarded reason why this won't work (TOS agreements probably) but it just seems like it should be a violation of some sort to screw with someones data. Isn't it illegal to intentionally fuck with someones phone call? You certainly have to have legal permission to 'view the data' of the call (i.e. wiretap) ... well, warrentless wiretapping aside.

Re:Net Neutrality

[zzsmirkzz]

ANY UNAUTHORIZED

Yes, but you signed a contract with your ISP. That contract, probably, has an elastic clause (whether this conscionable or enforceable, aside) which means that contract can change to say whatever they want it to say and you automatically agree to the new terms. Ergo, you have authorized them to do whatever they want, for now, and forever (as the terms of the contract, usually, are deemed to survive even after your business relationship with the ISP is terminated).

Mistyped URLs

[macraig]

"... additional revenue through advertising based on mistyped URLs."

This is why perfect spelling is so important.

MITM in the hosting provider's ISP

[tepples]

I don't see how Perspectives will help if the MITM is located in the hosting provider or its upstream ISP.

Re:MITM in the hosting provider's ISP

[GameboyRMH]

Well assuming the ISP hasn't set up a set of fake Perspectives project pages to serve you a tampered version to give false negative results, notary servers in other locations around the world (which you connect to using encryption keys already included in the Perspecives plugin) should see a different certificate, raising a warning.

Re:MITM in the hosting provider's ISP

[tepples]

notary servers in other locations around the world (which you connect to using encryption keys already included in the Perspecives plugin) should see a different certificate

A web server at a hosting provider has only one connection to the Internet, namely through the hosting provider's upstream ISP. So all notary servers outside the upstream ISP's own network would see the web server through this upstream ISP, including any MITMs that the upstream ISP has installed.

HTTPS requires an IP address per domain

[tepples]

You do not have to be Google to run all your content over HTTPS.

But you do pay more per month for hosting if you run your hobby site on HTTPS. Name-based virtual hosting of HTTPS sites requires SNI, but Internet Explorer on Windows XP doesn't support SNI, nor does Android 2.x. So until IE on XP passes out of use and Android 4 (Ice Cream Sandwich) has been out for a couple years, HTTPS will still need a dedicated IPv4 address per certificate, which in practice means per domain. And now that all the /8 blocks are used up, hosting providers such as Go Daddy have started to charge per IP address.

Re:HTTPS requires an IP address per domain

[Vrtigo1]

You can work around this problem with an SSL-enabled reverse proxy. Let's say you have three websites Web1.com, Web2.com and Web3.com and you want to SSL them. Just set up secure.yourdomain.com and reverse proxy the three sites through it..I.E. secure.yourdomain.com/web1/ would pull the content from web1.com, encrypt it and send it to the user. It's not ideal, but it does work when you are IP constrained.

Re:HTTPS requires an IP address per domain

[tepples]

Just set up secure.yourdomain.com and reverse proxy the three sites through it

But when will hosting companies such as Go Daddy offer such a reverse proxy service for customers on their budget plans?

Re:HTTPS requires an IP address per domain

[Vrtigo1]

Budget plans don't give you a dedicated IP, so SSL is a moot point. Those with the need to host multiple SSL websites are probably not using budget plans anyway. A VPS with a dedicated IP is what, $40/month or less? Doing what I suggested with that is more than feasible.

Re:HTTPS requires an IP address per domain

[tepples]

A VPS with a dedicated IP is what, $40/month or less?

Which is still an order of magnitude more than shared hosting. The cheapest HTTPS hosting plans I've found are on this list.

Re:Use https?

[isorox]

... or if you are feeling adventurous, you can always install your own resolver locally. Unless your ISP would hijack requests going to root servers (which is a whole other level of maliciousness)...

Or indeed any traffic on UDP53.

The solution is to therefore tunnel your DNS requests to a known server, or even just put everything through your own personal VPN, and terminate with a decent company.

Who makes theses decisions to hijack search traffi

[Stan92057]

Who makes theses decisions to hijack search traffic? Do any of theses corporation use there lawyers. I mean this is a no brainier stupid/illegal move and why did they think someone wouldn't find out? I have RCN i can say this hasn't happened to me but i don't use the search bar i search right from google.com.

Re:Use https?

[MightyMartian]

Which only helps you if your ISP isn't intercepting and redirect port 53 requests. If an ISP is evil enough to redirect search traffic through some lookalike service, I doubt they'll feel even the slightest twinge at redirecting DNS.

Unfortunately, at that point, the only real solution is surfing via some form of VPN, which has some very real performance consequences.

OpenDNS has been doing this for years

[BitZtream]

Its not like its new, anyone using OpenDNS has been subjected to this bullshit since day one. And for some reasons unknown to me, half of the slashdot user base still thinks opendns is a god send. The same people who were bitching like crazy when Network Solutions started returning itself instead of NXDOMAIN for missing names, everyone was ranting about how OpenDNS is the way to go ... ignoring the fact that they do exactly the same thing ... and its a feature. Idiots.

Re:Criminal

[Tim C]

Hijacking traffic like this is almost certainly a breach of RIPA and the Computer Misuse Act.

Both of which are UK laws.

Re:Netalyzr ?

[nweaver]

We don't say its BAD, we say its interesting: we alert on any non-legit reverse data for any site which would normally have a clean reverse. If you did these changes legitimately, it is a false positive, but since we want to detect all DNS-based blocking & modification of the significant name list, we always alert on these changes.

We check these particular names because there is malcode that changes BOTH these sites to malicious servers, and we alert on any change on theses sites.

Re:warn visitors

[BitZtream]

Yes, find their ISPs ip ranges in the WHOIS database, send a special notice to anyone coming from those IPs. You'll warn a few people that aren't effected like slashdotters with their own resolvers locally, but those people will get it anyway and probably think you're pretty cool for doing so.

IP allocation information is publicly available, though not always easy to find.

Re:Use https?

[EsbenMoseHansen]

That is the purpose of DNSSEC, which is currently being rolled out. Someday, your IP clients will even use it :)

Not only ISP's

[slashdotard]

There's one CLEC in the western US that provides dialup service to ISP's that also intercepts search requests, forwards the search to Yahoo, etc., and alters the search engine returns by changing links and inserting ads. You'd never know what was happening unless you were watching the traffic on the port and noticed that DNS was returning the same IP address for all the search engines.

Re:Netalyzr ?

[JASegler]

Yes because that couldn't be exploited at all..

What's that? The malware/trojan/root kit installed it's own root-certs and is running a proxy listening on 127.0.0.1:80 and 127.0.0.1:443?
That proxy is snarfing up all the data and shipping it off to some other server...

Just because you can't imagine how it could be abused doesn't mean it can't be abused.

Re:Use https?

[afidel]

DNSSEC solves that problem.

OpenDNS ebay hijaking

[krellboy]

A few years ago I had switched one of my private servers over to use OpenDNS instead of the ISP-provided servers, one day an automated ebay script I had mysteriously stopped working. The ebay cgi DNS A record via openDNS was now pointing to a non-ebay server, by manually running the query using wget and spoofing the browser ID (which is what my script did) I could get the page which was a redirect script that added an affiliate tag and then sent the query on it's way to the real ebay server. Needless to say I stopped using OpenDNS, I regret not making more of a stink about it at the time but I had other things on my mind. I should still have all of the intermediate files, logs and results, there isn't any point to OpenDNS denying that this happened and that they were doing it. So use OpenDNS at your own risk and realize that they might at any time also choose to hijack your traffic for their own gain.