Slashdot Mirror
Google Caught in Comcast Traffic Filtering?
marcan writes "Comcast users are reporting 'connection reset' errors while loading Google. The problem seems to have been coming and going over the past few days, and often disappears only to return a few minutes later. Apparently the problem only affects some of Google's IPs and services. Analysis of the PCAP packet dumps reveals several injected fake RSTs, which are very similar to the ones seen coming from the Great Firewall of China [PDF]. Did Google somehow get caught up in one of Comcast's blacklists, or are the heuristics flagging Google as a file-sharer due to the heavy traffic?"
Hard. Nothing worse than a pissed off multi-billion dollar company suing your ass off. That will teach them.
Knowledge is power. Knowledge shared is power lost.
If Google were being wrongly flagged, and Google ends up suing the ass off Comcast to put an end to this bullshit.
Maybe Google is including some spoofed information in their packets, to test what Comcast is filtering for (and/or to sabotage the filtering system with false positives). There was a time when it wouldn't have surprised us to see their "Don't be evil" policy extended to this kind of jab at an evil policy elsewhere.
My Photography - http://ian-x.com
The Deathlings (comic) - http://thedeathlings.com
use connection tracking on this one:
iptables -I INPUT -j LOG -p tcp -m tcp --tcp-flags RST RST -m conntrack --ctstate NEW,INVALID
The fake RST will probably not have a valid sequence number for the established TCP connection, so the Linux stack will flag it as a NEW connection, and the fact that you're getting a RST for a NEW connection should be good enough alarm.
Or maybe it would also work with just the matching code
iptables -I INPUT -j LOG -p tcp -m tcp --tcp-flags RST RST -m state --state NEW,INVALID
What do y'all think?
Not sure if this is anything, but I use Google Web Accelerator on Comcast at home. Lately, I have been getting a lot of DNS issues at home with it. When I take my laptop to school, I do not get any DNS issues.
What if Google, a (justifiably) huge advocate of network neutrality, is deliberately sending the type of RST packets that imitate Comcast's faked packets, specifically to Comcast IP addresses, knowing the inevitable fallout that would result? It would make an already bad situation for Comcast far, far worse, and it's likely that the requested Senate investigation would turn into nails in the coffin for those who want preferential treatment of packets on the Internet.
For a company that does no evil, if they could pull it off, it would be absolutely diabolical. But then, it could easily be one of those "ends justify the means" kinds of situations. At any rate, all I can say is "MWAH HAH HAH HAH HAH!!!! Suckers!"
(No, I don't actually believe that's what's happening, but man, what an AWESOME plan to make network neutrality happen once and for all.)
I have been unable to use Google maps for months now on Comcast. I have called them, but, you can guess how that went. Yahoo maps and Mapquest work fine, but on Google I get about half the tiles filled in before it stops. And I mean it stops. It ends up looking like a checkerboard. Occassionally it will finish a couple of minutes later, but typically it never does.
Getting Comcast to fix it seems unlikely.
Do you have ESP?
This looks like it could be extended - add a -j DROP rule after the -j LOG (log the offending packet, and then send it to the bit bucket).
Oh, no! You have walked into the slavering fangs of a lurking grue!
IPSec would thwart this sort of attack (since it encrypts at the IP layer, you can't forge a RST packet in the TCP header). Yeah, it costs more CPU, but that's not a problem for modern PC clients, and I suspect Google can handle it, too. Is it time for this to become SOP?
Now, whether MS would be cooperative in that, I dunno... I know XP supports it, but not too much about configuration specifics.
That's an interesting take on it. And as far as I'm aware there is no DSL provider in the United States doing anything like this. It certainly seems to be the case in the wireless world. The carriers removing or blocking features that may compete with their own content offerings.
One wonders what the solution to this is. Prohibit someone from being in the content business AND the delivery business at the same time? They'd fight you tooth and nail on that -- and you'd have the "free market" types after you as well.
In any case I think they will shoot themselves in the foot in the long run. What happens when all P2P traffic is encrypted and looks like any other encrypted protocol (ssh, ssl, etc)? At that point you may be able to identify WHICH subscriber is using p2p (bittorrent stands out like a sore thumb for the sheer volume of connections it establishes) but how will you identify which individual packet is p2p and shape it? Or will they just start sending random RST packets to ALL your connections, including (as TFA suggests) Google?
If bandwidth IS the issue then in the long run they only have two options. Invest in some upgrades or stop selling "unlimited" service. Personally I'd take the best of both worlds. I'd offer a "premium" package aimed at p2p users (no monthly bandwidth limit and/or higher speeds) and use the money from that to expand my network.
I want peace on earth and goodwill toward man.
We are the United States Government! We don't do that sort of thing.
I'm on Comcast and I do notice some unusual "connection reset" errors every now and then. More than I would normally expect, at least. They happen when I'm trying to telnet/SSH into my Linux box from outside, when I try to download something on Steam, in fact during nearly anything that requires a connection to be established for any significant period of time. I never used to have this problem before Comcast assimilated my previous cable provider. Makes me wonder if it's deliberate.
I recently moved from one house serviced by comcast to another and I can tell you there is DEFINTELY something screwy going on, and it's not just bittorrent trafic.
I've done bandwidth tests and my upstream STARTS at a nice 1.5MB/s and then 15 seconds later drops to 30K/s EVERY TIME.
What this does is give false results when people are doing speed tests. When you do your test you get great results (in my case 15Mb/s downstream and almost 2Mb/s upstream) for the first 15 or 20 seconds. Then after that it just BLOWS.
Past three days, fark.com's loaded just fine from work, but from 4.x.x.x, every page took tens of minutes. First 2-3 kilobytes of HTML come through fine, then it hangs for minutes and times out, or it takes 15-20 minutes for the page to trickle through, one packet at a time. From that same IP, cnn, Slashdot, google, the rest of teh Intarweb works fine. From a LVLT-leased IP, forums.fark.com was bogged down. Simultaneously, from a nearby wireless cafe, forums.fark.com worked just fine, so it wasn't on Fark's end.
I was working from home last week, so I was using my Comcast connection extensively every day. The problems with Google connection happened several times a day. Intermittently, my attempts to connect to www.google.com failed for 5-10 min at a time. Oddly enough, going directly to Google services (Gmail, Notebook, Bookmarks, etc.) worked just fine.
comcast.net search is still powered by google, I wonder if they looked at my search term "comcast [RST]" on the way out?
Apocalypse Cancelled, Sorry, No Ticket Refunds
Thanks for adding anecdotal noise to the discussion that adds absolutely nothing to the discussion.
Gee, I think that anecdotal evidence is interesting, especially if you're interested in understanding what rules Comcast uses to decide which packets to block. Questions like: "Is it the whole network or just portions (I suspect just portions)?" or "Is it all the time or during peak demand?" Please try to be civil. If a comment isn't valuable, it won't be modded up. If it is valuable it will.
I like my beverages with warning labels!
One option is openvpn with the default UDP port for those situations. I use it to connect to work's 1G/1G net connection. Also works great for a-hole hotels (I'm looking at YOU Hotel Valencia in San Jose...) that have their system configured to reset all connections every 3 minutes which makes it impossible to even download email. Morons.
That's the right way to handle traffic in the net - drop the priority for packages that aren't sensitive and promote packages that are sensitive to delays. If the lines are up to their throughput limit this is the way to go, and doing it right will not have any really bad effect on the users.
Intentionally dropping data packages is much more evil since that interferes with the functionality and ultimately drives up the network traffic - not down - since many more packages has to be sent and re-sent to provide communication. Bad network conditions also spins power-users to tweak their network settings to be more aggressive. And if the the conditions gets really bad there is a risk that P2P software developers circumvents this by sending redundant information driving the bandwidth use even higher.
But it also has to be figured out if this really is intentional or if the ISP is using equipment with bugs that actually causes this behavior. Since Google is one of the sites that's frequently used it may be that there is a buffer overflow in a router. And if there is a company policy for a certain vendor and a certain setup of that equipment this has a tendency to spread.
Anyway - one of the interesting things reported is that accessing Google through IP address works fine, but not through the DNS resolution. This makes me suspect that the problem is rather related to a certain server or a DNS resolution problem that triggers this problem. Can be an intermediate DNS server that can't handle load-balancing but instead directs all traffic to a single server, which ultimately gets soaked. (maybe not the server, but the channel to the server).
And ultimately - there possibilities available range from being evil to being stupid. Just the kind of story you can read in Dilbert.
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.