Slashdot Mirror
Google Caught in Comcast Traffic Filtering?
marcan writes "Comcast users are reporting 'connection reset' errors while loading Google. The problem seems to have been coming and going over the past few days, and often disappears only to return a few minutes later. Apparently the problem only affects some of Google's IPs and services. Analysis of the PCAP packet dumps reveals several injected fake RSTs, which are very similar to the ones seen coming from the Great Firewall of China [PDF]. Did Google somehow get caught up in one of Comcast's blacklists, or are the heuristics flagging Google as a file-sharer due to the heavy traffic?"
70% of all "file sharers" use Google. Anyone with even a small background in statistics can see that Google is behind all this piracy. Comcast is simply watching out for our economy. I say good for them. Now if they would only do something about that wretched Slashdot and its wanker community.
See my journal for slashdot ID's by year. Mine created in 2005. http://slashdot.org/journal/289875/slashdot-ids-by-year
After all, doesn't Google host more copyrighted content than any other person/company in the world? ;)
Starting yesterday my Gmail Notifier Firefox extension stopped working at home where we have Comcast, but at work it works just fine. I thought maybe the plugin had broken due to some API changes or something but I thought it was odd it worked one place and not the other. This really seems like it's related and even though I believe Gmail Notifier is a third party extension, it's still accessing Google's servers.
Comcast is really pissing me off. But what's my other option: Qwest DSL.
Reviewing just the first hour of video games.
Hard. Nothing worse than a pissed off multi-billion dollar company suing your ass off. That will teach them.
Knowledge is power. Knowledge shared is power lost.
Is the title clear enough? I can't imagine any judge or jury saying Comcast is allowed to impersonate Google and tell Comcast customers they're not allowed to use Google's services or that Google's services are overwhelmed and shutting down connections. That's essentially what forged, fraudulent RST packets from a MITM attack are doing. That can't possibly be considered a legitimate business practice in court.
If Google were being wrongly flagged, and Google ends up suing the ass off Comcast to put an end to this bullshit.
use connection tracking on this one:
iptables -I INPUT -j LOG -p tcp -m tcp --tcp-flags RST RST -m conntrack --ctstate NEW,INVALID
The fake RST will probably not have a valid sequence number for the established TCP connection, so the Linux stack will flag it as a NEW connection, and the fact that you're getting a RST for a NEW connection should be good enough alarm.
Or maybe it would also work with just the matching code
iptables -I INPUT -j LOG -p tcp -m tcp --tcp-flags RST RST -m state --state NEW,INVALID
What do y'all think?
When loading a Google Page, an intermediate page pops up saying
"Your ISP is interfering with the transmission of data requested from Google our users, and as a result we are unable to consistently provide advanced services to you. You will be redirected to a more basic version of Google's services so that we can provide as much as we can in the manner you have come to expect from us".
Wait 10 seconds, then redirect to Google's non-AJAX pages.
I predict hordes with torches and pitchforks (led by a little old lady with a claw hammer)
"As God is my witness, I thought turkeys could fly." A. Carlson
What if Google, a (justifiably) huge advocate of network neutrality, is deliberately sending the type of RST packets that imitate Comcast's faked packets, specifically to Comcast IP addresses, knowing the inevitable fallout that would result? It would make an already bad situation for Comcast far, far worse, and it's likely that the requested Senate investigation would turn into nails in the coffin for those who want preferential treatment of packets on the Internet.
For a company that does no evil, if they could pull it off, it would be absolutely diabolical. But then, it could easily be one of those "ends justify the means" kinds of situations. At any rate, all I can say is "MWAH HAH HAH HAH HAH!!!! Suckers!"
(No, I don't actually believe that's what's happening, but man, what an AWESOME plan to make network neutrality happen once and for all.)
I have been unable to use Google maps for months now on Comcast. I have called them, but, you can guess how that went. Yahoo maps and Mapquest work fine, but on Google I get about half the tiles filled in before it stops. And I mean it stops. It ends up looking like a checkerboard. Occassionally it will finish a couple of minutes later, but typically it never does.
Getting Comcast to fix it seems unlikely.
Do you have ESP?
This looks like it could be extended - add a -j DROP rule after the -j LOG (log the offending packet, and then send it to the bit bucket).
Oh, no! You have walked into the slavering fangs of a lurking grue!
It's called DNS caching.
Did you actually flush your DNS caches like, say, the one in your router, the one in your linksys box, the one on your PC? You can do it manually but the quickest way for a lot of equipment is to reboot. Hence the suggestion.
Additionally, it was quite likely google because something on your machine (maybe yourself "trying" the connection) had accessed google while the DNS redirection was in place (that was how they "redirected" you to their page). Once you'd done it once it'd linger until the TTL's had expired all the way back to your computer. Ping, nslookup, etc. would ALL show the Comcast IP until that happened, which could be minutes, hours, days, months, depending on your setup.
In your case, it looks like it was less than 24-hours, because it worked the next day without having to reboot. If you had rebooted immediately, it would have all worked when it came back up. That's WHY he was telling you that.
Before you start throwing accusations around, delve into such things just a little bit deeper.
You're looking at the date the posters joined the forum, not the date of the post.
I'm on Comcast, and I haven't had any problems either.
I also posted my Comcast anecdote on Slashdot, and haven't been flamed for it yet.
I'm on Comcast and I do notice some unusual "connection reset" errors every now and then. More than I would normally expect, at least. They happen when I'm trying to telnet/SSH into my Linux box from outside, when I try to download something on Steam, in fact during nearly anything that requires a connection to be established for any significant period of time. I never used to have this problem before Comcast assimilated my previous cable provider. Makes me wonder if it's deliberate.
I recently moved from one house serviced by comcast to another and I can tell you there is DEFINTELY something screwy going on, and it's not just bittorrent trafic.
I've done bandwidth tests and my upstream STARTS at a nice 1.5MB/s and then 15 seconds later drops to 30K/s EVERY TIME.
What this does is give false results when people are doing speed tests. When you do your test you get great results (in my case 15Mb/s downstream and almost 2Mb/s upstream) for the first 15 or 20 seconds. Then after that it just BLOWS.
Someone knowledgeable about this issue should update the wikipedia page about sandvine.
The way it's written now, everyone should use Sandvine - it sounds like wonderful software.
It's not that they can't figure it out, it's that they aren't even bothering to try and shape traffic. They'd rather interfere with it.
Back in my ISP days we ran our entire operation (400 dial-in lines and about 60 WISP clients) off two un-bonded T-1s (they went to different POPs for redundancy). We couldn't afford to add more bandwidth at the edge, so I hacked together a traffic shaping setup using Linux. It prioritized ssh, telnet, TCP ACKs, icmp packets, and the VPNs of our business clients. VoIP wasn't a big concern in those days but had it been I would have prioritized it as well. When online gaming started becoming big we started giving that traffic priority over bulk transfers as well.
The bulk downloaders/p2p'ers didn't notice or complain. They still got the lions share of the bandwidth -- and are you really going to notice if your transfer gets 139KB/s instead of 140KB/s due to that ssh packet moving ahead of you in the queue? During peak hours my T-1s were running at 90-95% of capacity but my users were all still humming along quite nicely, none the wiser. There was more to this then just traffic shaping (we also had a pretty slick squid setup), but the point is we got along just fine with our limited resources.
If we could fucking do it, then sure as hell Comcast could. They have apparently decided that it's better to block/drop the traffic then shape it. If they had real competition they'd probably pay for this over the long run.
I want peace on earth and goodwill toward man.
We are the United States Government! We don't do that sort of thing.
I'm rather certain the root of your woes is Comcast. I am not certain it's intentional.
Furthermore, the problem is very likely far more simple and less sophisticated than this issue of packet spoofing.
Set up a continuous ping to something "nearby" (your gateway, your DNS ser ver, your neighbor, whatever) in your Comcast network and tee it to a file. Leave it up for days and you'll likely see periods of time where you have no service for patches of time... often long enough to kill sessions.
I very often have problems with any sort of sessions (SSH, VPN, etc.) staying up for long periods of time because the underlying line level reliability is so poor. I can watch my cable modem logs and see many resets, timeouts, etc.
I laugh whenever asked about phone service via Comcast. Sadly, however, this pathetic reliability also precludes Vonage and the like. And I find this a bit sad since while I do not consider Comcast capable of running a world class network, I loathe the phone company. Those guys are more competent but much more directly evil.
There's a lot of guesswork here about what providers may or may not be doing; are there any applications for actually testing ISPs? Such testing apps would discover traffic shaping, port filtering, connectivity, and other traffic modifications by the ISP. Something like a bandwidth tester on steroids.
Putting an extraneous link in front of your posts like you did is spam. Having said that, putting the link into your signature is accepted practice here. It's less annoying and nobody will get upset.
Thanks for adding anecdotal noise to the discussion that adds absolutely nothing to the discussion.
Gee, I think that anecdotal evidence is interesting, especially if you're interested in understanding what rules Comcast uses to decide which packets to block. Questions like: "Is it the whole network or just portions (I suspect just portions)?" or "Is it all the time or during peak demand?" Please try to be civil. If a comment isn't valuable, it won't be modded up. If it is valuable it will.
I like my beverages with warning labels!
choke on it... it IS comcast. Your intermittent problems keeping a session open are inarguably unacceptable in view of the wider experience of broadband users in North America. My provider is rock solid in my area. I regularly keep open as many as 6 sessions that do not see lost packets, never mind service unavailable. for example: active SL connection(s), Vonage call, Internet Radio, NNTP session, and active web browsing. None of these suffer a problem. In fact, the only problems I've had were / are on the wireless links. My microwave and wireless router apparently disagree on the topic of which is more powerful.
If we look at what is promised, what is purchased, what is possible, and compare that to what is experienced, it is clear that some ISPs suck, and there is a reason that they suck. Suckiness is not 'normal' or 'average' or acceptable. With the FCC ruling to allow multiple ISP connectivity to many homes, the quality of service should improve to prevent customer churn. My advice is to switch if complaints are not resolved if you can. If not, register a complaint with the authority who gave your ISP broadband monopoly in your area. Document the complaint process and responses. The BBB, I believe, can be consulted in cases where they clearly are not giving you what you paid for.
Support NYCountryLawyer RIAA vs People
*Comcast phone ringing at head office*
... Uh, um, I- I'll talk to our engineers about getting this straighted up right away... sir.
Comcast Secretary: Hello, thank you for calling Com-
Google Big Cheese: This is Google Inc. calling, I want to talk to whoever's in charge. Now.
Comcast Secretary: I don't know who you think you are but-
Google: Go visit google.com right now.
*secretary visits google.com, google recognizes the comcast head office IP range and serves up a pdf of a lawsuit document (Comcast as defendant) instead of the google homepage*
Secretary: Oh my, one moment please I'll transfer you.
Comcast Big Boss: What? I'm busy lining my socks with money and throwing darts at customer photos.
Google: This is Google Inc. You know why I'm calling.
Comcast: *stutters* y-yes, but we have the right to do whatever we need to, to ensure that our networks....
Google: Seriously?
Comcast: Seriously what?
Google: Seriously, you want to mess with us? Are you sure?
Comcast: *Long pause, and painful griding noises of "thinking"* Well... I think you overestimate how powerful you a-
Google: You have a lot to lose 'my friend'. You have 823 employees using Gmail. 138 office locations on Google Maps, 2,345 website pages indexed by the google search engine that recieve a collective 546 thousand search hits per day from Google Search. You currently rank first for the search term "cable internet" and nearly all your press releases are picked up by Google News. Do I need to go on?
Comcast: *speechless silence*
Google: That's right. And be quick about it. *snaps fingers*
--
(All numbers are made up)
Yeah, that's what I see coming...
Nah, the basic problem is that the bigger the company, the higher the density of PHBs. Once you get to a certain concentration, you hit stupidity critical mass. From the outside it looks like malice, but it's really just highly focused incompetence.
Oh, if that were really the only problem.
There are two kinds of big mistakes you can make: those that are big for a company your size, and those that are just plain big. In a big company with lots of customers, small mistakes are multiplied by volume into just plain big mistakes. If you've got gross revenues of a million dollars, a mistake with a potential $100,000 impact is big for your business, but not that big. You can survive it, you can reestablish credibility with your customers (whom you know face to face) by personally eating a helping of crow in front of each and every one. If you're in a company a 100x as big, you're talking maybe a $10M impact that if laid to the account of any individual employee is a disaster beyond that individual's ability to make right.
That's why large companies can develop a special kind of stupidity, preferring a status quo that is certainly wrong to any alternative that is only probably right. Individuals protect themselves using exactly the same strategy that schooling fish employ. Any decision has to have so many fingerprints on it that firing the people who can be tied to a mistake is like cutting off your right arm. That's why big defense contractors are probably the most bureaucratic organizations on the planet. Ordinary mortals have to make decisions that can have impacts measured in hundreds of millions of dollars. In any such situation, you obviously need a form of collective responsibility, the question is what form it takes. It's all to easy to develop an organization that protects individuals by being unable to detect and respond to most problems. We didn't know about it, if we had we probably couldn't do anything about it, and if we could have, it wasn't my job.
The problem is not that a typical PHB is necessarily stupid. The problem is that organizations are built in a way that rewards people for acting in a stupid way. But stupidity is all too common. Even stupid people can manage to be cunning in bad organizations, because they are problems in an organization built around willful blindness to problems. It's more of a challenge for intelligent people I suppose, because it's hard for people with imagination to find much satisfaction in what it takes to get ahead in these places. It has even been suggested that sociopaths make good managers, which I doubt. But I can well believe that feigned stupidity is better in some cases than the real thing.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.