Slashdot Mirror


Google Caught in Comcast Traffic Filtering?

marcan writes "Comcast users are reporting 'connection reset' errors while loading Google. The problem seems to have been coming and going over the past few days, and often disappears only to return a few minutes later. Apparently the problem only affects some of Google's IPs and services. Analysis of the PCAP packet dumps reveals several injected fake RSTs, which are very similar to the ones seen coming from the Great Firewall of China [PDF]. Did Google somehow get caught up in one of Comcast's blacklists, or are the heuristics flagging Google as a file-sharer due to the heavy traffic?"

15 of 385 comments (clear)

  1. Not me... by omeomi · · Score: 2, Informative

    I'm on Comcast, and haven't had any problems. Doesn't mean they're not doing it elsewhere, but they don't seem to be doing it here.

    1. Re:Not me... by Shakrai · · Score: 5, Informative

      But in this case it just sounds like they can't figure out how to do it right.

      It's not that they can't figure it out, it's that they aren't even bothering to try and shape traffic. They'd rather interfere with it.

      Back in my ISP days we ran our entire operation (400 dial-in lines and about 60 WISP clients) off two un-bonded T-1s (they went to different POPs for redundancy). We couldn't afford to add more bandwidth at the edge, so I hacked together a traffic shaping setup using Linux. It prioritized ssh, telnet, TCP ACKs, icmp packets, and the VPNs of our business clients. VoIP wasn't a big concern in those days but had it been I would have prioritized it as well. When online gaming started becoming big we started giving that traffic priority over bulk transfers as well.

      The bulk downloaders/p2p'ers didn't notice or complain. They still got the lions share of the bandwidth -- and are you really going to notice if your transfer gets 139KB/s instead of 140KB/s due to that ssh packet moving ahead of you in the queue? During peak hours my T-1s were running at 90-95% of capacity but my users were all still humming along quite nicely, none the wiser. There was more to this then just traffic shaping (we also had a pretty slick squid setup), but the point is we got along just fine with our limited resources.

      If we could fucking do it, then sure as hell Comcast could. They have apparently decided that it's better to block/drop the traffic then shape it. If they had real competition they'd probably pay for this over the long run.

      --
      I want peace on earth and goodwill toward man.
      We are the United States Government! We don't do that sort of thing.
    2. Re:Not me... by ChromaticDragon · · Score: 5, Informative

      I'm rather certain the root of your woes is Comcast. I am not certain it's intentional.

      Furthermore, the problem is very likely far more simple and less sophisticated than this issue of packet spoofing.

      Set up a continuous ping to something "nearby" (your gateway, your DNS ser ver, your neighbor, whatever) in your Comcast network and tee it to a file. Leave it up for days and you'll likely see periods of time where you have no service for patches of time... often long enough to kill sessions.

      I very often have problems with any sort of sessions (SSH, VPN, etc.) staying up for long periods of time because the underlying line level reliability is so poor. I can watch my cable modem logs and see many resets, timeouts, etc.

      I laugh whenever asked about phone service via Comcast. Sadly, however, this pathetic reliability also precludes Vonage and the like. And I find this a bit sad since while I do not consider Comcast capable of running a world class network, I loathe the phone company. Those guys are more competent but much more directly evil.

    3. Re:Not me... by pthor1231 · · Score: 2, Informative

      You can get by their customer ID process really easily. Call in, give them the subscribers phone number, and if they ask the name on the account, tell them. Doesn't necessarily mean its your name. If they ask for more identifying info, just say you are a flatmate with the subscriber, and you are calling on his behalf because of a problem. I have never had an issue with that.

    4. Re:Not me... by zappepcs · · Score: 3, Informative

      choke on it... it IS comcast. Your intermittent problems keeping a session open are inarguably unacceptable in view of the wider experience of broadband users in North America. My provider is rock solid in my area. I regularly keep open as many as 6 sessions that do not see lost packets, never mind service unavailable. for example: active SL connection(s), Vonage call, Internet Radio, NNTP session, and active web browsing. None of these suffer a problem. In fact, the only problems I've had were / are on the wireless links. My microwave and wireless router apparently disagree on the topic of which is more powerful.

      If we look at what is promised, what is purchased, what is possible, and compare that to what is experienced, it is clear that some ISPs suck, and there is a reason that they suck. Suckiness is not 'normal' or 'average' or acceptable. With the FCC ruling to allow multiple ISP connectivity to many homes, the quality of service should improve to prevent customer churn. My advice is to switch if complaints are not resolved if you can. If not, register a complaint with the authority who gave your ISP broadband monopoly in your area. Document the complaint process and responses. The BBB, I believe, can be consulted in cases where they clearly are not giving you what you paid for.

  2. Gmail Notifier by hansamurai · · Score: 4, Informative

    Starting yesterday my Gmail Notifier Firefox extension stopped working at home where we have Comcast, but at work it works just fine. I thought maybe the plugin had broken due to some API changes or something but I thought it was odd it worked one place and not the other. This really seems like it's related and even though I believe Gmail Notifier is a third party extension, it's still accessing Google's servers.

    Comcast is really pissing me off. But what's my other option: Qwest DSL.

  3. Re:It could be technical incompetence by reset_button · · Score: 2, Informative
    It seems like it's not DNS. From the forum:

    I'm in Houston on Comcast and noticed this as well. For the record, I use the OpenDNS servers, so unless multiple DNS servers are having trouble reaching Google, the problem is most likely with Comcast.

    I noticed this same thing in Seattle on Comcast. I use my works DNS so its definitely not a DNS issue as I can do a "ping google.com" and get the ip lookup address. The ping times out but typing the ip address into my browser works.

    I've experienced problems connecting to google for a couple months and have been following the DSL reports thread. DNS has been eliminated from the equation so it appears that the problem is due to some unforeseen consequence of sandvine filtering or some other massive screwup at Comcast.
    The problem is spoofed RST packets.
  4. Re:First hand experience here by ledow · · Score: 5, Informative

    It's called DNS caching.

    Did you actually flush your DNS caches like, say, the one in your router, the one in your linksys box, the one on your PC? You can do it manually but the quickest way for a lot of equipment is to reboot. Hence the suggestion.

    Additionally, it was quite likely google because something on your machine (maybe yourself "trying" the connection) had accessed google while the DNS redirection was in place (that was how they "redirected" you to their page). Once you'd done it once it'd linger until the TTL's had expired all the way back to your computer. Ping, nslookup, etc. would ALL show the Comcast IP until that happened, which could be minutes, hours, days, months, depending on your setup.

    In your case, it looks like it was less than 24-hours, because it worked the next day without having to reboot. If you had rebooted immediately, it would have all worked when it came back up. That's WHY he was telling you that.

    Before you start throwing accusations around, delve into such things just a little bit deeper.

  5. Re:Can we please pay attention to the dates... by marx · · Score: 3, Informative

    You're looking at the date the posters joined the forum, not the date of the post.

  6. Not comcast by The+MAZZTer · · Score: 2, Informative

    Your OWN COMPUTER was redirecting you to Comcast (maybe you should be indignant towards Microsoft? >_>). It's called DNS caching.

    In Windows a simple ipconfig /flushdns can take care of that, although some applications, such as Firefox, keep their own DNS caches which must also be cleared (In Firefox there's a DNS cache timeout in about:config somewhere, you just set it to 0 and then back and that should flush the cache).

    Also the tech was almost right... restarting your computer WOULD have fixed it (since DNS caches are only kept in memory and would have been wiped when you rebooted) although it wouldn't have been the OPTIMAL solution.

    Let me take you through the steps your computer took.

    1. You try to access Google while your billing issue is present.
    2. The Comcast DNS server gets your request for www.google.com.
    3. The DNS server sees you haven't been paying your bills (so they think, anyways) so instead of returning the IP address of google.com, it returns the IP address of the Comcast server.
    4. Your computer receives this address. It has no way of knowing it's not really Google.
    5. It saves the address in the DNS cache so it won't have to look it up later.
    6. Your computer connects to this IP address and requests the webpage.
    7. The Comcast server returns a boilerplate "GIVE ME MONEY" page.
    8. Time passes and you fix the billing problem.
    9. The Comcast servers take you off the "redirect all traffic to Comcast" list so all future DNS requests will be correct.
    10. You try to access Google again.
    11. Your computer notes that you've already accessed this website, so it already knows the IP address (so it thinks). It skips the DNS step and uses the already known IP address (which is actually Comcast's).
    12. Your computer connects to this IP address and requests the webpage.
    13. The Comcast server returns a boilerplate "GIVE ME MONEY" page.
    14. You call tech support and complain, and fail to implement the proposed solution.
    15. You leave for the airport.
    16. Your computer (assuming you left it on) notes that it's been a while since you DNSed www.google.com. Thus it deletes the IP from it's cache, and will requery it again.
    17. You return from the airport and try google.com again.
    18. The Comcast DNS server gets your request for www.google.com.
    19. No billing issue, so it returns the proper address for Google.
    20. Your computer receives this address.
    21. It saves the address in the DNS cache so it won't have to look it up later.
    22. Your computer connects to this IP address and requests the webpage.
    23. Google returns it's homepage.
  7. Wikipedia page by sunderland56 · · Score: 4, Informative

    Someone knowledgeable about this issue should update the wikipedia page about sandvine.

    The way it's written now, everyone should use Sandvine - it sounds like wonderful software.

  8. Re:Get the facts by sumdumass · · Score: 2, Informative

    Lol.. I wasn't talking only about myself. I surf at -1 and see a lot of the comments modded down. And yes, I know when something is off topic like this is.

    You wouldn't happen to be one of the people I talked about attempting to dispel knowledge of this are you? There we go, the tinfoil hat is back in place and everythign feels right again.

    Either look around or keep your eyes shut. It doesn't matter much to me. But I call them as I see them. I haven't been wrong often.

  9. From the guy in the second link by aderusha · · Score: 2, Informative

    This had me up far too late yesterday trying to figure out WTF is going on.

    Here's the condensed version:
    * Pings work fine, other websites work fine - only HTTP to google.com with a "google.com" host header is affected
    * HTTP 1.0 without host header isn't affected
    * Going to one of google's web servers by IP works fine (no "google.com" host header)
    * I am typically seeding torrents and was at the time of each service interruption
    * TCP RSTs follow a specific pattern. 2 RSTs in rapid succession in response to the initial GET statement (1 with a valid SEQ, one with a SEQ in the 12xxx range), followed by a second batch of the same. As the article here states (and as I posted in the linked thread), this matches perfectly with results from the China firewall
    * The problem went away at almost exactly 12:00am EDT this morning (give or take a minute)
    * This is from a Comcast subscriber in Grand Rapids, MI.

    For more detail, visit the thread linked. I have links to the raw packet capture data in .pcap format if you'd like to take a look.

  10. Comcast & DNS by DieByWire · · Score: 2, Informative

    I'm on a Comcast business account. I recently had a problem where a working, light loaded Postfix installation suddenly had 10-20% of my outbound email traffic just hang. Verbose logging showed that the problem always occured at the DNS query stage. Mail sent through a backup server suffered the same fate.

    Using tcpdump showed that all the bad dns queries stopped after 4 frames, while the successful ones went 68 or 70 frames.

    Switching from Comcast's regional DNS servers to their national DNS servers fixed the problem immediately.

    Makes me wonder what they're doing on the regional ones.

    --
    Never shake hands with a man you meet in a fertility clinic.
  11. Re:First hand experience here by PitaBred · · Score: 2, Informative

    Just hoping for an informative here:

    I believe that 4.2.2.1 - 4.2.2.5 (or maybe 6) are all DNS servers for Level3, in case you want multiples available.