Slashdot Mirror


Belkin Router Owners Suffering Massive Outages

An anonymous reader writes: ISPs around the country are being kept busy today answering calls from frustrated customers with Belkin routers. Overnight, a firmware issue left many of the Belkin devices with no access to the customer's broadband connection. Initial speculation was that a faulty firmware upgrade caused the devices to lose connectivity, but even users with automatic updates disabled are running into trouble. The problem seems to be that the routers "occasionally ping heartbeat.belkin.com to detect network connectivity," but are suddenly unable to get a response. Belkin has acknowledged the issue and posted a workaround while they work on a fix.

25 of 191 comments (clear)

  1. Oh hey, consumers! by fuzzyfuzzyfungus · · Score: 5, Funny

    Yeah, we sold an untold number of plastic boxes that don't work correctly if we ever stop responding to pings... Why would that ever be a problem? Companies never go bankrupt, deliberately kill old products, or 'change strategic direction'.

    1. Re:Oh hey, consumers! by gandhi_2 · · Score: 4, Funny

      I'm going to be naming all my deadman switches "heartbeat" now. (:

    2. Re:Oh hey, consumers! by i+kan+reed · · Score: 4, Funny

      It was a part of their advanced simulation program that slipped out. This particular feature was designed to simulate what it's like to be a Comcast customer.

    3. Re:Oh hey, consumers! by ArmoredDragon · · Score: 4, Insightful

      I think the most awesome revelation about this whole thing is that you can stop a lot of people from accessing the internet if you simply DDoS heartbeat.belkin.com.

    4. Re:Oh hey, consumers! by sbrown7792 · · Score: 4, Funny

      Just as long as you ensure that your botnet doesn't use Belkin routers.

  2. Live by the cloud, die by the cloud. by Animats · · Score: 4, Insightful

    Did Belkin tell you their router was dependent on their site being up?

    "When I die, the world ends." - Belkin policy

    1. Re:Live by the cloud, die by the cloud. by Animats · · Score: 4, Interesting

      Why not using that DNS server has fixed the heartbeat ping issue.

      Their router may be trying to distinguish, as Windows and most things that connect through WiFi now have to, between real Internet connectivity and fake Internet connectivity. Fake Internet connectivity is when some WiFi access point hijacks all DNS requests to take you to some login web page or ad. So, many devices try to connect to some known site which produces a known response to verify that they can connect to the outside world.

      It's the choice of "known site", and not having alternatives for it, that's the problem.

    2. Re:Live by the cloud, die by the cloud. by mcrbids · · Score: 4, Interesting

      Fake Internet connectivity is when some WiFi access point hijacks all DNS requests to take you to some login web page or ad.

      So my company presents at trade shows. Trade shows often have Internet service available at ridiculous prices, and frequently, performance is horrible. Often, rather than pay that ridiculous price, we have a laptop set up with the same configuration as our servers, and run with a recent backup copied onto the laptop. This lets us demonstrate our products with a "sandbox" - same as we use for development - without having to bother with the on site Internet.

      Our mobile "server" is set up to wildcard DNS to a locally hosted copy of our website. Other vendors, of course, see our hot spot and figure they can use it to get Internet service on somebody else's dime. When they find that all they can get to is our website and product, it's typical for them to get upset - more than once we've been accused of hacking!

      Now, set up the hot spot with an SSID like "NoInternetHere" as a way of discouraging trouble.

      --
      I have no problem with your religion until you decide it's reason to deprive others of the truth.
  3. Bad Belkin by JohnFen · · Score: 5, Insightful

    No router should ever be dependent on phoning home to a server in order to work. (No router should be engaging in any communication at all that I haven't told it to!) This is BAD - Broken As Designed. I'm awfully glad that I don't use Belkin stuff.

    1. Re:Bad Belkin by Mike_EE_U_of_I · · Score: 5, Informative

      Agreed. I am astonished that this happened. I thought the router companies would have learned their lesson after the SNTP nonsense over a decade ago. Clearly I was mistaken. For those that do not know about that incident, here you go:

      http://pages.cs.wisc.edu/~plon...

    2. Re:Bad Belkin by AmiMoJo · · Score: 4, Interesting

      It's a feature. By pining Belkin's servers they can keep tabs on their customers. See how long they keep their old routers for, what reliability is like, if they replace it with another Belkin etc. Even just knowing the number of active users is valuable marketing data for them.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
  4. Mod parent up. by khasim · · Score: 4, Insightful

    Yeah! Who the fuck thought that was a good idea?

    Sounds more like "all of the internets is broken because this one site won't work" complaint I get all the time.

    It's a ROUTER. If the physical link is up then try pushing packets through it. That's all.

    If you want to show connectivity to a specific site then show that in the diagnostic page on that router. But keep pushing packets.

    1. Re:Mod parent up. by DougOtto · · Score: 4, Informative

      Actually, according to links in the article, it DOES still push packets. The issue, specifically, is that it breaks DNS if it can't get it's heartbeat. It's still stupid but the device continues to be a router.

      --
      Solving Unix problems since 1989...
    2. Re:Mod parent up. by Trepidity · · Score: 4, Interesting

      Apple does that too, though on end-user machines. When connecting to wifi, it doesn't enable the connection until it first verifies you're really connected. It does that by trying to pull a specific known Apple URL. If it doesn't get the expected contents, it guesses you're behind a wifi hotspot's login wall, and pops up the "please log in" page. The intent of this is to make sure apps like Dropbox and your email and whatever don't think they're back online and start failing connections, in the time between when you connect to a hotspot wifi and when you log in. But it also means that if Apple's URL goes down, wifi connection will end up with extra hoops to jump through to get it to work.

    3. Re:Mod parent up. by h4ck7h3p14n37 · · Score: 4, Interesting

      I used to work at Bank of America, they had their Internet facing routers set to ping microsoft.com and to remove themselves from the pool if the pings didn't come back. Sure enough, microsoft.com had issues one day and the entire BofA organization lost Internet access.

    4. Re:Mod parent up. by Kazoo+the+Clown · · Score: 4, Funny

      If Microsoft decides to ping BofA to determine if the internet is alive, that'd be just about right.

    5. Re:Mod parent up. by NoMaster · · Score: 4, Informative

      It's a ROUTER.

      No, it's a BELKIN. They've been pulling stupid shit like this for the last 10 years at least , so why anyone should still be surprised is beyond me.

      Remember that - whatever you buy from them, it'll always be a Belkin first and <device> second.

      --
      What part of "a well regulated militia" do you not understand?
    6. Re:Mod parent up. by ShaunC · · Score: 5, Informative

      And adding to the list, Windows does it as well. It's called Network Connection Status Indicator.

      NCSI is designed to respond to changes in network conditions, and examines the status of a network connection in a variety of ways. First it uses an active probe to determine the status. For example, in an active probe NCSI tests connectivity by trying to reach http://www.msftncsi.com/ a simple Web site that exists only to support the functionality of NCSI. Eventually, as other programs begin generating Internet traffic, NCSI switches to a passive monitoring process that assumes responsibility for detecting changes to the network status.

      Every time a network configuration event occurs (meaning that something has changed in the network configuration), the NCSI process performs several tests to identify the network's connectivity status. The first step NCSI performs is a DNS query for www.msftncsi.com. The second step is and HTTP get request for http://www.msftncsi.com/ncsi.t.... This file is a plain-text file and contains only the text "Microsoft NCSI." Last it will perform a DNS query for dns.msftncsi.com.

      The URLs used by NCSI can be changed via Group Policy, i.e. you can have it check for the presence of some local server, so that it doesn't bug the shit out of users on a network without external connectivity. Several weeks ago, Microsoft was having global DNS troubles, and many users reported seeing the "trouble" icon in the tray even though their internet connection was working just fine; the problem was that msftncsi.com wasn't resolving. Whoops.

      --
      Thanks to the War on Drugs, it's easier to buy meth than it is to buy cold medicine!
  5. I quit using Belkin years ago, by pecosdave · · Score: 5, Informative

    any type of device, they won't get my money for even a power strip.

    They earned my boycott honestly years ago. I still haven't let them off the hook. Comments on that article exposed other reasons not to even give them the satisfaction of wiping your ass their products.

    --
    The preceding post was not a Slashvertisement.
  6. Re:One rule comes to mind... by gstoddart · · Score: 5, Funny

    ALWAYS be the master of your own network.

    Dude, such a wasted opportunity to say "Master of your own domain ".

    --
    Lost at C:>. Found at C.
  7. Re:First! by Anonymous Coward · · Score: 5, Funny

    Looks like you were behind a Belkin router.

  8. Re:Ummm - did we forget the obvious? by dunkindave · · Score: 4, Funny

    Many years ago I had a similar problem with Comcast. Their system's DHCP wasn't giving me an address, so I called the tech support number. The person on the phone told me that he couldn't help me with my problem since help with all DHCP issues was only handled through their new online text chat system. I pointed out that I couldn't get to their handy online text chat system because I COULDN'T GET AN IP ADDRESS. His only response was that maybe I could use a neighbor's computer. Sigh.

  9. Arbitrarily breaking HTTP is a bad idea. Who knew? by Behrooz · · Score: 4, Interesting

    Entertainingly enough, I've run into this issue before. You will encounter the same issue when trying to connect the affected Belkin routers through the Cisco Clean Access NAC here (AKA Campus Housing), because devices are quarantined in the VLAN ghetto until successfully authenticated and associated.

    So, these terrible, terrible Belkin routers try to phone home, and when they are unsuccessful they redirect all HTTP requests to the router's administration page. Since sessions are required to authenticate via HTTPS, there is no way to login. Extensive investigation revealed no way to disable this behavior on the client side, SOP for anyone calling with connection problems involving a Belkin router became "Officially unsupported. Return it and get something else that isn't a Belkin."

    I am beyond pleased that this incredibly foolish decision on Belkin's part has come back to bite them in general, and hilariously entertained to see that Belkin's temporary workaround was effectively "spoof DNS traffic to heartbeat.belkin.com to a server on your local network that will pingback to fix your ISP's broken clients"

    --
    "We have to go forth and crush every world view that doesn't believe in tolerance and free speech." - David Brin
  10. Re:In retrospect by Garfong · · Score: 5, Informative

    I think you need to review your boolean logic. !(a && b) is equivalent to !a || !b

  11. Re:Why a fixed hostname? by ledow · · Score: 4, Interesting

    I wrote scripts to do ADSL load-balancing / failover using kernel patches that allowed all kinds of things that aren't in the base kernel. Problem was that our ADSL modems were sucky and wouldn't bring the connection back up when it could have done, so I stuck a £5 Velleman electronics kit and a couple of relays into a box, and with USB control we could reset them from the router itself.

    It ran a school for 5+ years, even able to stick 3G sticks into the list and let it failover to them when a dead connection was spotted. And, handily, the 3G sticks worked as a perfect "text-to-fix" receptor and also sent out and received text messages on behalf of the school at the same time. Hell, in one emergency, we even just bought a shed-load of SIMs and every time we hit 1Gb data on a SIM, it turned it off, we changed the SIM for the next and threw the first one away (to get around stupid low data limits). We literally didn't have anyone know we'd done it, from inside the school, except myself and the bursar who bought the SIMs. Everything just worked seamlessly.

    But, just watch number of packets incoming on connection. It's much easier. If your external DNS is down, you aren't going anywhere anyway, without manual intervention. If the root DNS is down, you're fucked. If traceroute can't trace to your ISP's gateway, you're probably dead anyway. All of these work, there's no need to get too complex and ping-out.

    So if you aren't getting DNS packets coming back from simple queries, you might as well consider the connection dead and move onto the next. That's what we did. Then a few second later you'd hear a click, the lights would flash on one of the modems, and in a couple of minutes it's would be back up and pass traffic again.

    The biggest problem? We had to put TWO IP's on the external VPN list because you were never quite sure what line was up and handling the route for the VPN. It could be either. Plug both in, let the VPN client try both. End of problem.

    Was so sad when I realised that I'd left the hardware and scripts for that at my previous workplace.