Slashdot Mirror


Internet Routing, Looming Disaster?

wiredmikey writes "The Internet's leading architects have considered the rapid growth and fragmentation of core routing tables one of the most significant threats to the long-term stability and scalability of the Internet. In April 2010, about 15% of the world's Internet traffic was hijacked by a set of servers owned by China Telecom. In the technical world, this is typically called a prefix hijack, and it happened due to a couple of wrong tweaks made at China Telecom. Whether this was intentional or not is unknown, but such routing accidents are all too common online. While BGP is the de-facto protocol for inter-domain routing on the Internet, actual routing occurs without checking whether the originator of the route is authorized to do so. The global routing system itself is made up of autonomous systems (AS) which are simply loosely interconnected routing domains. Each autonomous system decides, unilaterally, and even arbitrarily, to trust everything it hears from any other AS, to use that information without validation, and to further transmit that information to its other peers..."

41 of 109 comments (clear)

  1. ...news? by phyrexianshaw.ca · · Score: 2, Insightful

    And this is news because?

    This is how the BGP internet functions. the last proposed solution was to centralize the BGP trust tables, which is likely a WORSE solution.

    if you can't trust your peers: go work in another kitchen.

    1. Re:...news? by wiredmikey · · Score: 3, Insightful

      It's not so much news as it is insight. If you're an experienced network expert it may not be surprising, but too many people in the tech world still don't have a clue on some of the challenges, dangers, problems that are happening currently and that we face moving forward with the overall internet infrastructure.

    2. Re:...news? by phyrexianshaw.ca · · Score: 3, Funny

      So it's "omfg, we non-technical people just learned how BGP works! it's scary!"

      seeing something like this coming from an AP site, or Fox, I would have just brushed it aside and ignored it. but really? slashdot?


      Owner: "you mean I can hijack someone else's traffic!!? omfg!!"
      *pays to have someone implement it*
      Owner: "WHY DOESN'T IT WORK!!?"
      Tech: "I have no idea.. it should! I read an article on /. about china doing it!"
      *phone rings*
      ISP: "you seem to have a configuration issue on your equipment, you're trying to advertise routes that belong to someone else. you'll have to get that fixed before we continue routing your prefixes to you. "
      Owner: "omg, [isp] called me.. undo it all..."

    3. Re:...news? by anti-NAT · · Score: 3, Informative

      "If you're an experienced network expert it may not be surprising, ..."

      and they're the people at ISPs who're running it (I used to be one of them). Running the Internet backbone is self regulating, because everybody who does it also has a vested interest in policing it. This article is FUD. The clueless tech people can continue to remain clueless.

      --
      The Internet's nature is peer to peer - 20050301_cs_profs.pdf
    4. Re:...news? by Anonymous Coward · · Score: 2, Funny

      Tech to owner: you mean just fix it?
      Owner. NO NO NO OMG OMG OMG Take it all out, turn it all off, cut all the wire. Cut the electricity to all of it and shotgun the machinery. We have to stop NOW! IT all BorKen!

    5. Re:...news? by protektor · · Score: 2

      I used to run an ISP. I started back a bit after the national science foundation stopped paying for the majority of the backbone. I have seen this happen many many times over the years. Some ISP or backbone provider will be working on a router and fat finger their routing tables, and then suddenly all kinds of traffic is re-routed and then they instantly notice a huge upswing in traffic, or rarely don't notice until they get a phone call, and then its "oh crap I have to fix that quick". Most of the time people didn't freak out as long as you jumped on it right away and fixed it. I even did it once. I instantly notice a huge surge in traffic, looked at the traffic, then routing table, oops typo, better fix that quick. Yes I did get a phone call about it, and told them yes I know why your calling, and yes I made a mistake, and yes I already fixed it. It occasionally happens and people usually instantly notice, and usually it isn't a big deal, since everyone gets right on fixing the problem. I never saw anyone do it on purpose. Everyone jumps on fixing it, because it effects your own and ability to get out to certain parts of the internet, not mention it could flood your connection so nothing usable gets in or out, when you screw up the routing. So no one wants that, and especially not to stay that way.

  2. It's called a filter by Tolaris · · Score: 5, Informative

    No, each ISP chooses what routes to accept from what peers. It's called a filter. Smart ISP use routing databases like RIPE to verify what they'll accept and reject automatically. Others do it by hand. Dumb ones accept updates from peers without filtering. It's this last group that needs to update their practices.

    1. Re:It's called a filter by phyrexianshaw.ca · · Score: 4, Interesting

      That's not entirely true.

      though you choose what MAJOR prefixes you accept routing information for, nobody cares about the /8's.

      If I had say a /24 assigned to me, and I decided to have it routed to my building in Toronto, but then decided to move a /28 to a location in Dallas, what would be the easiest way to go about that?

      if I had enough other locations to assign /28's to, I could simply retrieve an AS number and advertise each /28 to the parents at each location. this would then trail up to the largest area that my /24 exists under, and the traffic would be routed locally to each location.

      sure, many ISP's that you deal with in North America may have policies regarding what exact prefixes you advertise at each peering location, but at some point you become large enough to be "trusted". once you start carrying your own traffic internally is often the breaking point.

      say I decided to lease some dark fiber between my two locations: then suddenly my rates may be cheaper than the existing path the ISP is taking between the two. (HIGHLY unlikely, unless your IT department has WAY too much money and you've got a few ISP's interested in sharing a portion of your pipe, though it can seriously reduce the cost of some 100Mbit customer facing links in some cases)
      this then leads to an interesting predicament: how does one know what prefixes will be advertised over that pipe? sure, each ISP sharing the connection MAY decide to restrict advertisements: but few have the capacity to do so for many of the smaller /24's or /28's that exist. keep in mind that each /16 has 256-/24's which in turn each have 32 /28's each.
      customers don't buy /16's (regularly) they buy a /27-/30. this means that the /8 you oversee as an ISP may have as many as 4,194,000+ /30 prefixes to account for.

    2. Re:It's called a filter by Spazmania · · Score: 3, Interesting

      Not exactly. Most ISPs filter their customers announcements that way, but its highly impractical to implement such filters when peering with other ISPs.

      The solution boils down to:

      1. Temporary filter installed for errant routes
      2. Peering POC at source ISP gets a stern lecture and a depeering threat
      3. Peering is so valuable (and so costly to lose) that peering POC smacks around the person who allowed the leak in the first place.
      4. Mistake repeats because the staff who originally allowed it are incompetent
      5. Source ISP gets depeered so he has to pay for all his Internet traffic via a connection that actually is filtered
      6. Source ISP fires the fool who screwed up in the first place, cancels the customer contract (if it was customer originated).
      7. Source ISP most likely never recovers and ends up being bought out while in or near bankruptcy.

      Okay, so steps 4 onward are an artful exaggeration. But seriously, senior network engineers get really bent out of shape when a peer slips them a bum route.

      --
      Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
    3. Re:It's called a filter by ghjm · · Score: 2

      Cage match: "senior network engineers" vs the government of China. Who are you betting on?

  3. Oh, bullshit... by autocracy · · Score: 5, Informative

    Anybody who touches BGP needs to understand route filtering.

      * Would I trust everything I see from Sprint? Yes.
      * Would I trust anything except what I expect from the local ISP I route to? No.
      * Would I expect Sprint to execute the same filtering as above? Yes.

    BGP nodes should always have filters on their connections that describe what is allowed to be accepted. Every failure I can think of... and I'm sure most notable ones that have happened... have been caused by failure to properly filter incoming routes.

    --
    SIG: HUP
    1. Re:Oh, bullshit... by vuke69 · · Score: 3, Insightful

      In a nutshell, that's pretty much the problem and the solution.

      Tier 1 providers pretty much have no choice but to accept any update from other Tier 1s because they could each legitimately have routes to pretty much any network. It is also each of their responsibilities to make sure they don't get any bunk routes from downstream. One weak link, the chain breaks and, and everyone suffers. Obviously you wouldn't (shouldn't) be accepting a zero bit mask route from anyone; but besides the basic idiot proofing, you have to put a lot of faith in your peers, and their ability/diligence.

      --
      Time is an illusion. Lunchtime doubly so. ~ Douglas Adams
  4. The internet needs an upgrade... by digitaldc · · Score: 2

    ...just like every other aging technology that increases its workload and interoperability on a scale that was never originally intended.

    --
    He who knows best knows how little he knows. - Thomas Jefferson
    1. Re:The internet needs an upgrade... by Monkeedude1212 · · Score: 2

      The problem is not that the amount of traffic increases to stresses it can't handle, those are upgrades we have the technology for but just aren't spending the money - we'll worry about those when it actually becomes a problem.

      The issue they are talking about abstractly is the way trust issues work on the net - and how its possible for a Chinese ISP to get 15% of traffic by saying "I'm super trustworthy".

      In Laymens terms, they want to un-naive the interwebs.

    2. Re:The internet needs an upgrade... by $RANDOMLUSER · · Score: 2

      "rogers" on that one. What's this about "looming"? Internet routing started being a disaster two orders of (traffic) magnitude ago.

      --
      No folly is more costly than the folly of intolerant idealism. - Winston Churchill
  5. 15% by vxice · · Score: 2, Insightful

    before we throw this number around anymore, does anyone know approx. how much internet traffic normally goes through China? is the 15% number 15% more than normal, and additional 15%. a baseline is an incredibly important thing.

    --
    every anarchist is a baffled dictator. Benito_Mussolini
    1. Re:15% by genkaos · · Score: 4, Informative

      Actually it was 15% of the internet's prefixes, not 15% of traffic.

    2. Re:15% by Unequivocal · · Score: 4, Insightful

      From what I've read so far on this, the 15% number is a red herring. The real problem was that China was able to route traffic for domains/networks which it had nothing to do with including dell.com and some US DoD networks. Volume wasn't the main issue (though surely it was causing problems in terms of latency and throughput) -- the main issue was that China was seeing packets that it shouldn't have.

      Now we all know that no one routes traffic over the public internet that it doesn't assume bad actors will see. Right?

    3. Re:15% by camperdave · · Score: 3, Funny

      Now we all know that no one routes traffic over the public internet that it doesn't assume bad actors will see. Right?

      Keanu sees my packets?

      --
      When our name is on the back of your car, we're behind you all the way!
  6. Imminent death of Internet predicted... by EriktheGreen · · Score: 5, Insightful

    It's always amusing when a new pundit discovers exactly how the Internet actually works.

    Until they gain enough technical knowledge to be dangerous, they assume that the Internet is just as Hollywood portrays... A rock-solid utility run by the Government that only PhDs and arcanely skilled teenage geniuses can control or understand.

    Then they discover just how "fragile" it is, and start telling the people who've been making it work all along that they need to straighten up and fly right, or else a major disaster is going to happen. Good thing they told us.

    It's sad that they can't just say "Oh, I guess I didn't understand.". Instead they have to "take charge" of things because otherwise they'd have to accept their own irrelevance, or even (gasp) accept that despite their new-found expertise, they *still* don't really understand.

    So straighten up, Cisco... it's obvious to this guy you don't know what you're doing. Fix that BGP thing and do it NOW, you hear him?

    1. Re:Imminent death of Internet predicted... by abigor · · Score: 2

      You are absolutely right. Reminds me of that hysterical article from a few years back: "Is Linus Killing Linux?"

  7. Where does IPv6 stand in this? by Midnight+Thunder · · Score: 2

    Since we are now getting to the final blocks of IPv4, how does this issue effect IPv6? Is this currently an IPv4 issue or will it impact IPv6 too?

    --
    Jumpstart the tartan drive.
    1. Re:Where does IPv6 stand in this? by Melkman · · Score: 2

      BGP works the same for IPv6 and IPv4, so filtering peers according to trust is still required. However the fragmentation issue is way worse for IPv4. This is because IPv6 allocations are that much bigger. To service 100k customers it is not uncommon for an ISP to use more than 10 IPv4 allocations which normally are not continuous. That is because the ISP can only request extra IPv4 address space from the RIR after he has assigned his current allocation to existing customers. To route those allocations the ISP has to announce more than 10 routes (one for each allocation) with BGP. This is one of the reasons the full internet routing table approaches 500k routes atm. For IPv6 the ISP can probably service all of its customers with 1 or 2 allocations. So the routing table will be about a factor 10 smaller. (the figures are guestimates, but I think they are about right.)

    2. Re:Where does IPv6 stand in this? by zn0k · · Score: 2

      That's only true if you ignore that virtually all businesses of decent size are going to want provider independent space. IPv6 was indeed designed to be strictly hierarchical and to have everyone take ISP IP space - but that doesn't work for larger businesses in practice. Larger businesses need to multihome with multiple providers to protect against provider failure. There are some design proposals out there for 'shims' that would let you run a server on an address from ISP 1 and recover the session with a client to an IP address from ISP 2, but those aren't real yet. The only real solution we have is to give businesses provider independent space that they then announce to both ISPs - and that point you're off worse than you are with IPv4 as there are far more potential routes due to the larger address space.

    3. Re:Where does IPv6 stand in this? by 19thNervousBreakdown · · Score: 2

      More routable IPs means it is harder to route them.

      Not necessarily. Fragmentation is the biggest issue--you can very often collapse a huge number of routes down to one since you only have to worry about the next hop. Resolve those routes as soon as you get the tables, then for everything that shares a prefix, collapse it into a single route. If there's small chunks taken out of it that need to go elsewhere, put them higher up in the priority list.

      But, as we start to run out of addresses, an ISP who needs 4 million addresses is going to have to scrounge from hundreds of tiny prefixes that still exist instead of getting a large contiguous block of addresses. Route collapsing isn't going to have nearly as much effect in that case.

      If IPv6 significantly reduces the address pressure, and it should unless those giving out prefixes are completely incompetent, complex routing tables will be able to be made simpler again. There will still be fragmentation issues for PI addresses, but hopefully those are all under a single dedicated prefix so they at least don't make the rest of the space worse as well.

      --
      <xml><I><am><so><damn>Web 2.0</damn></so></am></I></xml>
  8. Re:Authentication by bhcompy · · Score: 3, Insightful

    Overhead. What might take a few milliseconds now takes a few more milliseconds. Not a problem on your little Belkin router, but when you're routing thousands of packets a second, it adds up. You can be sure there are many interests non-technical in nature that would be against raising their latency, even by milliseconds. Particularly, Wall Street.

  9. This B.S. again? Lies never die ;-) by sribe · · Score: 2

    In April 2010, about 15% of the world's Internet traffic was hijacked by a set of servers owned by China Telecom...

    Except of course that after the initial flurry of headlines, analysis showed that the 15% figure was a wild exaggeration, orders of magnitude off...

  10. why not, worked great for the banking system by Anonymous Coward · · Score: 3, Funny

    "would i trust everything i see from bear stearns?"

    yes

    "would i trust everything i see from lehman brothers?"

    yes

    oh wait..

    1. Re:why not, worked great for the banking system by $RANDOMLUSER · · Score: 2

      Sarcasm detected. You really can, however, trust everything you see from Goldman Sachs, since they are The Government.

      --
      No folly is more costly than the folly of intolerant idealism. - Winston Churchill
    2. Re:why not, worked great for the banking system by dgatwood · · Score: 2

      Oh, a sarcasm detector. That's a real (sic) useful invention.

      --Comic Book Guy

      --

      Check out my sci-fi/humor trilogy at PatriotsBooks.

  11. Re:Authentication by rakuen · · Score: 2

    I know overhead can be a problem, especially with how much overhead there already is in exchanging information. However, the authentication should only be applied to the packets generated by the routing protocol, and not all packets. Therefore, overhead is limited. That is, unless every packet has to be authenticated. I'm not that far in my studies yet.

  12. Let me get this straight. by Nailer235 · · Score: 2

    When we realize the government has inadequate security we leap together in unison and scream, "Why didn't they fix that loophole before??" But when someone tries to raise awareness about the need to take preventative measures on a large scale, all of a sudden it's "lulz silly journalist." Also, the author is not even a journalist. His name is Ram Mohan, "Executive Vice President and Chief Technology Officer at Afilias, a global provider of Internet infrastructure services including domain name registry and DNS solutions. "

  13. Re:Authentication by TheLink · · Score: 2

    The overhead is only in deciding whether to accept _changes_ to the routing table. If your router design isn't broken, that doesn't have to increase overheads of routing each packet at all.

    For example, say I give you a piece of paper with a list telling you where to send stuff. So you just follow that.

    Later, I could have a long talk with someone about what should be on a new list, but that does not have to affect you at all.

    Once I'm done with that, I pass you the resulting list, and you use it.

    --
  14. Wait a second.... by SirThe · · Score: 3, Informative

    In April 2010, about 15% of the world's Internet traffic was hijacked by a set of servers owned by China Telecom.

    Wasn't there an article yesterday about how this wasn't true?

  15. And yet /. promotes IPv6 by ugen · · Score: 2

    It's amazing that in the same breath (definitely on the same page) there are posts promoting/demanding immediate/accelerated acceptance/implementation of IPv6 and then this.

    People, wake up - there are significant problems running the current, well compacted address space. Things will only get worse when address space becomes extremely sparse and, for all practical purposes, infinite.

    1. Re:And yet /. promotes IPv6 by Raptoer · · Score: 2

      Perhaps, but what choice so we have? Once we run out of v4 addresses we have to do something.
      Also: IPv6 is initially allocated via geographical areas.

      More importantly, it doesn't matter how sparse the table is as long as each section is contiguous. If I know I can send any traffic from (made up protocol) hosts 1 to 1000 to router 1200, and any hosts from 10,000,000 to 10,010,000 to router 4500, then my table is just fine.

      As the life of an address space goes on it will tend to become less compacted, switching to a new one that is huge will make a sparse, but compacted table.

  16. IPv6 & Fragmentation by xkr · · Score: 2

    The author complains about "fragmentation of routing tables," but then goes on to talk about route hijacking. Doesn't IPv6 largely fix routing table fragmentation? (Real question -- hoping for answer.) Route hijacking is largely fixed by good routing filter hygiene, as explained in previous posts. Most routing protocols support encryption, which won't help if a trusted router sends you bad routes, but can at least make sure you can tell the difference between trusted and untrusted route updates. I don't think BGP supports encrypted advertisements. Anybody know?

    --
    I will create a sig when innovation restarts in the U.S.
    1. Re:IPv6 & Fragmentation by xkr · · Score: 2

      Also, IPv6 assigns addresses in geographic blocks, so you can easily tell of routes don't make any sense at all, like US to US routing via China.

      --
      I will create a sig when innovation restarts in the U.S.
  17. Re:n the technical world by sharkey · · Score: 2

    Capitalized letters are prefixes, aren't they?

    --

    --
    "Outlook not so good." That magic 8-ball knows everything! I'll ask about Exchange Server next.
  18. Re:Authentication by Froggie · · Score: 2

    I am a tier 1 ISP and wish to send a packet to Sprint. My peering with Sprint is down (for whatever reason). Comcast tell me they can route to Sprint. I have two options: trust them, or don't trust them.

    I can't actually say that Comcast are advertising a legitimate route to Sprint. But I also can't tell that they aren't snooping all the traffic, or terminating it at drive-by-malware sites, even if the route *is* legitimate. So there has to be trust at the tier 1 level.

  19. maybe something not intended to it by LostMyBeaver · · Score: 2

    I actually implemented BGP in our equipment (I mean wrote the protocol implementation) and since I'm advertising as opposed to handling heavy routing, the title of the article got me thinking a little.

    By now, the top traffic routers are probably facing such a massive problem with fragmentation of address pools, that it has to be getting nearly impossible to perform any form of routing without enormous tables.

    I'm speculating now.

    These days if you (as an ISP) need a new /24 for your customers, it's very likely you can go straight up to a top teir provider and not be able to get that /24 from an existing pool of addresses. So, they'll get those addresses from somewhere else. The /8 you get it from might already be being routed to another top teir. Then the /10 is routed to a second tier provider elsewhere who then sold the /12 to a provider on another provider and so forth and so on.

    A top teir provider might have a routing table of their own which, cached could consume hundreds of megabytes of entries. Since top tier routers may actually have millions of open routes at a given time, the cache has to be HUGE!!! Far more than could fit comfortably in fast RAM (SRAM for example and certainly no in register space within an ASIC).

    The point being that top tier routers, if they worked on more optimal lookup tables would probably be able to handle much higher throughputs reliably. Less dropped packets etc...

    So, I'd say that fragmentation of the address space is probably murder on the top tier routers. It would make a great deal of sense that ICANN or whoever would attempt to start detangling the address space a bit. In the case of ISPs who often have a huge pool of /8s that they got at different periods, they could try and recover all the /8s in a /12 by arranging trades.

    Additionally, it might even be possible to recover a few hundred thousand usuable IPs by consolidating the pools and therefore cutting down the massive number of addresses wasted on line and subnet IPs.

    I'm pretty sure the "what needs to be done" list could be generated programmatically by data mining.

    There'd be a great deal of benefits to it, but at the cost of actually having to do work to make it happen.