Slashdot Mirror
Easing Backbone Traffic By Scanning The Net
A reader writes "Of the schemes being concocted to ease traffic among Internet backbone providers, InterNap Network Services Corp. may have the most ambitious: a setup that bypasses the peering process entirely by scanning the Net for optimal routes.
EEtimes has the full story on their plan."
If they actually go through with this, and it catches on, this will lead to the end of the internet as we know it. In place of the well interlaced internet we know today, we will have numerous, smaller networks, connected only peripherally.
Today, the only time we lose service to Slashdot is when Exodus's subnets start dropping packets. But imagine how it would be if major Internet backbone providers go through with their scheme to bypass peering and choose to route traffic as they see fit?
I forsee a day when European's or those on the west coast of the US don't even have access to Slashdot during high traffic times. If the backbone providers route IP traffic based on how convenient it is for them, we can never be gauranteed continuous service.
So, there are some of the many reasons why this is bad news. And I didn't even discuss the unwholesome possibilities this technology would provide if the government gets involved.
A couple things worth mentioning: what they do is not private peering, its private unidirectional peering, dynamic routing, and a bunch of other very clever technologies. Proof is in the pudding, they've definitely (from experience) got the best connectivity out there.
They peer with all the big networks, but don't allow the big networks to route traffic back through them. They're also not like a typical colocation facility in that they've got a large number (or were planning a large number, at least) of PNAP locations, and they provided mostly leased lines except for a couple of larger data centers. They're really expensive, but you get what you pay for.
I couldn't read the article because the link seemed to be broken, so it may have mentioned this, but last I knew their technology that maps network connectivity and dynamically modifies their packet routing through modification of router tables within the networks they peer with via BGP is all Linux based, and has been from day one.
Its very slick stuff.
Most folks running networks employ "hot potato(e)" routing methods - the idea is to get it off your network at the earliest possible time.
Internap, instead, attempts to minimize transport time, which usually means reducing the number of hops as much as possible. In practice, this means modifying BGP to approximate solving the Travelling Salesman problem. You can't, but you can make a good guess. So, if your ISP uses them, traffic to a server on the other end of the country will probably not have to pass through the major hubs. And if you are communicating with another Internap customer, you buypass the public net entirely. They will sell you SLAs with pretty low maximum latencies.
Not affiliated with them, but I did almost use them for my company.
I forget what 8 was for.
Get a modem like me. Things have to be pretty far gone before you notice that the rest of the internet is moving less than 56K. It works in real life too. That guy going 45 MPH in the right lane of the highway doesn't wait for ANYBODY.
If tits were wings it'd be flying around.
because let's say that goodnet doesnt peer directly with sprintnet. well, guess what? that means you go through a public interexchange, i.e. mae-east, -west, etc.. and well, they're rather congested, and tend to drop packets whenever its convienent for them. By providing a direct peering relationship for these big providers, you cut out the public interexchange. Also, InterNAP pays its providers to carry it's traffic for as *long* as possible, versus the traditional method where providers attempt to dump traffic at the first public peering point. Trust me, there is a *huge* difference between an internap DS3 and a UUNet DS3. You pay alot more, but if you have traffic/applications that need bandwidth that's low latency and low jitter, use internap. (no, I dont work for them. I'm a network engineer for growing DSL/VoIP provider, and they're a godsend.)
void clue();
Frankly, I don't think it's anything revolutionary. If I wanted to emulate their setup, I'd fork out a buttload of cash to some large providers to pay for transit, next I'd hire some guys like Avi, or someone of his calibre to do it right....
--
The unsig!
IANAL but I am a network engineer at a "Tier 1" ISP. This article is a joke. It's very clear that the author doesn't understand what he's writing and is most likely simply regurgitating marketing materials being fed to him from the company that is the subject of the article. They are bleeding money and would probably like their stock price to go up.
The most obvious error is that OC-3 is not 622Mbps, it is 155Mbps. OC-12 is 622Mbps.
How do you think InterNAP gets the 11 major backbones to honor BGP local prefs? Very simply, InterNAP establishes a BGP peering session between its router and one of the routers of the ISP that it is purchasing service from.
Is the software they use revolutionary? Perhaps, but I also know of a major Tier-1 provider that uses some clever software to re-compute static routes for every router on their network every single night rather than use a proper IGP like OSPF or IS-IS. Unfortunately this software is so clever that no one completely understands how it all works. Except for that guy that did the clever bits, and he's long gone.
In the end, InterNAP is very simply a hosting provider that instead of being multi-homed to a couple of ISPs is multi-homed to 11 ISPs. They are doing nothing different than anyone else on the network. Hell, if they convinced all those backbone providers to use MPLS and used that to shunt the traffic to them, I would be impressed. They're just using the same old BGP4 that everyone else is using (Cisco's).
As I understand it, AOL does something like this. I've seen web server access logs that show a user(logged in with a session) accessing a page from an AOL account. Two pictures and the html all went to different IP's, presumably of caching web proxies.
I suppose I'm not too threatening, presently, but wait till I start Nautilus
InterNap is an interesting contradiction. On one hand they loooove linux, they've been a linux shop since just about day one. On the other hand, they've applied for patents on their special BGP routing algorithms. I think such a patent is barely better than a software patent, because as you say, anybody who knows how BGP works can figure it out the basics of they've patented...
When information is power, privacy is freedom.
How to compete with InterNap:
People have been doing this for years, it's called a Tier 2 provider, the only difference here is most tier twos have 2-3 backbone providers, where they get 11.
Now, would you rather be with a provider who has a 11 connections to big players, or would you rather go with a real backbone that peers with 200-400 other providers directly? I mean, it's simple math, if there are 1000 networks on the Internet, and they connect to 11, for 989 you will go through a middleman (the backbones they connect to). If you go with a peering provider you might get direct access to 400, and have a middleman for the other 600.
Here's the other issue. Large providers generally share costs when they peer, making it relatively cheap. InterNap takes a solid stance of buying all their bandwidth. So, if you're a customer, and you use 10 meg more InterNap will have to pay for 10 meg more...where a real backbone will simply have to share costs with their peering parters. Who will be able to upgrade first? Not InterNap.
Bottom line, it's not better, and it costs more. It can be made to look better while they are small though.
If IP multicasting were implimented on not only video/audio applications, but say downloads of large popular files (eg. Redhat ISO images etc.) then I would imagine that internet traffic across backbones would decrease.
Imagine it! "To download the latest version of yourdistro-ver.iso tune in on the half-hour."
---
Desperation is a stinky cologne
It doesn't take any genius to figure this out. A little more complicated are the shortest (weighted) path algorithms, but I really don't see the big deal here. This isn't some huge new great idea. Unless I'm way out to lunch here, in which case instead of just flaming me and mod'ing as flamebait, why not post something useful, like an explanation of what I'm missing.
So the question is: will the majors peer with sufficient bandwidth, and keep upgrading as traffic increases, or will they intentionally keep peering poor to sell their own backbone connections as the best way to reach their "eyeballs"? If you believe the former, InterNap and competitors are dead. If you believe the latter, they'll probably still lose money for a while, but this business will have a niche.
sulli
sulli
RTFJ.
NOW I CAN GET TO SLASHDOT FASTER!
Oh Wait, I forgot, my cable modem will still be capped off and my service will still suck at home.
YEAH!
It's only nP for sufficiently large numbers of nodes. The whole universe isn't wired yet, so we're mostly still talking about cleaning up the congestion between a few major sites in the US (one of the few places with the capital to fund one of these ventures, besides). Like with much of CS, systems rarely match the models with sufficient accuracy to produce all those hairy results we've come to fear and loathe.
internally for optimal routes in their cache hierarchy, and also as a hook into their modified bind (or whatever named they're using) so that www.ak.customer.com always points to the "closest" ghost server to the end-user.
Akamai has many more data points from which to deduce traffic flow information, but internap has higher-quality ones.
Of course the services you can get are different, but I wouldn't be surprised if Internap started offering services akin to what Akamai currently does..
-o
All they really are is one big colocation site/ISP. They just happen to have line from all the major backbone providers. There service is great those heavy hits site that want to make sure the they get the best connection possible.
-- Give him Head? Be a Beacon?
-- Give him Head? Be a Beacon? :P)
(If you can't figure out how to E-Mail me, Don't.
I hate to burst your bubble, but no.
1). They create large private peering points which are in general overutilized and badly managed. Individual, private peers create just as much bandwidth without concetrating routes into a single facility, which also provides more redundancy.
2). Huh? The Tier 1 ISPs (which InterNap is _not_, the Tier 1 ISP which I am employed by does not consider InterNap a peer, but a customer.) all have meshed BGP backbones these days and diverse paths on their backbone trunks. Network redundancy is a simple matter of planning, and nothing revolutionary.
3). Actually, it's called peering. InterNap has to pay for half of these peers with other Tier 2 and smaller-scale Tier 1 carriers which consider them a peer, and they have to pay for bandwidth from the top ISPs who consider them a customer.
The ISP world is much, much different behind the scenes than it is in the ISP's marketing materials. They in NO way portray a truthful picture of the workings of the Internet backbone.
....and what would they be using Linux for? Routers? I sure hope not. Certainly not switches. How would the desktop machine they use in their Noc or as a statistics monitor affect their backbone performance in any way?
//Phizzy
"Most European technology just isn't worth our stealing," -- Former CIA chief James Woolsey, referring to Echelon
This is close to an idea I had. Place web caches/proxies close to major ISP in the network and serve content out of them instead. The Netscape and MSN homepages must be the most heavily hit pages, why not use local caches that update every 2 minutes co-located at major ISP's like Earthlink. More advanced caches could be used for dynamic content like eBay and the various stock services. The end result is faster response time (less traffic, and less distance traveled, but its not like anyone will notice the added 50ms), and a lower network load.
I've seen work into resolving nP complete into polynomial time, but it is at the very basic stages. The best I've seen is, in worst case, cubic time, but it still can't be proven. (That, and most of the nP theory behind the cubic solution is heavily beyond my understanding.)
What you really have to look at is that while the computer is solving this shortest path, it is not loading the page. It has to find the path before it can even load the first graphic or bit of text. And while it is not loading that web page, the user is sitting there waiting. Maybe on a 1 GHz, it takes a lot less time then on my 'old' 166 MHz, but depending on how the nP algorithm is coded, you could still have a lot of time where the browser is just sitting there, apparently doing nothing (at least from the user standpoint).
And I think I can speak for a large chunk of the online populace when I say I find that waiting for a web page to load is one of the more boring things I can think of doing.
Kierthos
Mr. Hu is not a ninja.
I took two tours with them awhile back and was explained the process...
:)
They buy pipes from anyone with more than 1% of the global routing table on the net. They put all of these pipes in a PNAP in a location and they provide full redundancy on all of the links and equipment.
They pull in all of the routes, shoot them to a Linux box that massages the routing tables so that if a customer packet is destined for Alter.net, it will only travel down Alter.net's network, thus bypassing clogged peering edge routers. It doesn't rely on AS-PATH decisions at that point.
The edge peering routers are, traditionally, the most clogged/slow of the links on a providers network. Think about it, are you going to spend more money on your core routers that support YOUR network, or routers that pass global internet traffic to other networks? BBN planet was having these problems this week, in fact at some of their peering routers. It was all broken.
It is really quite an original idea. Very expensive to maintain all of the different links to all of the providers, but they only accept DS3 customers and higher, and you do get VERY good performance.
This isn't that big a deal. If you read the article you'll find that all they're doing is laying connections between the busiest WWW sites.
It's not scanning or anything, just laying new fiber and forcing people to pay. Calling this new technology is like calling a toll road revolutionary.
To me, this is the most interesting point in the whole article: "The money issue is important because to date, no company is turning a profit at providing backbone connections. And InterNap itself is still losing money -- the publicly traded company reported losses of $43.4 million on revenues of $22.5 million for the first six months of this year."
As the backbone providers ratchet rates up to alleviate this red ink, InterNap will start to make more money as demand rises for their colo service (since this means less traffic over the backbones), but I'm most curious how this sort of thing will play out when a business realizes that 90% of its customers are all on one node and why should they pay for backbone traffic at all if they can serve most of their customers without it?
I do not have a signature
This is nothing new. InterNAP has been doing this for years now. Which is why they're so goddamn expensive. But I must say that they offer the *best* data pipes you can possibly get. They peer with 8-9 of the largest providers in each PNAP and your traffic goes to the provider that has the best route. They do an exhaustive systematic search through the global BGP routing table and pick and choose their routes individually. I would assume their route-maps are freaking gigantic. Their technology is unfortunately not real time... (yet. ;) Anyone who knows how BGP works can figure out how they do this.. it seems rather simple (I deal with them on a regular basis) but they came up with it first.
void clue();
I've got some tinfoil and a ten-watt smurf nightlight for you, and I'm prepared to undercut their offer. Is it a deal?