Slashdot Mirror
Duke Research Experiment Disrupts Internet Traffic
alphadogg writes with this excerpt from Network World about an experiment gone wrong which affected a big chunk of internet traffic yesterday morning: "It was kicked off when RIPE NCC (Reseaux IP Europeens Network Coordination Centre) and Duke ran an experiment that involved the Border Gateway Protocol (BGP) — used by routers to know where to send their traffic on the Internet. RIPE started announcing BGP routes that were configured a little differently from normal because they used an experimental data format. RIPE's data was soon passed from router to router on the Internet, and within minutes it became clear that this was causing problems. ... [f]or a brief period Friday morning, about 1 percent of all the Internet's traffic was affected by the snafu, as routers could not properly process the BGP routes they were being sent."
What's there to research? 3D Realms announced publicly in 2001 that Duke Nukem Forever would be released simply "when it's done"
I seem to recall that CERN produces about 1% of all data that goes through the internet every day. Hmm...
1% you say? Ah, so they somehow only affected the non-porn traffic?
Down with the career politician! SUPPORT TERM LIMITS
Yesterday, there were a lot of feedback regarding some really mysterious cuts to popular sites. As .tr Govt. is known to censor Internet, people thought something was wrong at the boxes which does the censoring job.
That experiment really went out of hand I think. And, 1% of Internet in 2010 is... Huge. Really huge.
BGP is, like all routing protocols, very secure in and of itself. The difficulty is that a router peering with all routers on the internet can "inject" bad routes, and the "mail" gets reliably delivered to a wrong address. This is ONLY a difficulty if you can somehow gain access to a router that is directly connected to a backbone, and has peering status. You will have to have your own Autonomous System number also, although I am sure you could fake that.
The only time that I have seen even isolated internet routing issues is due to mis-configuration of the router by the owner. Well, that and the extremely rare (yes, really, it is rare) OWNing of an edge router.
I am not all- knowing on this subject; far from it. If someone has something to add/update/correct, please do.
Maybe, yes. BGP has been identified as vulnerable for a long time, and this is further proof. On the other hand, this research is probably motivated by fixing the problem. But the Internet is no longer something you can just shut down or reboot to upgrade; you must operate on a live patient. It does make you wonder, though, if well-intentioned people can do this trying to help, what somebody malicious could do. Hopefully governments will decline to use this as a weapon - like poisoning the ocean.
Any ISP network engineer has some good BGP stories.
For me I was I fighting for over a year to get some of MY blocks back from another provider. They simply continued to announce the routes for them and made it uttererly worthless. It was also fairly horrible to get any upstream traction against the offender.
Eventually, we simply started announcing the routes for those blocks and caused turmoil for those who were using them. It didn't take long to get that issue cleaned up afterwards. Though it was funny because they had asked my guys to stop announcing.
BGP is a bit of a trust relationship, but it isn't the end of world when everything goes to shit.
Admins will get up for their beds and start clearing issues. Things will be sluggish for a bit, but eventually things work out.
"You should always go to other people's funerals; otherwise, they won't come to yours." -- Yogi Berra
Fake it? Not in the last five years!
unless you know of some BGP peers that refuse the standard peering protocol, 1) they are required to only listen to routes from known surrounding peers, 2) will not be listening to what's being advertised by your router unless you have instructed them ahead of time what AS you manage and what prefixes you will be advertising to them.
if for some strange reason, you manage to be adjacent to a backbone CORE router, and wanted to spend a few years moving traffic from core's to edges of the internet, you could start injecting routes for a short span of time after having been trusted and your metric's lowered, (at some point BGP will fail to converge and your advertisements will begin being ignored by the AS)
for research purposes here in Canada, we have access to a major core router, and are able to inject routes to get traffic routed through a particular peer for a few minutes at a time. wirecapping the lines at that router, we can then monitor for organisational security compliance for penetration testing. (you'd be surprised how often usernames and passwords get sent in clear text, or how often people THINK intra building traffic is being encrypted via a VPN only to find out it's badly midconfigured.)
I too am far from all knowing on the ins and outs of global BGP, but every peering agreement I've read (from about twelve countries and almost a hundred cities) have always been the same. "you are required to listen to ASxxxxx for advertisments for this super block, you are required to listen to these private peers with multi-homing agreements, you are required to advertise with the AS number assigned to you only, you are required to advertise only the prefixes you privately manage, and to contact and update the peers directly adjacent to you if assigned a new superblock. etc"
That's kind of the point isn't it? 1% isn't; a few hundred million is. That's the risk of using percentages: they tend to minimize the significance of the real numbers -- or alternatively, overstate their significance.
http://www.cisco.com/en/US/products/products_security_advisory09186a0080b4411f.shtml
I wouldn't say countless. There have probably only been less than 10 blackhole type events with BGP/routing that affected a significant amount of Internet traffic in the past 15 years. The big one being back in 1997. There is a website somewhere that keeps track of them and explains what happened.
But the Internet is no longer something you can just shut down or reboot to upgrade; you must operate on a live patient.
That's a really important point that often goes undiscussed - it's been suggested that if the Internet did go down (major solar storm, EMP, etc.) that it's not likely that it, or the interconnected systems (electrical grid, etc.) could come back up. Too many race conditions, mostly unknown/undocumented. Sure, eventually it would all get back on track, but it could be weeks-to-months. I'm planning to hike the Appalachian Trail while it gets straightened out. ;)
Hopefully governments will decline to use this as a weapon - like poisoning the ocean.
That sounds like a major societal vulnerability that needs to be patched. Nuclear weapons marked an important turning point in history where governments became too dangerous to keep around.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
Fake it? Not in the last five years!
unless you know of some BGP peers that refuse the standard peering protocol, 1) they are required to only listen to routes from known surrounding peers, 2) will not be listening to what's being advertised by your router unless you have instructed them ahead of time what AS you manage and what prefixes you will be advertising to them.
No. Period, fucking no. Most BGP sessions run between customers and carriers are still basically allowing whatever. Even the big boys basically don't care what you advertise. It would cause too many problems to go and begin filtering, so only regions that seem to have routing DBs (RIPE region) are even remotely participating in this. For the most part, thats a few places here and there, but the carriers will let you do what you want.
Don't believe the hype: BGP is still as weak in public IP as it ever has been. The difference is that if you do decide to hijack someone else's prefixes (don't even include bogons, because the carriers will probably let you advertise those!), everyone will know and you will get your upstream looking at you.
For those of you who don't use Valve's Steam storefront/game launch application, the app has a graph that shows usage rates at various scales. Typically it shows the last 48 hours, and typically the graph is sinusoidal. On Friday morning, at about twenty to eleven and at the top of a wave, connections plunged from 2.2 million to under 300,000, before leaping straight back up to 2 million-odd shortly after eleven.
What "big boys" are you talking about?
for every major carrier that I've worked with, filtering isn't optional, it's mandatory.
at the tier one level, Qwest, AT&T, Sprint and L3 all dampen their allowable routes to what they know the immediate peers will advertise. at tier two, there will be many smaller ISP's who will haply pass routes to whomever wants to advertise them, but is not going to be listening to BGP messages on customer facing ports. (unless that customer has already made an agreement with that peer to make an AS entry on both sides)