Slashdot Mirror
Pakistan YouTube Block Breaks the World
Allen54 noted a followup to yesterday's story about Pakistan's decision to block YouTube. He notes that "The telecom company that carries most of Pakistan's traffic, PCCW, has found it necessary to shut Pakistan off from the Internet while they filter out the malicious routes that a Pakistani ISP, PieNet, announced earlier today. Evidently PieNet took this step to enforce a decree from the Pakistani government that ISP's must block access to YouTube because it was a source of blasphemous content. YouTube has announced more granular routes so that at least in the US they supercede the routes announced by PieNet. The rest of the world is still struggling."
PCCW apparently wasn't filtering prefixes announced by PieNet. Very stupid.
There is a NANOG thread about this. Apparently a more specific IP route was advertised.
Here's what the BBC have to say about it.
It goes some way to explaining why YouTube became unavailable, but doesn't go into detail.
The route was announced by AS17557.
Maybe if the article would link to the actual story instead of linking every word to its definition in order to appease the losers who are incapable of tying their own shoelaces or using google, it would be easier to understand for those of us who actually DO have technical know-how and don't need it spoonfed to us.
http://arstechnica.com/news.ars/post/20080225-insecure-routing-redirects-youtube-to-pakistan.html
Basically, pakistan telecom blackholed the network using BGP to advertise that all traffic to those destinations should go to a nonexistant router in their network. Then, instead of keeping that in their own network, they "accidentally" (never ascribe to incompetence what can be explained by rabid extremists) published their route publicly, where it was picked up by the rest of the internet, because the rest of the internet just assumes that nobody would ever fuck up something as arcane and complex as BGP, and makes no attempt to determine whether or not the origination of a given routing rule matches with who should actually be in charge of routing that network.
A lot has happened since the original story was written.
It's too bad that my comment from yesterday, which links to detailed technical information, is still languishing buried.
Better technical explanations of the event are available from the Renesys blog and Data Center Knowledge. The erroneous IP assignments spread across the net within 1 minute, 45 seconds of its announcement by Pakistan Telecom, according to a timeline by Renesys. It took about 80 minutes for YouTube to inform its providers that the route had been hijacked. YouTube says it is "investigating and working with others in the Internet community to prevent this from happening again."
Probably just search for BGP routing and AS numbers on wikipedia.
All Things Pakistan points out that this may have a political rather than a "cultural" reason - given that a number of videos of election rigging were posted.
The BGP article on Wikipedia is as good a place to start as any. Beyond that you can do some Google searches for it.
Basically BGP is the protocol used by routers to exchange route information with each other. A real oversimplification would involve three networks/routers, A, B and C. C receives it's network connectivity through A. C announces the networks it's responsible for to A, whom aggregates them before announcing them (and it's own networks) to B.
In theory, A shouldn't accept any routes from C for IP addresses not owned by C. Apparently that wasn't the case here though, or Pakistan's little stunt wouldn't have impacted anybody outside of Pakistan.
I want peace on earth and goodwill toward man.
We are the United States Government! We don't do that sort of thing.
http://arstechnica.com/news.ars/post/20080225-insecure-routing-redirects-youtube-to-pakistan.html
Professor Karmadillo Songs of Science
I'll check back for related questions to fill in any blanks later :)
SIG: HUP
To understand why Pakistan took Youtube out, you have to first read the SourceOfAllHumanKnoledge(tm)(c) Wikipedia articles mentioned above. For the lazy ( like me ) here goest a shorter explanation.
BGP is the core routing protocol of the Internet. It's used to connect really big blocks of IP address to the net, telling the rest of the Internet how to reach them. For BGP to work, all BGPs routers have to be connected to one another in a mesh network ( TCP connections ) and must have a unique number in this mesh. This number is unique in the world and is assigned by IANA. ISPs that have such number have the "power" to change global routes, and this nutball Pakistani ISP is one of them. What they did was to tell the internet, by updating a global route, that the route to Youtube is now through some router they had.
What they expected is obvious, they wanted toi redirect all request to Youtube to another site. But instead of propagating a new route to the INSIDE of their network, they updated a global route and screwed everything. Ok, they could have just changed an DNS entry and redirected all DNS queries to their DNS server, thus negating the workaround of using OpenDNS.
If they knew the consequences of changing the route is only relevant on the lawsuite Youtube should put on them. They should loose they AS number immediately. And someone should redraft RFC 4271 to give routes an "owner", so If you receive a update on your router from someone not the owner you just discard it. In this way, an router can only update their give block.
take the '/' off the end.
For large sets, this will be our guide even unto death, for the LORD will work for each type of data it is applied to...
Because multi-homed networks may wish to have finer control over which links traffic comes in on then allowed for with simple static routes. Because my above example (A/B & C) was a drastic oversimplification and the actual internet involves tens of thousands (hundreds?) of different networks connected in different ways and trying to manage static routes for all of them would be virtually impossible.
Imagine if the process of connecting a new network to the internet involved having to update static routing tables in every core router on the internet and you'll start to understand why BGP exists. Hell, there are routing protocols meant for internal usage (OSPF), because static routes become unmanageable if you have more then a few routers/netmasks to contend with.
BGP inherently depends on the honor system - that is the crux of the problemNot strictly. The "honor system" should really be limited to the Tier 1 providers. Anybody else really should be filtering the routes that they will accept. There are already provisions in place to remove the "honor system" from consideration -- it seems that Pakistan's upstream provider choose not to use them.
This really isn't anything new. This kind of stuff has happened before. It's not even unique to the internet either -- the POTS network has a routing protocol used to setup calls/announce which switch is responsible for which number/range. One would suspect that SS7 can be abused by "bad" telcos as easily as BGP can be abused by "bad" ISPs.
I want peace on earth and goodwill toward man.
We are the United States Government! We don't do that sort of thing.
Same mistake I'm occasionally still guilty of, which is leaving a trailing slash on the URL, probably because of the example code in the post screen.
Try it without the slash:
http://en.wikipedia.org/wiki/Border_Gateway_Protocol
For those of you who actually want to know "How they did it?" posted from: Renesys Blog
/24 that has been hijacked to its providers
/25 routes are first seen from 36561
/25 routes from 36561
which was found from Cydeweys which is updating as the story progresses. Both of those sites seem to be running a bit slow, so hesitate before clicking.
Full text of Reneysys: Pakistan hijacks YouTube.
A few hours ago, Pakistan Telecom (AS 17557) began advertising a small part of YouTube's (AS 36561) assigned network. This story is almost as old as BGP. Old hands will recognize this as, fundamentally, the same problem as the http://merit.edu/mail.archives/nanog/1997-04/msg00380.html">infamous AS 7007 from 1997, a more recent ConEd mistake of early 2006 and even TTNet's Christmas Eve gift 2005.
Just before 18:48 UTC, Pakistan Telecom, in response to government order to block access to YouTube (see news item) started advertising a route for 208.65.153.0/24 to its provider, PCCW (AS 3491). For those unfamiliar with BGP, this is a more specific route than the ones used by YouTube (208.65.152.0/22), and therefore most routers would choose to send traffic to Pakistan Telecom for this slice of YouTube's network.
I became interested in this immediately as I was concerned that I wouldn't be able to spend my evening watching imbecilic videos of cats doing foolish things (even for a cat). Then, I started to examine our mountains of BGP data and quickly noticed that the correct AS path ("Will the real YouTube please stand up?") was getting restored to most of our peers.
The data points identified below are culled from over 250 peering sessions with 170 unique ASNs. While it is hard to describe exactly how widely this hijacked prefix was seen, we estimate that it was seen by a bit more than two-thirds of the Internet.
This table shows the timing of the event and how quickly the route propagated (this is actually a fairly normal propagation pattern). The ASNs seeing the prefix were mostly transit ASNs below, so this means that these routes were distributed broadly across the Internet. Almost all of the default free zone (DFZ) carried the hijacked route at least briefly.
18:47:00uninterrupted videos of exploding jello
18:47:45first evidence of hijacked route propagating in Asia, AS path 3491 17557
18:48:00several big trans-Pacific providers carrying hijacked route (9 ASNs)
18:48:30several DFZ providers now carrying the bad route (and 47 ASNs)
18:49:00most of the DFZ now carrying the bad route (and 93 ASNs)
18:49:30all providers who will carry the hijacked route have it (total 97 ASNs)
20:07:25YouTube, AS 36561 advertises the
20:07:30several DFZ providers stop carrying the erroneous route
20:08:00many downstream providers also drop the bad route
20:08:30and a total of 40 some-odd providers have stopped using the hijacked route
20:18:43and now, two more specific
20:19:3725 more providers prefer the
20:28:12peers of 36561 start seeing the routes that were advertised to transit at 20:07
20:50:59evidence of attempted prepending, AS path was 3491 17557 17557
20:59:39hijacked prefix is withdrawn by 3491,
BBC said the outage was only for two hours.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
I debated whether or not to even mention my involvement with him because half of the Democratic Party has yet to accept him, let alone the other half of the electorate.
I'll have to look into what his positions are on our trampled rightsFWIW, he was a civil rights attorney at one point in his life.
because Obama is the kind of "fresh air candidate" who will at least listen to the people.He's still going to need help. To quote him "Good ideas go to Washington to die". We'll never see any meaningful change come out of Washington until we decide to hold our Congressman to account for their actions. Nobody does though. How else do you explain that Congress (as a whole) has approval ratings in the 20s, yet people continue to send their existing Representatives back, year after year?
People are going to need to get involved in the process and speak louder and more forcefully then the special interests/lobbyists that have hijacked Congress. If that happens then I'm actually very hopeful that Obama can manage to unite this country. If it doesn't happen then I still feel that he will be a force for good -- but his grander ideas will probably fall off and die.
I want peace on earth and goodwill toward man.
We are the United States Government! We don't do that sort of thing.
Just about any ISP is going to get themselves a BGP Autonomous System Number and use BGP to communicate with other ISPs.
A long long time ago, when the Internet was smaller and more trusting, long enough ago that I've forgotten the names of the guilty parties, some company in Virginia made a mistake in configuring their router, and announced that their T1 was a really really good route to MAE-East, and about 1/3 of the packets on the Internet decided to go use their T1, for a couple of seconds before it melted... Since then, it's become a Best Current Practice for ISPs to filter out routing announcements from their customers, and most ISPs also filter their peering links with other ISPs, though some are more aggressive about it than others (plus they tend to have limits on how specific a route can be announced, just to keep router table sizes from exploding.)
But even with that, occasional glitches can happen. A couple of years ago, an ISP in South America did a bad job of route summarization (probably using RIP internally, which uses the old Class A/B/C system instead of CIDR), and announced a route for the
It's highly unlikely that PCTL was trying to block YouTube access for the whole world, as opposed to just for their country. That doesn't mean what they did was competent, of course, but it's not too surprising that somebody exported a route to their peers that they really only intended for their customers. Their upstream provider probably should have filtered out the announcements as well. But things like this do happen, and if you're likely to be a major target, either of malice or of incompetence, you need to do the extra work to monitor route announcements that include your address space.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
If you're talking about reverend Åke Green he didn't just say "I think homosexuality is bad", he called homosexuality an "abnormal, a horrible cancerous tumor in the body of society". And he while he did stand trial he was not found guilty of any crime.
/Mikael
Greylisting is to SMTP as NAT is to IPv4
You must have missed the part of history class where they taught about the Lend-Lease Act. The US was very much involved in the war starting in March of 1941, we might not have had boots on the ground but without our help the UK wouldn't have stood much of a chance. Even before the formal act the US had been sending quite a lot of supplies to the UK under various other programs.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
Statistically, Pakistan has the one of the worst records of religious tolerance in the world, and is listed as a country of particular concern by the USCIRF (http://www.uscirf.gov/countries/countriesconcerns/index.html http://en.wikipedia.org/wiki/1971_Bangladesh_atrocities). Even middle-eastern countries are actually doing somewhat better.
l'Homme n'est Rien l'Oeuvre Tout: Gustave Flaubert to George Sand
Indeed. And I've seen some really great stuff happen since number portability came out. Case in point:
Friend has Time Warner's "digital phone" VoIP offering. Switches to Vonage to save money and ports her number. Now Time Warner customers are unable to call her number -- they get a generic error message. Everybody else gets through just fine.
Vonage refuses to do anything about it (their customer service really sucks, doesn't it?) because it's "Time Warner's problem". Time Warner refuses to do anything about it because "You aren't a customer anymore". My friend gets screwed.
Worst part is, there doesn't really seem to be any appeals process for this type of thing, other then leaving Vonage and going to a provider that would actually enforce her number portability rights. At least with regulated POTS service you could likely file a complaint with the state regulatory agency (the PSC here in New York) and get something done that way. No such avenue for people relying on VoIP products.
I want peace on earth and goodwill toward man.
We are the United States Government! We don't do that sort of thing.