Pakistan YouTube Block Breaks the World

Allen54 noted a followup to yesterday's story about Pakistan's decision to block YouTube. He notes that "The telecom company that carries most of Pakistan's traffic, PCCW, has found it necessary to shut Pakistan off from the Internet while they filter out the malicious routes that a Pakistani ISP, PieNet, announced earlier today. Evidently PieNet took this step to enforce a decree from the Pakistani government that ISP's must block access to YouTube because it was a source of blasphemous content. YouTube has announced more granular routes so that at least in the US they supercede the routes announced by PieNet. The rest of the world is still struggling."

Comments

A more technical explanation/discussion is here

[br00tus]

There is a NANOG thread about this. Apparently a more specific IP route was advertised.

Re:But how did they do it?

[Anonymous Coward]

The route was announced by AS17557.

A Better Technical Explanation

[1sockchuck]

Better technical explanations of the event are available from the Renesys blog and Data Center Knowledge. The erroneous IP assignments spread across the net within 1 minute, 45 seconds of its announcement by Pakistan Telecom, according to a timeline by Renesys. It took about 80 minutes for YouTube to inform its providers that the route had been hijacked. YouTube says it is "investigating and working with others in the Internet community to prevent this from happening again."

Political, not religious reasons.

[Spy der Mann]

All Things Pakistan points out that this may have a political rather than a "cultural" reason - given that a number of videos of election rigging were posted.

Re:But how did they do it?

[Shakrai]

The BGP article on Wikipedia is as good a place to start as any. Beyond that you can do some Google searches for it.

Basically BGP is the protocol used by routers to exchange route information with each other. A real oversimplification would involve three networks/routers, A, B and C. C receives it's network connectivity through A. C announces the networks it's responsible for to A, whom aggregates them before announcing them (and it's own networks) to B.

In theory, A shouldn't accept any routes from C for IP addresses not owned by C. Apparently that wasn't the case here though, or Pakistan's little stunt wouldn't have impacted anybody outside of Pakistan.

Arstechnica explains

[rishistar]

http://arstechnica.com/news.ars/post/20080225-insecure-routing-redirects-youtube-to-pakistan.html

Why it broke, in techie

[autocracy]

I submitted this article yesterday while it was happening, but of course at that time details were even more sparse (speed vs. informative.. oh well). Some of the BGP routing information I captured is printed out on Wikinews. The basic idea is that Pakistan Telecon, BGP Autonomous System number 17557 began being chatty, saying that it owned Youtube's netblock. It did this using a /24 routing prefix, whereas Youtube exports its route as a /22 (which it should...). Because the /24 was more specific, it became the primary route of reference. This is similar to the "AS 7007" incident (Google it... there's no one good link) back in the late 1990s (one of two incidents in the history of the Internet that has brought the entire Internet down, IIRC).

I'll check back for related questions to fill in any blanks later :)

Re:But how did they do it?

[Shakrai]

If you already know whose IP address are whose, then what do you need the routing protocol for in the first place?

Because multi-homed networks may wish to have finer control over which links traffic comes in on then allowed for with simple static routes. Because my above example (A/B & C) was a drastic oversimplification and the actual internet involves tens of thousands (hundreds?) of different networks connected in different ways and trying to manage static routes for all of them would be virtually impossible.

Imagine if the process of connecting a new network to the internet involved having to update static routing tables in every core router on the internet and you'll start to understand why BGP exists. Hell, there are routing protocols meant for internal usage (OSPF), because static routes become unmanageable if you have more then a few routers/netmasks to contend with.

BGP inherently depends on the honor system - that is the crux of the problem

Not strictly. The "honor system" should really be limited to the Tier 1 providers. Anybody else really should be filtering the routes that they will accept. There are already provisions in place to remove the "honor system" from consideration -- it seems that Pakistan's upstream provider choose not to use them.

This really isn't anything new. This kind of stuff has happened before. It's not even unique to the internet either -- the POTS network has a routing protocol used to setup calls/announce which switch is responsible for which number/range. One would suspect that SS7 can be abused by "bad" telcos as easily as BGP can be abused by "bad" ISPs.

But how did they do it?

[greedyturtle]

For those of you who actually want to know "How they did it?" posted from: Renesys Blog
which was found from Cydeweys which is updating as the story progresses. Both of those sites seem to be running a bit slow, so hesitate before clicking.

Full text of Reneysys: Pakistan hijacks YouTube.

A few hours ago, Pakistan Telecom (AS 17557) began advertising a small part of YouTube's (AS 36561) assigned network. This story is almost as old as BGP. Old hands will recognize this as, fundamentally, the same problem as the http://merit.edu/mail.archives/nanog/1997-04/msg00380.html">infamous AS 7007 from 1997, a more recent ConEd mistake of early 2006 and even TTNet's Christmas Eve gift 2005.

Just before 18:48 UTC, Pakistan Telecom, in response to government order to block access to YouTube (see news item) started advertising a route for 208.65.153.0/24 to its provider, PCCW (AS 3491). For those unfamiliar with BGP, this is a more specific route than the ones used by YouTube (208.65.152.0/22), and therefore most routers would choose to send traffic to Pakistan Telecom for this slice of YouTube's network.

I became interested in this immediately as I was concerned that I wouldn't be able to spend my evening watching imbecilic videos of cats doing foolish things (even for a cat). Then, I started to examine our mountains of BGP data and quickly noticed that the correct AS path ("Will the real YouTube please stand up?") was getting restored to most of our peers.

The data points identified below are culled from over 250 peering sessions with 170 unique ASNs. While it is hard to describe exactly how widely this hijacked prefix was seen, we estimate that it was seen by a bit more than two-thirds of the Internet.

This table shows the timing of the event and how quickly the route propagated (this is actually a fairly normal propagation pattern). The ASNs seeing the prefix were mostly transit ASNs below, so this means that these routes were distributed broadly across the Internet. Almost all of the default free zone (DFZ) carried the hijacked route at least briefly.

18:47:00uninterrupted videos of exploding jello

18:47:45first evidence of hijacked route propagating in Asia, AS path 3491 17557

18:48:00several big trans-Pacific providers carrying hijacked route (9 ASNs)

18:48:30several DFZ providers now carrying the bad route (and 47 ASNs)

18:49:00most of the DFZ now carrying the bad route (and 93 ASNs)

18:49:30all providers who will carry the hijacked route have it (total 97 ASNs)

20:07:25YouTube, AS 36561 advertises the /24 that has been hijacked to its providers

20:07:30several DFZ providers stop carrying the erroneous route

20:08:00many downstream providers also drop the bad route

20:08:30and a total of 40 some-odd providers have stopped using the hijacked route

20:18:43and now, two more specific /25 routes are first seen from 36561

20:19:3725 more providers prefer the /25 routes from 36561

20:28:12peers of 36561 start seeing the routes that were advertised to transit at 20:07

20:50:59evidence of attempted prepending, AS path was 3491 17557 17557

20:59:39hijacked prefix is withdrawn by 3491,