Microsoft Worms and Global Routing Instability

James Cowie writes: "Fresh analysis here indicates that worm propagation periods correlate very strongly with global BGP routing instability, as measured by sustained exponential increases in the number of prefix announcements and withdrawals seen in BGP message traces."

Comments

Story misleading?

[baptiste]

The story seems to imply that the works spread faster because of BGP instability when the paper seems to be saying the BGP instability is being CAUSED by the worms.

In this online note, we summarize our preliminary analysis of the surprisingly strong impact of the Internet propagation of Microsoft worms (such as Code Red and Nimda) on the stability of the global routing system.

Re:Story misleading?

[DCheesi]

Err, no, you're just reading too much into it. The story only mentions a correlation between the two phenomena; there's no implication of causality there. In fact, my impression upon reading it was that the worms cause the instability --probably because that's the only scenario that really makes sense to me.

Re:Story misleading?

[leto]

They say "routing instability" not "BGP instability".

However, further down in the article they mention that people might need to give BGP packets some preference so that they don't get dropped when something like a microsoft virus sweeps through your routers, causing BGP reconnects (and thus BGP instability)

Leto

Caching and port-scanning

[osolemirnix]

I would assume that this effect is in part due to the nature of port-scanning a wide range of IP adresses with a small data packet. This kind of traffic is different from "regular" traffic where a lot more data gets sent along the same route.

Consequently, since routes time out after a while (and get cached), the IP adress sweeping increases the necessity to figure out more seperate routes than usually (or FIFO caches are too small so routes get purged from the cache faster?).

This would logically increase the load on route discovery protocols such as BGP. A whole new class of DoS attacks...

Re:Caching and port-scanning

[figment]

> These same high-end routers often have traffic
> shaping/prioritization features. You'd think >that they could be configured so that the
>routing-protocol packets have a very high
>priority so that they're among the last to be
>dropped even at high load.


Not necessarily. In a lot of cases, mostly with multiple exit routers, it's more desireable for a hosed router to withdraw it's own route, presumably because you have another un-hosed router which can pick up the slack. In most cases, withdrawing a route is a lot better than advertising a route that doesn't work.

Re:Caching and port-scanning

[Salamander]

I think you missed the point of what I was saying. The problem that the original article talked about was BGP traffic getting dropped due to load. If that's happening, you can't add routes, you can't modify routes, you can't withdraw routes. What I was talking about was using existing facilities that allow you to prioritize traffic by type to ensure that the BGP packets get through even if nothing else does. Once you've done that, you can manipulate routes however you want to adapt to conditions.

What's happening now is like allowing emergency vehicles to get stranded in traffic because they don't have lights and sirens. I say give them lights and sirens, let them zip past the regular traffic so they can do something about the conditions that led to the traffic jam.

Re:Caching and port-scanning

[figment]

Very good explanation, but there's one pseudo-misunderstanding that a lot of people didn't pick up on. Routers can normally handle a lot of traffic (well good ones can), but are still susceptible to cpu overload due to the massive ip scanning that these worms do, which overloads the arp subsystem of the router. arp is mainly to blame, not necessarily increased ip traffic.

Assuming that the router has an interface with a larger than /30 subnet, the router has to do an arp request for every ip on that subnet during a scan, and if enough of these ips just don't exist, then it has to wait for a massive amount of timeouts, then rerequests again, etc. Endlessly.

While you suggest that saturated WAN links could be the problem (and it very well could be given enough infected machines and a small enough link), the data i have indicates that most, if not all, of the problems within our organization are because of excessive the excessive arp requests. A router at one of our pops doesn't run bgp and our traffic data shows it had plenty of bandwidth, but it's cpu usage was at 100% for 3 hours during the first nimda attack. We see similar cpu increases on other CPE equipment with no dynamic routing or any significant increase in traffic.

(ccie in progress ;)

/. effect in action!

[obidex]

mirror at http://dangermouse.pod4.org/nimda/bgp_instability. html

slashdotted

[kingdon]

I've put up a mirror (article there now, images should be up by the time you read this).

As for the article itself, this kind of published analysis is what makes the internet great - compare with the telephone system where each company keeps (more of) their analysis to themselves and engages in more finger-pointing.

Re:so why weren't they in criticalupdate?

[harvardian]

Criticalupdate is not for server admins. Hotfixes are for server admins.

If you're a server admin and you get your security updates from criticalupdate, your intranet is in big trouble.

I saw this at ms downloads..

[Steveftoth]

The top ten downloads according to MS themself are......

Top Downloads
1. Internet Explorer 6
2. Internet Explorer 5.5 Service Pack 2
3. Windows Media Player 7.1
4. Internet Explorer Security Update: (IE 5.5 SP1 and Internet Tools)
5. DirectX for Windows 95, 98 and Windows Me
6. MSN Messenger Service
7. Internet Explorer 5.01 Service Pack 2
8. Internet Explorer Security Update: Late May 2001 5.5 SP1
9. Internet Explorer Security Update: (IE 5.01 SP1)
10. Office 2000 Service Release 1a (SR-1a) Update

Yes.. about half of this list comprises security updates to the MS browser.

Re:Fascinating... (Kill Whitey!)

[superdk]

Additionally, ISPs should start cutting off infected users without hesitation now.

Some ISPs do. I know because I get to cut them off after giving them a warning and ample time to fix the trouble. What's the problem with all of this?

Imagine the following...

Hi, this is Joe Tech from ISP X's Network center, we're seeing that your machine on x.x.x.x is infected with Nimda and this is affecting our network. Your service will be suspended if you don't take care of this.

Customer: uhhhh... how do I fix that? Will the guy at Dell fix it? Why can't you just fix my server and keep this from happening again?

My point, for every 10 business customer's I have only one of them knows A) they even have a web server on their connection B) they had their server's pants down to the whole world C) what nimda is.

besides, people paying business T1 prices don't like being shut off right or wrong.