Slashdot Mirror
Lessons Learned From Blaster
CowboyRobot writes "It's been nearly a year since Blaster struck, causing hundreds of millions of dollars in fixes and lost revenue.
Jim Morrison of Symantec goes step-by-step in looking at how the Blaster worm got out of control so quickly, and what lessons can be learned from that event, by studying how one utility company dealt with it." The story is written as a fun, technothriller narrative; here's an snippet: "The laptops, usually out in the field, were always a hit-and-miss proposition to find on the network and deliver a patch or to have the user take the machine to a field office. That meant that on the 16th they could see a flood of traffic launched against Microsoft. The second phase of Blaster, launching a DoS (denial of service) attack against windowsupdate.com, was imminent."
Eheh, I couldn't help but chuckle when I read "Jim Morrison". Totally destroys the seriousness of the article.
All Hail Discordia. Hail Eris. Fnord.
I thought Blaster was a RPC virus, i.e. not one broacast via email? I'm sure that's the one that got me a couple of times before I installed a decent firewall (you have 5 seconds to close all work...). Bloody swine of a thing it was - I'd always seem to be winning at Counterstrike too! (Well, that was my excuse, anyway)
Back when Messenger Service popups happened and started using $80 hardware firewalls that doubled as Internet sharing boxes.
When Blaster hit I was sitting pretty and so was every client that took my advice.
*yawn*
Use Evolution instead of Outlook? Bewa
First off... I personally agree with that statement.
Second... I was working a dead end call center job for an ISP when Blaster was running rampant.
Even though this was a Windows problem (and should have been sent to Microsoft), we trouble shooted it since it did technically stop a customer from getting online.
I think after that, nearly every Joe 6-pack finally realized that the thing in his "Start Menu" called "Windows Update" was something to use often.
A contractor using the guest offices brought Blaster inside. His laptop infected the security-counter image-storage system, which then found its way to the HR server. That in turn spawned the infections to the HR XP laptops where the patch failed.
The first thing you learn in ANY security job is that most breaches are from the inside.
As someone standing right behind the front lines, I will tell you that employees with laptops are the worst. Most end up with administrator access (not that hard to crack if you don't have it). And the fact that they bring their computers home and on the road makes them feel a certain entitlement to install whatever they feel like. Contractors are even worse, since most of the time these laptops ARE their personal PCs. Desktops and servers inside the DMZ are the least likely originators of malware. (Not to say you couldn't surf pr0n on the company mail server as an admin. But then you deserve what you get.)
Network admins need to lock down MAC addresses and start treating their network like the PBX folks. Nothing gets wired except approved company equipment.
Have you Meta Moderated t
If it gets really high, eg 50 or so (your average AOL user) automatically turn on the Windows Firewall, and include a flag on every outgoing packet indicating that the user cannot be trusted to operate their computer in a safe fashion. Webmasters can then block traffic from these PC's at their discretion - Problem solved.
Making the moon less necessary since 1998.
A key paragraph in the story...
"We had to do some research, but we found out that the way we locked down the users prevented the patch from running properly," lamented one of the policy admins. "What we discovered was that the software restriction policy for the local computer allowed only local computer administrators to select trusted publishers. Because our patch agent ran as a pseudo user, the agent did not have the necessary rights. This was causing the failure. We changed the group policy for the HR systems so that we can patch remotely from now on."
Sometimes, locking your system too tightly ends up locking the keys in the car. When you really need something to run, it doesn't...
On the one hand, virus writers are aggressively pursued and prosecuted with claimed damages of billions of dollars; on the other hand, these losses are not included in the TCO of Windows! What gives?
The conference room used for the first discussions had been converted to a war room. The whiteboards were filled with IP addresses gathered by the help desk of systems suspected of being infected and trying to propagate the worm. Another list for all of the nonfunctional pay systems covered the entire portable whiteboard. These systems would have to be patched before they could be used to receive payments again.
:)
Red Alert! All senior officers to the battle bridge. Prepare for saucer seperation in T minus 3 minutes and counting.
Picard: Data, can you locate the origin of infection?
Data: It will take aproximatly 10 minutes to scan each subnet.
Picard: We don't have that kind of time. Number One, options?
Riker: Disconnect the OC3 and raise the firewall, leave no ports open.
Captain: That should buy us some time but we need a better solution than that.
Diana: I am sensing something captain, it feels as if the SUS server has fallen offline, we may have missed the latest patches
Data: Her hypothesis could be correct
We are the Borg, We will assimilate you!
Captain: Damn, and here I was thinking it was The Boy and his nanites again
No offense Wil
Im dreaming ofa big bndwdth, That can resist the
The Blaster Worm awoke before dawn.
He put his boots on.
There are even ATMs that run on Windows.
VPNs can be owned too so can "tursted" links to remote controled system. We had a (XP?) box deep inside our network get compromised with a virus that stayed in memory. It got there over a remote control system from another PC that was sometimes hooked to the net. The box deep inside the network then started hunting for other boxes to own, and it found a NT 4 server that could make outbound connections to the net and it set up a nice little email proxy. Lucky for me, my test network isn't as open as it appeared and my freebsd box clampled down on the outbound smtp traffic. A few new rules later (to let the SMTP traffic appear to go out) and the NT box was trying to spam AOL as fast as it could.
There are some tricky things out there that will take advantage of "internal trust" so my new rule is no PC talks to anything else but its samba, proxy or email server. Windows PC's can't talk to any other Windows PC.
Blaster was a worm, and of worms in general I would say that there is little new to be learned from them. I did learn something new with blaster though.
I was doing some security work for an ISP at the time of blaster. They have a number of Cisco 12000 series GSR routers as well as Foundry Big Iron Switches. For those who are not familiar with the Cisco 12000 series routers, let it be sufficed to say that it is Cisco's biggest, baddest router that stands up to 6 feet tall and comes from the factory with a 4 barrel carburetor, dual testosterone modules and a custom paint job with flames painted on the side (pin stripes are optional). These switches are designed to handle hundreds of gigs of traffic across their backplane and through their interfaces. If the ISP were forewarned that they would be seeing 300 mbps of traffic coming from the MS Blaster worm, they would have said "Bring it on!"
For those of us that aren't CCIE's, Cisco routers and Layer 3 switches have a function called CEF, or Cisco Express Forwarding. CEF is a technology that by its simplest definition caches routes.
If a packet from my computer is destined for yahoo.com, it will first hit the DNS server to resolve the host name to its IP address. My computer will then send packets to my ISP with the destination IP of yahoo.com (66.218.71.198). My ISP's router, presuming it's a Cisco router with CEF enabled, will look at its internet BGP tables and determine the optimal route my packet should take on the internet to arrive at that destination. Once the router has processed the route, it caches it so that all future packets coming from my home IP address, destined for yahoo.com will automatically be routed using the cached route. This takes a tremendous load off the router CPU as each packet no longer needs to be processed by the CPU, hence the term "Express Forwarding".
What the blaster worm did was send out hundreds of thousands of ICMP pings per second. This usually wouldn't be a problem for the router, except for each packet was destined for a unique IP address. What started happening is that each route was looked up, routed, and stored in its cache for future packets - only there weren't any future packets. What happened next was the memory space allocated for caching CEF routes filled up, and once full, the router simply purged its cache so that every packet had to then go to the CPU to be routed. Once this happened, all hell broke loose.
CPU utilization on the routers jumped to 100%, which should never happen under normal conditions, but this was clearly not a normal condition, and the internet came to a crawl.
There we were, with a router that should handle hundreds of gigs across the backplane without breaking a sweat being brought to its knees by 100mb of traffic... it was incredible.
Good security is based upon reality and common sense. Common sense is a function of having common knowledge.