Slashdot Mirror

Oh no? Look at this map of the systems infected by the sapphire worm What does this look like to the terrorists? All of their intended targets hit with some 'inconvenience' as you call it all at the same time. If a terrorist saw this map, I can imagine him sending a donkey up to Bin Laden's hideout with a copy saying, "Now we know how to it just the evil infidels"

Actually 'the Sapphire Worm' was just 376 bytes long. Not much extra code in that assembly program to track an author by.

Hi Barry,

Thanks for doing this interview :)

I'm not really satisfied with your answer to my question about dollar cost of spam, but that's OK, you don't have to satisfy me :)

I did want to clear one thing up. I had written:

"As far as I can tell, SMTP traffic is at most 2-5% of net traffic."

And you responded:

"Your figures for the percentage of bandwidth which is spam are far too low. Others have put the numbers much higher. NewsFactor cites studies putting the figure somewhere between 17 and 38%."

I totally accept that spam is about 17-38% of SMTP traffic, that sounds roughly correct to me.

My point there was that SMTP traffic is a very small fraction of total net traffic.

I haven't found any recent statistics on this -- partly because I don't think anyone publishes these numbers anymore, and partly because it's a real pain to try to find with Google. (Do a search on "SMTP NNTP HTTP bandwidth backbone" and you turn up a zillion ISPs bragging about all the protocols they support and how many backbones they're connected to.)

Here's one example of the crappy data out there, a six-year-old report from a link near a backbone showing that SMTP traffic totaled 2.2% of all network traffic:

http://www.nlanr.net/NA/Learn/popular.html

Here's another survey of a backbone, this one five years old, showing SMTP traffic as 3.3% of all network traffic:

http://traffic.caida.org/Reading/Papers/Inet98/

My point was just that if we're trying to assign a dollar figure to what spam costs an ISP, we might as well ignore connectivity charges, because SMTP itself uses so little bandwidth.

As for what all the other costs add up to... I still don't know.

This worm required rougly 10 minutes to spread worldwide making it by far the fastest worm to date. In the early stages the worm was doubling in size every 8.5 seconds. At its peak, achieved approximately 3 minutes after it was released, Sapphire scanned the net at over 55 million IP addresses per second. It infected at least 75,000 victims and probably considerably more.

I read that and my jaw just dropped.

This worm, from what I've read (these aren't my conclusions; I'm not that smart), did two very interesting things. The first is that it used one UDP to spread: no waiting around for the three-way TCP handshake, no hanging waiting for a reply, just send and move on to the next one. From what I understand, that's pretty new. Second, it caused most of its damage not by trashing filesystems or anything like that, but just by spewing *huge* amounts of traffic.

The first is interesting because as a tactic, it'll almost certainly be copied. The second is interesting because it probably won't be copied.

Well worth your time; it's fascinating -- and frightening -- reading. Get it here:

http://www.caida.org/analysis/security/sapphire

From: "Nicholas Weaver"
Date: Fri, 31 Jan 2003 6:09 PM
To: bugtraq@securityfocus.com
Subject: The Spread of the Sapphire/Slammer SQL Worm
We have completed our preliminary analysis of the spread of the Sapphire/Slammer SQL worm. This worm required roughly 10 minutes to spread worldwide making it by far the fastest worm to date. In the early stages the worm was doubling in size every 8.5 seconds. At its peak, achieved approximately 3 minutes after it was released, Sapphire scanned the net at over 55 million IP addresses per second. It infected at least 75,000 victims and probably considerably more.

This remarkable speed, nearly two orders of magnitude faster than Code Red, was the result of a bandwidth-limited scanner. Since Sapphire didn't need to wait for responses, each copy could scan at the maximum rate that the processor and network bandwidth could support.

There were also two noteworthy bugs in the pseudo-random number generator which complicated our analysis and limited our ability to estimate the total infection but did not slow the spread of the worm.

The full analysis is available at

• http://www.caida.org/analysis/security/sapphire/
• http://www.silicondefense.com/sapphire/
• http://www.cs.berkeley.edu/~nweaver/sapphire/

David Moore, CAIDA & UCSD CSE
Vern Paxson, ICIR & LBNL
Stefan Savage, UCSD CSE
Colleen Shannon, CAIDA
Stuart Staniford, Silicon Defense
Nicholas Weaver, Silicon Defense and UC
Berkeley EECS

This is slashdot, after all...

Look at the actualanalysis:

We also made an attempt to identify problem-prone end user applications. Our analysis helped to find and fix a bug in Microsoft Win2k resolver.

50% of /. posts are duplicates

for more about bad dns data look here what dns server get for stupid data.

Despite the scale of the attack, which lasted about an hour, Internet users worldwide were largely unaffected, experts said.
Indeed, no traffic slowdown, no more than usual support calls. The system works as expected, even under attack.

Worth a read: Caida DNS analysis, and more specifically those graphs. It would be interesting to know which DNS sustained the attack, in regard to the graphs.

Despite the scale of the attack, which lasted about an hour, Internet users worldwide were largely unaffected, experts said.
Indeed, no traffic slowdown, no more than usual support calls. The system works as expected, even under attack.

Worth a read: Caida DNS analysis, and more specifically those graphs. It would be interesting to know which DNS sustained the attack, in regard to the graphs.

Yes, I actually do believe that we are somewhere near the peak. Maybe not quite yet, maybe we've already passed it.

Why? Because of worm propagation history. Slapper is old news by now.

Compare this graph:
http://www.caida.org/analysis/security/code-red/co deredv2_analysis.xml#infectionrate

It shows that CodeReds growth was exponential at the critical time, which measured only a few hours. Days have passed since Slapper hit the 10k mark, and we haven't seen any considerably higher estimates.

Pathchar - It sends multiple ICMP Packages of diferent sizes to diferent hosts and it shows you how much bandwidth you get. Don't confuse with throughput...

Install it and give it a try. It's the smartest tool that I have used so far.

http://www.caida.org/analysis/security/code-red/co deredv2_analysis.xml#animations

and the mov version.

http://www.caida.org/analysis/security/code-red/ne wframes-small-log.mov

http://www.caida.org/analysis/security/code-red/co deredv2_analysis.xml#animations

and the mov version.

http://www.caida.org/analysis/security/code-red/ne wframes-small-log.mov

from the original analysis by David Moore:

UK Mirror
UK FTP
AU Mirror
Flipbook animation (207k .FLI)
Quicktime animation of growth by geographic breakdown (200K .mov {requires QuickTime v3 or newer} )
original www.caida.org gif animation

from the original analysis by David Moore:

UK Mirror
UK FTP
AU Mirror
Flipbook animation (207k .FLI)
Quicktime animation of growth by geographic breakdown (200K .mov {requires QuickTime v3 or newer} )
original www.caida.org gif animation

from the original analysis by David Moore:

UK Mirror
UK FTP
AU Mirror
Flipbook animation (207k .FLI)
Quicktime animation of growth by geographic breakdown (200K .mov {requires QuickTime v3 or newer} )
original www.caida.org gif animation

from the original analysis by David Moore:

UK Mirror
UK FTP
AU Mirror
Flipbook animation (207k .FLI)
Quicktime animation of growth by geographic breakdown (200K .mov {requires QuickTime v3 or newer} )
original www.caida.org gif animation

Interesting that no one has suggested using tools (e.g. one, two) to actually test to see what happens to a network as TCP or UDP traffic increases. There are a variety of ways to test the way both streams perform under various/different network conditions. The tools allow you to configure these metrics to your whim. (Caveat emptor, if you use these tools, there is a VERY good chance that people will notice and either get mad or just blackhole everything form your IP address.)

----
"they said there's too much caffeine in your bloodstream..."

this is a map of AS paths & peering relationships on the internet. take a close look at the center.

just to illustrate the importance of AS701, take a look at this graph of AS paths & peering relationships on the internet. take a close look at the very center. i can only hope that uunet is spun back off as an independent isp, because if microsoft buys it we're all doomed. also, someone mod the parent up.

It's not ridiculous.
see here: http://www.caida.org/analysis/topology/as_core_net work/pics/ascoreApr2002.gif

What this article points to most of all, at least for me, is the need for better tools to map abitrary dynamic non-hierarchical networks. Social networks, interlinked buearocracies, realms of knowledge (that whole noosphere thing), the internet itself, the list goes on. There are specific projects about looking into one or another of these, but few share the tools they develop to do the analysis, and those that do tend to release things very specialized to whatever they're studying.

I know I for one am interested in collecting and mapping several datasets, for intellectual and practical gain, but lack the time, resources, knowledge and skill to develop full dynamic network visualization software (preferably in web-friendly form) all by my lonesome.

So, uh... Hey! You! Open source developers! Get to work, chop chop!

(to pre-emptively answer the 'why don't you start a project then?' question, I'm just an artist with geek tendencies who can write a little code, and I do mean a little)

Actually, it doesn't even do that good. Lookup the IP address of www.slashdot.org (64.28.67.150) on their interactive server. It lists a location in California and domain of Exodus.net.

There's a public database called NetGeo which will convert IP addresses to latitude and longitude locations. I created a script called IP-Atlas to get a visual location of the lat and lon coords.

even if you're using Linux

Actually, that was probably not the best example :). Perhaps I should have said something like "even if you're using 'four years without a remote hole in the default install' OpenBSD."

Heres the analysis of Code Red, 359,000 hosts in under 14 hours.

Some more points to ponder, assuming a hypothetical new Virus ("Outlook worm", to be more accurate) that takes advantage of some new as yet unknown MS exploit:

• Could be released while you're sleeping. Or do you post admins at the servers 24 hours a day watching for suspicous mail activity? Should every company have to?
• Many people leave their computers running at night, with Outlook open, so the worm would spread during the night.
• No AV software currently on the market can intelligently sniff out with 100% accuracy if an email is a new virus, or a legitimate email. How could a mail server possibly always be able to tell the difference between a legit email and a virus? Not all viruses might require obvious things like executable attachments. Only a human can tell the difference reliably, and even then its often far from obvious.
• The worms payload could be formatting hard disks, or a simple time-triggered deleting of everything on the computer and all network shares it can find, or it could be to get and/or crack password hashes and post them to an internet site, or it might be that the payload sends out some of your company's most valuable or private intellectual property to 'everyone in the address book' etc.

Obviously every precaution you take as a sysadmin reduces the *risk* of getting hit by a virus, but the probability will NEVER be 0, unless you unplug your computers from the network. Now, given the potential for such huge amounts of damage (depending on the payload), is it worth taking the risk of using software that has a known track record of disaster? No, a good sysadmin should choose software that has the best track record - the risk is just not worth it.

On July 19, 2001 more than 359,000 computers were infected with the Code-Red (CRv2) worm in less than 14 hours. At the peak of the infection frenzy, more than 2,000 new hosts were infected each minute.

That was "over-hyped?" what would it take for it to be "valid concern?" Yes, Code-Red didn't do the damage it intended to...but it still did a heck of a lot of damage. Claiming that some anti-virus nonsense "top 10" has any bearing on the actual amount of damage done is just stupid.

This problem, among with many, many others, was described in a CAIDA paper, "DNS Measurements at a Root Server." They basically ran TCPDump on root server F, and analyzed the traffic. An amazing number of invalid requests are sent all the time. It really shows how important it is for network admins to correctly set up their name services, but it also identifies problems caused by bugs in software. Very interesting read: http://www.caida.org/outreach/papers/2001/DNSMeasR oot/

K Claffy gave an interesting presentation at the last Nanog that illustrates the futility of the Record Companies Efforts. See, in particular, her graph on file sharing usage.

The result of years of litigation and bad law making :

Napster is shut down, its successors have over 5 times the file sharing volume, and are used perhaps 100 times as much as the "legitimate" pressplay and music net services.

And they call it a famous victory...

There is a program called pathchar which seems to do a pretty good job of characterizing pipe size. I've used this to monitor my DSL bandwidth; PacBell has a 45Mbit line heading out of it's DSLAM's (at least in my area). It was designed to be used with symmetric connections, my DSL line (1.5/128) reports like 330K, but otherwise it's a good start at measuring paths.

From my office to microsoft's ftp servers I was easily able to determine that the slowest link is our T1 bewteen the ISP's T3 and our 10Mbps interface on our external router.

Yup! That was the one. Thanks a lot...

And this slide sums up the whole paper basically - it gives an overview of how the backscatter idea works.

If I remember correctly, there was also a follow up to this paper on ACM regarding some statistical survey of existing DoS attacks and the ones mentioned in this paper.

And moderators - pls mod up the parent to this post, that is a useful link (just in case - http://www.caida.org/outreach/papers/backscatter/i ndex.xml)

Yup! That was the one. Thanks a lot...

And this slide sums up the whole paper basically - it gives an overview of how the backscatter idea works.

If I remember correctly, there was also a follow up to this paper on ACM regarding some statistical survey of existing DoS attacks and the ones mentioned in this paper.

And moderators - pls mod up the parent to this post, that is a useful link (just in case - http://www.caida.org/outreach/papers/backscatter/i ndex.xml)

In this paper, we seek to answer a simple question: "How prevalent are denial-of-service attacks in the Internet today?". Our motivation is to understand quantitatively the nature of the current threat as well as to enable longer-term analyses of trends and recurring patterns of attacks. We present a new technique, called "backscatter analysis", that provides an estimate of worldwide denial-of-service activity.

Before offering up your code as a competitor for the roots, you should make sure it performs adequately. This paper http://cider.caida.org/~evi/sigcomm/paper.html says that F root handled 5000 queries per second, while A root was taking 12,000 queries per second in January. How fast is your code?

Good for us. Let's also assume that half of the Red Hat installations have a security problem (which, given Linux' security is clearly an exageration). This would mean that we have at least (assuming 140000 Code Red boxes at the peak, according to Caida):
140000*1000*2 = 280000000 Linux boxes out there!
And that's even taking an extra-ordinarily high ratio of vulnerability. If we take a more realistic ratio of 1% of RHAT boxes being vulnerable, we get:
140000*1000*100 = 14000000000 Linux boxen!
Now how's that for popularity? These are more than people on earth (including Third World countries where most cannot even afford a computer...), and some have the gall to claim that Linux' market penetration is negligible!

http://www.caida.org/dynamic/analysis/security/cod e-red/index.html

There are still about 100.000 vulnerable (and by now... infected) machines out there.

As of the time of my posting this, there are about 130,000 infected hosts. Go to:

http://www.caida.org/dynamic/analysis/security/cod e-red/index.html

There are still about 100.000 vulnerable (and by now... infected) machines out there.

As of the time of my posting this, there are about 130,000 infected hosts. Go to:

http://www.caida.org/dynamic/analysis/security/cod e-red/index.html

For those of you who like pretty graphs, look at caida's nearly-live graphs: [normal scale] [logarithmic scale]

For those of you who like pretty graphs, look at caida's nearly-live graphs: [normal scale] [logarithmic scale]

• The first version of the worm appeared on July 13 or so.
  • It had an unseeded random number generator, so the IP's it scanned were a fixed sequence -- BUT it contained the code to seed the random number generator; this code was disabled.(*)
  • Its DoS attack was set to bomb a particular fixed IP address, AND not even send the bomb packets if that IP could not be reached
  • It contained code to deface web pages served making its presence very visable well before the bombing attack was scheduled to take place
  • It contained code to deactivate its spread if a particular file (c:\notworm) was present.
  • It contained code to deactivate its spread after the "attack phase" began
• On July 19, a second version was introduced.
  • The second version re-enabled the random number generating seed but was otherwise no less shackled than the first version.
  • This version spread exponentially, with growth finally being limited by the number of susceptible servers connected to the internet and the fact that it reached the time of the "attack phase"
  • This version infected over 359,000 hosts in under 14 hours.

The point? The worm author has carefully controlled the attack to cause alarm but not do real damage. When the initial version failed to cause serious alarm, it was loosened slightly from its shackles but still extremely restrained. More to the point? If the worm author -- or anyone else among the thousands with the technical skills to do so -- chose to, they could DoS basically the whole internet. According to netsizer.com, there are about 121 million internet hosts right now, so that gives a ratio of 1 infected computer to 300 hosts. That sounds like too small of a ratio to DoS all of them, but remember to shut things down all that has to happen is to saturate bandwidth, not overload servers. The only reason we're using the net happily today is that the worm author and others with those skills choose to restrain themselves.

So far, I've counted 11 attacks today, versus 86 in total for last month.

Here's some graphs posted to NANOG earlier today:

http://www.caida.org/analysis/security/code-red/au g1-live-hosts.gif

http://www.caida.org/analysis/security/code-red/au g1-live-hosts-log.gif

http://www.caida.org/analysis/security/code-red/gi fs/cumulative-ts.log.gif

(I don't know if that last one includes the top two, but it's supposed to be the cumulative graph for 19-20 July)

So far, I've counted 11 attacks today, versus 86 in total for last month.

Here's some graphs posted to NANOG earlier today:

http://www.caida.org/analysis/security/code-red/au g1-live-hosts.gif

http://www.caida.org/analysis/security/code-red/au g1-live-hosts-log.gif

http://www.caida.org/analysis/security/code-red/gi fs/cumulative-ts.log.gif

(I don't know if that last one includes the top two, but it's supposed to be the cumulative graph for 19-20 July)

So far, I've counted 11 attacks today, versus 86 in total for last month.

Here's some graphs posted to NANOG earlier today:

http://www.caida.org/analysis/security/code-red/au g1-live-hosts.gif

http://www.caida.org/analysis/security/code-red/au g1-live-hosts-log.gif

http://www.caida.org/analysis/security/code-red/gi fs/cumulative-ts.log.gif

(I don't know if that last one includes the top two, but it's supposed to be the cumulative graph for 19-20 July)

IIRC, doesn't win2k do a default instal of IIS with the service on? (Thought I read that some where, but I don't run Win2k, so I can't verify.) This means that there are plenty of machines that are vunerable and their owners don't know it.

According to stats collected by CAIDA, the top 4 identifiable infected domains, with over 7% of the infections, are home.com (cable), rr.com (cable),t-dialin.net (? dial-up?), and pacbell.net (dial-up and DSL). Add in a few more to the list and you are above 10%.

The way I read this, most of those companies are geared to home and individual users (or fairly small businesses). These people are *NOT* Apache customers (otherwise they wouldn't be infected) but nor would they be the kind to purchase Apache. They are small businesses (home business) or home users that either have a cute web site up for their friends, or don't even know they have IIS running.

These people are the ones that don't know about the updates and couldn't care (but can't figure out why their Quake latency is so high).

So, I am a little afraid about this "slice of the pie." Not only is it potentially bigger than the "official server" base, but also is it less informed, and more of a potential threat.

[What happens if Steve Gibson's WinXP concerns are correct and insecure software is being put in the hands of every Joe/Jane User that allows for/facilitates massive global attacks? (I realize that Steve's issue is slightly different, but I bring it up here as it illustrates that the nature of the "pie" is shifting.)]

______

I can't figure out all this chicken little/sky is falling media coverage (well hey its yet another SCARY Internet story, but still). CNN had an article that kinda made me chuckle. It was a story on ISS founder and "worm splattering" "worm hunter" Chris Klaus. It talked about how the 'patch may not hold' What a great thing to be telling everyone. If a new version of the worm hits and spreads liek wildfire, it will be due to a new vulnerability I'd expect. Amazing how mainstream media tries to cover situations like this.

As for the real threat, I expect there will be a large # of infections tonight/tomorrow. Why? Just look at the analysis at CAIDA They found that the majority of servers infected were from domains used primarily by small businesses and residential users (@home, etc) While many of these will have patched themselves, I'm sure many just restarted when problems arose and the problem went away - problem solved. I mean that's standard MO with a Microsoft OS - if it starts acting strangely, reboot.

The good news is, perhaps ISPs have been able to put plans in place to try and block the worm from spreading. Only time will tell.

Don't get me wrong - I think publicizing this issue is a good thing. But I expect that the problem will not be as awful as the media is trying to protray (Internet slowdown, websites knocked offline, etc)

Of course on the flip side - we know that the patch won't be applied to every IIS server out there - what will be done and by who to track down and irradicate the remaining servers that are still infected or are being re infected day after day? I'd expect hte ISPs but given the service level of many DSL and cable providers - you haev to wonder if they'll all pursue this diligently unless the courts get involved (yuck)

The Code-Red worm is a wake-up call. This exploit demonstrates clearly the need to keep machines up-to-date with security developments

We should assess our response to the attack -- How quickly and reliably can we disseminate news about the threat? How quickly can infected hosts be located, isolated, and repaired? In the case of the Code-Red worm, even windowsupdate.microsoft.com was infected, and many hosts were re-infected during attempts to patch them.

(the last line included in regards to a separate post in this thread).

and now back to mp3s -

talking about Code Red in the file sharing column made me think that it would be interesting to distribute files via http requests in a fashion similar to Code Red's exploit attempts via GET requests.
This hides sharing a file in some other protocol, steganographically transferring a file.

I couldn't find anything out there like that, so i did some quick coding and came up with:
stegweb, a method to use HTTP GET requests and your web logs to distribute files.

the code is sloppy, the idea is impractical, but oh well it was fun to code.

-f

For the person who made it it is just a game, but for some other people it was reality. Lot of people had to go patching up. Lets see it.....360.000 server * 10 minutes each ~4 man year work.

If you have a link that supports that number, I'd be interested in seeing. Of course, 2/3 of all statistics are made-up :).

Let's say that Traceloop does 1 million traces a day. Each one causes 30 out-going and 30 in-coming packets to be sent. That's a total of 60M packets per day, or 700 per second, which is a drop in the bucket. Even if you go up to 1 billion traces, it's still insignificant to the Internet as a whole.

Try http://www.caida.org/tools/visualization/mapnet/. It does some cute java based maps.

A few folks have talked about how we're running out of IPv4 addresses and need IPv6 yesterday. Others are saying "CIDR fixes this, or at least mitigates it."

All I have to offer is data. CAIDA has a chart of the IPv4 address space. Look at all of that wasted space.

IF we could CIDR-ize and allocate IPv4 more efficiently, the problem will go away.

Will we ever go to IPv6? If there's a compelling reason to (and not just "it's better" or "it's more technically correct"), then we will. Otherwise, we'll continue to hack on IPv4 for as long as it'll hold up.