Email Servers Will Choke, Says Spamhaus

Rub3X writes, "The legal battle between antispam organization Spamhaus and e360 Insight is heating up. Spamhaus has a user base of around 650 million, and its lists block some fifty billion spam emails per day, according to the project's CEO Steve Linford. Spamhaus CIO Richard Cox says the immediate issue is that if the domain is suspended, the torrent of bulk mail hitting the world's mail servers would cause many of them to fail. More than 90% of of all email is now spam, Cox says, and he doubts that servers worldwide would be able to handle a ten-fold increase in traffic." Others estimate Spamhaus's blocking efficacy as closer to 75%; by this metric spam would increase four-fold, not ten-fold, if Spamhaus went unavailable. The article paraphrases CIO Cox as saying that the service will continue "even if there is a short-term degradation."

Comments

kdawson at it again.

[Inoshiro]

Here's the dnscache (part of the djbdns family) solution: /service/dnscache/root/servers# cat spamhaus.org
216.168.28.44
204.69.234.1
204.74.101.1
204.152.184.186
#

No need to HUP -- once the file is created and filled with those IPs, it'll pick them up automatically. You can easily install dnscache with the other tools on your mail servers for 0 interuption of service.

Cheers.

Re:I say let the spam come

[ray-auch]

Actually, the problem (if you read the lawyers who've written on this) is that originally they _did_ go to court.

IIRC they asked the original (state, district ?) court to move the case to federal.

_Then_ they didn't turn up at the federal court because they _then_ decided they didn't accept its jurisdiction.

Re:I say let the spam come

[cortana]

You're leaving out the part where their solicitors requested the venue change without instructions. AFAIK Spamhaus dismissed them and are taking them to court for creating this whole fucking mess in the first place.

Questioning the Math/Assumptions

[carpeweb]

More than 90% of of all email is now spam
Others estimate Spamhaus's blocking efficacy as closer to 75%; by this metric spam would increase four-fold, not ten-fold, if Spamhaus went unavailable

I think the math is a lot more complicated than this implies. Here's how I'd work it:

• P = % Spam (% of all sent mail)
• S(T) = Total Mail Sent
• S(S) = Spam Sent
• S(N) = Non-Spam Sent
• E(T) = Overall Filter Efficiency (% spam detected, Spamhaus + All Other Filters)
• E(S) = Spamhaus Filter Efficiency (% spam detected, Spamhaus Only)
• E(O) = Other Filter Efficiency (% spam detected, All Other Filters w/o Spamhaus)
• F(T) = Overall Type II Error Rate (% false positive, Spamhaus + All Other Filters)
• F(S) = Spamhaus Type II Error Rate (% false positive, Spamhaus Only)
• F(O) = Other Type II Error Rate (% false positive, All Other Filters w/o Spamhaus)
• R(T) = Total Mail Received
• R(S) = Spam Received
• R(N) = Non-Spam Received

We're interested in R(T) and what happens to it with and without Spamhaus. (Assuming we're still interested at all, since math sometimes does that ...).

With Spamhaus:

• R(T) = R(S) + R(N)
• R(T) = S(S) x [1-E(T)] + S(N) x [ 1-F(T)]
• R(T) = P x S(T) x [1-E(T)] + (1-P) x S(T) x [1-F(T)]

Without Spamhaus:

• R(T) = R(S) + R(N)
• R(T) = S(S) x [1-E(O)] + S(N) x [ 1-F(O)]
• R(T) = P x S(O) x [1-E(O)] + (1-P) x S(O) x [1-F(O)]

The difference, expressed as a ratio of (Without Spamhaus - With Spamhaus)/(With Spamhaus), is

[ P x S(O) x [1-E(O)] + (1-P) x S(O) x [1-F(O)] ] - [ P x S(T) x [1-E(T)] + (1-P) x S(T) x [1-F(T)] ]

Divided By

[ P x S(T) x [1-E(T)] + (1-P) x S(T) x [1-F(T)] ]

The assumptions yielding either the ten-fold or the four-fold increase seem to be that E(O)=0, and of course that false positives don't matter. Even with these assumptions, the math in the OP is a bit fuzzy to me:

• E(O) = 0
• E(T) = E(S)
• F(O) = 0
• F(T) = 0 [i.e., F(S) = 0 as well]
  yields (reducing above ratio):
• [ P x S(T) + [ (1-P) x S(T) ] - [ P x S(T) x (1-E(T)) + [ (1-P) x S(T) ] ]
  
  Divided By
  
  [ P x S(T) x (1-E(T)) + [ (1-P) x S(T) ] ]
• Which Reduces To:
  
  P x E(T) / [ 1 - [ P x E(T) ] ]

The ten-fold increase seems to be predicated upon both P=.9 and E(S)=E(T)=1. However, even if that were true, the increase would actually be nine-fold (.9/.1).

The four-fold increase seems to be predicated upon P=.9 and E(S)=E(T)=.75. However, this would yield about a two-fold increase of

[.9 x .75] / [ 1 - (.9 x .75) ] = 27/13 = 2.08 (approx.)

Factoring in false positives might actually make the Without Spamhaus scenario more dire, but clearly it would be less dire if we assume that E(O) is not zero. A better approximation would use the marginal efficiency of Spamhaus. Even with a generous assumption that Spamhaus catches an additional third of all spams sent (vs. all others without Spamhaus, and ignoring false positives), the overall increase in R(T) looks less than 50% to me (.3/.7, or approximately 43%).

Re:I say let the spam come

[Anonymous Coward]

I hate spam just like the next guy, but when you make a profitable business from spam fighting, you need at least some clue about how the legal system works.

"The" legal system? You make it sound like you think there's only one. Here's a clue: the US legal system is just one of many legal systems in the world. Spamhaus is based in the UK, where we have a somewhat different legal system. It is not reasonable to expect people based outside the USA to know (or care) how the US legal system works.

servers choking...

[ninjaz]

First, some stats on the mail server I use from a year ago yesterday and yesterday:

October 15 2005 :

Pieces of spam blocked by realtime blocklists: 9062

Top blocklists:
sbl-xbl.spamhaus.org 7193
bl.spamcop.net 1648
dnsbl.njabl.org 221

October 15 2006:

Pieces of spam blocked by realtime blocklists: 47429

Top blocklists:
sbl-xbl.spamhaus.org 40631
bl.spamcop.net 5240
dnsbl.njabl.org 1558

As spamhaus is currently rejecting 40631 emails which consequently don't have to be processed by spamassassin, it would be definitely be felt on this server were Spamhaus to become available. In fact, the reason I started using RBLs to begin with was due to one of the Spamhaus ROKSO culprits sending about 20,000 messages per hour to a dictionary list of users at a hosted domain. The server was dying then, but using OpenBSD's pf databases together with the spamhaus SBL, the problem was stopped cold.