Slashdot Mirror
Email Servers Will Choke, Says Spamhaus
Rub3X writes, "The legal battle between antispam organization Spamhaus and e360 Insight is heating up. Spamhaus has a user base of around 650 million, and its lists block some fifty billion spam emails per day, according to the project's CEO Steve Linford. Spamhaus CIO Richard Cox says the immediate issue is that if the domain is suspended, the torrent of bulk mail hitting the world's mail servers would cause many of them to fail. More than 90% of of all email is now spam, Cox says, and he doubts that servers worldwide would be able to handle a ten-fold increase in traffic." Others estimate Spamhaus's blocking efficacy as closer to 75%; by this metric spam would increase four-fold, not ten-fold, if Spamhaus went unavailable. The article paraphrases CIO Cox as saying that the service will continue "even if there is a short-term degradation."
It would be interesting if all email server admins suddenly opened the flood gates for a day or two. Maybe then the general population will gain a better appreciate of the scale of the matter.
I still think they 3360 guys just look and smell like spammers. That spamhaus aggrees just adds to this conclusion. Here's what seems to amount to the spam histroy of the "plantiff".
"Thanks for all the money you paid to us. We've used it to buy off ISO among other things" -Microsoft
...tilling for weeds and replacing your entire front yard with rocks.
I hope, when they die, cartoon characters have to answer for their sins.
I am so ready to walk away from email. I just need someone to point me to a workable replacement.
Maybe some legal problems could be avoided by having two lists. One, a list of spammers. The second list is people who are not spammers (cough) who have threatened or engaged in legal action to be removed from the first list. In other words a list of plaintiffs in court cases. Mail server admins could choose whether to use one list or both for blocking mail.
-- Ed Avis ed@membled.com
Here's the dnscache (part of the djbdns family) solution: /service/dnscache/root/servers# cat spamhaus.org
216.168.28.44
204.69.234.1
204.74.101.1
204.152.184.186
#
No need to HUP -- once the file is created and filled with those IPs, it'll pick them up automatically. You can easily install dnscache with the other tools on your mail servers for 0 interuption of service.
Cheers.
--
Internet Explorer (n): Another bug -- that is, a feature that can't be turned off -- in Windows.
holding the pipes and tubes to the internet screaming with his war face "BRING IT ON!"
now thats a slashdot experiance
I can back up the AC's statement. I work for an IT multinational and our e-mail servers run close to the edge. If we were to see a significant increase in e-mail levels, be it x4 or x10, or even x2, our e-mail system would grind to a halt. We, along with every organisation have become totally dependant on e-mail. For example, one of our customers requires that financial information it sent to the Bank of England by close of play every day. It is sent using (encrypted) e-mail. A delay of a few hours would give us major headaches. And yes, we could use alternative methods but it would take some time to put these in place.
If the preditions came true it would be bad for us.
init 11 - for when you need that edge.
Use the UK domain system, e.g. http://www.spamhaus.org.uk/ . It works, and it's not subject to US law.
Wow, looks like an innovative use of BitTorrent...
I'm starting to wonder about the sanity of Spamhaus' lawyers -- or if they really have lawyers at all. So far their arguments seem to have been
1. This case is at the wrong court, it should go to a federal court instead.
2. (to the federal court) We agreed that you had jurisdiction over this, but we're going to pretend that we didn't say that.
3. What? You've decided that we broke the law? Well, you shouldn't punish us because we're really nice people.
While I do not doubt Spamhaus' credentials as really nice people, this is hardly relevant to the case in question.
Tarsnap: Online backups for the truly paranoid
Spamhaus is correct in saying that 90% of SMTP traffic on the net is spam. Based on my analysis we're seeing somewhere around 93%. People do not realize how much spam is blocked by relay blacklisting that never even gets to content-based filter systems. Virtually all major ISPs, including AOL, are heavily using relay blacklisting.
If Spamhaus goes down though, ten more RBLs will pop up. It's necessary to stop spam. And they're right... most mail servers on the Internet are not capable of handling the sheer amount of traffic if they were not also hanging up on bogus SMTP connections before even receiving content information. You ever wonder why your e-mail is delayed? This is because your ISP is queing mail processing because they can't handle it all at once. Without relay blacklisting, e-mail would be even slower and likely interrupted. I'm not suggesting that Spamhaus is that important, but what they do in theory, is.
All I can say is, pray that IPv6 doesn't get adopted or it will be even worse.
Why don't spamhaus just remove the e360 adresses from their regular spam lists and add them to a new list named "addresses no longer blacklisted becuase we were sued and ordered to remove them"?
:)...
That list would then serve as a perfect permanent black list for all sysadmins who happen to think that people who sue spam lists might not be the kind of people who send worthwhile emails.
I would actually recommend even higher priority to that list in the spamassassin config file than spamhaus' regular blacklists
Open Materials Database
Meanwhile the rest of the planet will treat an unenforcable court order from this judge about as seriously as they would a court order from the judge in this case.
GP was missing the link above.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
Grundgesetz * 23. Mai 1949 - 30. November 2007 - http://www.vorratsdatenspeicherung.de/
Most of the comments I've read so far seem to be in favour of Spamhaus, and while I agree that they do some good work, they are not all good. Specifically, they seem over keen to blacklist address ranges without providing any proof, and very reluctant to unblock these.
/27 subnet), and their explanation was that we were hosting someone from their ROKSO list.
I work for an ISP providing dedicated server hosting & colocation. Recently a couple of our customers contacted us saying that they had appeared on the Spamhaus blacklist, and were consequently having trouble sending e-mails. They claimed that they had not involved in any spamming activities, and that this listing was therefore incorrect. We found out that Spamhaus had blacklisted a range of our IP addresses (specifically a
While it was indeed true that we were hosting a server for this person, Spamhaus had a) blocked an address range larger than the IP addresses involved with this spammer, and b) would not offer any proof that the spammer had been using the server we host for him to involve in any spamming activities. When we contacted them, they refused to unblock this range unless we suspended the account of this spammer (again without providing any proof of activities conducted from our network that would breach our TOS), even though they acknowledged that the range they were blocking involved innocent customers. For us to suspend him at the request of Spamhaus would have been US breaking our contract with him, as there was no indication that he had violated our AUP (which DOES prohibit involvement with spam).
When we refused to break our contract with our customer at the request of a third party (perfectly acceptable position imho!), Spamhaus said that if they blocked any of our customers in future, they would blacklist our entire network (which is a considerable amount of addresses). This is unacceptable in my view, they are essentially trying to hold us to ransom without providing any proof of activities. When talking with some other ISPs, we heard of similar stories. In one case, the ISP concerned suspended the spammer's account and contacted Spamhaus to have their blacklist removed, and were told that "due to under-staffing, Spamhaus would not be able to remove the blacklist entry for a couple of days. however, if they would like to make a donation to spamhaus, they would remove the entry much sooner".
To reiterate my earlier point, Spamhaus does provide a valuable service, there's not much doubt of this. But they way in which they are organised leaves a lot to be desired!
please forward this slashdot story to 20 of your friends in order to fight spam.... actually just to be sure email it to them twice.
actually I am happy to see you, however that is in fact a banana in my pocket.
No one will be hiding behind NAT's or using dynamic IP's with IPv6. These two abuses of IPv4 addressing are the main reason why it is so difficult these days to track down and control sources of network abuse, including spam. This will make it easier to make computers and people responsible for them accountable for their actions, which means spammers and people who insist on running insecure operating systems can no longer hide or deny responsibility so easily as they can now.
Qu'on me donne six lignes écrites de la main du plus honnête homme, j'y trouverai de quoi le faire pendre.
After the failed attempt of the illegal alien crowd to shut down the USA by telling immigrants to march on one day (they don't differentiate between illegal and legal), ...
This is garbage and as such damages any argument you might try to make regarding the subject being discussed (spam). The goal of the Day Without Immigrants protest was to call attention to both the plight and the influence of immigrants. Apparently you are uptight about being part of a system that explicitly relies on undocumented immigrant labor? Perhaps a bright future awaits you in the agricultural or travel industries? There was no attempt to shut down the US, and during the protests it was common to see expressions of patriotism including displays of the flag and replicas of the Statue of Liberty.
Absolutely everyone differentiates between illegal and legal. That is the whole point. In order to become a legal immigrant there should be a process. The existing process typically takes in excess of ten years simply to review an application, never mind actually approving one and letting someone in. Many of these people who wait for ten years or typically more may do quite a bit of productive work in the interim. While the rules for entrance get endless argument Americans show they want immigrants by hiring them and endorsing the products that are associated with them by forking over money.
Perhaps you might be able to kick start your empathy if you moved away from the focus on illegality and thought more about the criteria involved. If someone is willing to work hard and has skills that are valued, does a waiting period of at least ten years make sense as an initial barrier before other barriers are introduced? Hint: There would be fewer undocumented workers if the process for documenting them functioned at all, even functioned as designed, better yet functioned by more common criteria.
Suuuure, it's worked so well to get Americans to give up their SUVs and take public transit to slow the flow of all the oil money that supports terrorists. And those bounties have helped us get Osama Bin Laden in custody. Right?
Start a happiness pandemic
Others estimate Spamhaus's blocking efficacy as closer to 75%; by this metric spam would increase four-fold, not ten-fold, if Spamhaus went unavailable
I think the math is a lot more complicated than this implies. Here's how I'd work it:
- P = % Spam (% of all sent mail)
- S(T) = Total Mail Sent
- S(S) = Spam Sent
- S(N) = Non-Spam Sent
- E(T) = Overall Filter Efficiency (% spam detected, Spamhaus + All Other Filters)
- E(S) = Spamhaus Filter Efficiency (% spam detected, Spamhaus Only)
- E(O) = Other Filter Efficiency (% spam detected, All Other Filters w/o Spamhaus)
- F(T) = Overall Type II Error Rate (% false positive, Spamhaus + All Other Filters)
- F(S) = Spamhaus Type II Error Rate (% false positive, Spamhaus Only)
- F(O) = Other Type II Error Rate (% false positive, All Other Filters w/o Spamhaus)
- R(T) = Total Mail Received
- R(S) = Spam Received
- R(N) = Non-Spam Received
We're interested in R(T) and what happens to it with and without Spamhaus. (Assuming we're still interested at all, since math sometimes does thatWith Spamhaus:
- R(T) = R(S) + R(N)
- R(T) = S(S) x [1-E(T)] + S(N) x [ 1-F(T)]
- R(T) = P x S(T) x [1-E(T)] + (1-P) x S(T) x [1-F(T)]
Without Spamhaus:- R(T) = R(S) + R(N)
- R(T) = S(S) x [1-E(O)] + S(N) x [ 1-F(O)]
- R(T) = P x S(O) x [1-E(O)] + (1-P) x S(O) x [1-F(O)]
The difference, expressed as a ratio of (Without Spamhaus - With Spamhaus)/(With Spamhaus), is[ P x S(O) x [1-E(O)] + (1-P) x S(O) x [1-F(O)] ] - [ P x S(T) x [1-E(T)] + (1-P) x S(T) x [1-F(T)] ]
Divided By
[ P x S(T) x [1-E(T)] + (1-P) x S(T) x [1-F(T)] ]
The assumptions yielding either the ten-fold or the four-fold increase seem to be that E(O)=0, and of course that false positives don't matter. Even with these assumptions, the math in the OP is a bit fuzzy to me:
- E(O) = 0
- E(T) = E(S)
- F(O) = 0
- F(T) = 0 [i.e., F(S) = 0 as well]
- [ P x S(T) + [ (1-P) x S(T) ] - [ P x S(T) x (1-E(T)) + [ (1-P) x S(T) ] ]
- Which Reduces To:
The ten-fold increase seems to be predicated upon both P=.9 and E(S)=E(T)=1. However, even if that were true, the increase would actually be nine-fold (.9/.1).yields (reducing above ratio):
Divided By
[ P x S(T) x (1-E(T)) + [ (1-P) x S(T) ] ]
P x E(T) / [ 1 - [ P x E(T) ] ]
The four-fold increase seems to be predicated upon P=.9 and E(S)=E(T)=.75. However, this would yield about a two-fold increase of
[.9 x
Factoring in false positives might actually make the Without Spamhaus scenario more dire, but clearly it would be less dire if we assume that E(O) is not zero. A better approximation would use the marginal efficiency of Spamhaus. Even with a generous assumption that Spamhaus catches an additional third of all spams sent (vs. all others without Spamhaus, and ignoring false positives), the overall increase in R(T) looks less than 50% to me (.3/.7, or approximately 43%).
It is _not_ a valid defense to say that something would break without you - while you might be right, that is the wrong argument to be pushing here.
Would slashdot give Microsoft so much slack if they were put on trial for monopolistic behaviour, and said the world's computers would become vulnerable if they were put out of business?
Real men don't write sigs
Doesn't really matter if the land in question is a foreign land, does it?
If Spamhaus goes down, then the difference will be semantic once the crapflood hits.
Spam is not a Microsoft problem, spam is a clueless user problem. It's just as easy to write a trojan spam bot that works under Linux or OS X as it is to write one that runs under Windows. All you need to do is trick someone into installing it as root/admin. Right now that's unlikely, as there are (relatively) so few Linux boxes and the maintainers and users are (relatively) so much more clued-up about this sort of thing. If the masses ever migrate away from Windows, they'll be just as clueless and likely to root themselves on their new platform.
I'm not defending MS (who have worked quite hard to make PCs easier to use, with the side effect that the more clueless user can use them) or denigrating Linux. I'm just pointing out that actually spam is a social problem; the average user doesn't know enough to keep their machines clean. A lot of users don't even care, as long as their machine works for them, they don't care who it might be working against.
Education is our only hope. Personally, I think we're doomed.
It's official. Most of you are morons.
This is nonsense. Spamhaus is a voluntary list of places you might not want to allow to deliver email to you. The people that subscribe to the list do so out of choice, they can configure their servers to block or score higher (usually) based on a listing in the Spamhaus list. Where in all of this is there place for a Judge, a court or even a whiny little Spam company ? No Judge in the world can force a delisting from Spamhaus. It's no different from me posting a list of companies that I don't like - for whatever reason - and because some people see my list and also decide they aren't going to like them either - being told I must like them. This is bollocks of the most objectionable level.
When are the courts and the politicians going to start serving the people ? Corporations are all about money and self interest - start protecting the populace not the highest bidder.
. . . by threatening judges with impending doom.
Really. It doesn't work, unless, of course, you are the President, warning judges about terrorists.
Still, I've argued this point before; there's at least a few points of dispute regarding jurisidiction, and spamhaus should have showed up in court.
It doesn't matter if they are ultimately right; what matters is that it is not 100% clear cut, and as such, a judge will give a plaintiff a great deal of leeway in a default situation.
WhiteWolf666 an exBush supporter. All you new-school,compassionate,save the children Republicans can rot in hell
I can imagine the judges reaction when he realises that he decision has just sabotaged his own personal email. and the reaction of his/her friends when they find out that he/she is to blame for all of the extra spam they are suddenly getting.
"It is a greater offense to steal men's labor, than their clothes"
Spam percentage of a 474 message inbox could only be 100%, 99.78903%, 99.57805%, 99.367089%, 99.156118% ....
Thought it would be funny, but it is not, but I am not going to waste all that typing calculation I did, so will hide behind anonymity ;-)
I pity the foo that isn't metasyntactic
I don't mod up people who say "could of".
Honestly, the "out of office" autoreply feature (most notably used in MS Outlook) could use some work. For starters, it really needs to be designed so users turning it on are immediately prompted for whether they'd like it to respond to all incoming email, or only to internal corporate mail. Quite often, I've emailed a salesperson at some company, only to get back an auto-reply that's intended only for other employees of his/her business -- not outside customers.
First, some stats on the mail server I use from a year ago yesterday and yesterday:
October 15 2005 :
Pieces of spam blocked by realtime blocklists: 9062
Top blocklists:
sbl-xbl.spamhaus.org 7193
bl.spamcop.net 1648
dnsbl.njabl.org 221
October 15 2006:
Pieces of spam blocked by realtime blocklists: 47429
Top blocklists:
sbl-xbl.spamhaus.org 40631
bl.spamcop.net 5240
dnsbl.njabl.org 1558
As spamhaus is currently rejecting 40631 emails which consequently don't have to be processed by spamassassin, it would be definitely be felt on this server were Spamhaus to become available. In fact, the reason I started using RBLs to begin with was due to one of the Spamhaus ROKSO culprits sending about 20,000 messages per hour to a dictionary list of users at a hosted domain. The server was dying then, but using OpenBSD's pf databases together with the spamhaus SBL, the problem was stopped cold.
I did like the way the poster's poor grammar is consistent in both posts though...(read the following in 'HULK SMASH' voice)
GP - "Spamhaus could of done a better job in front of jury."
PARENT - "Damn, why no mod points right now."
The best way to get enough spam to swamp almost any filter is to fwd all mail for a domain to a single inbox.
Google has reported 60K spam over the last 30 days, and about 10 messages in hour still get through to my inbox.
Worse is these asscactuses start sending mail that looks like it was from my domain, so I get all the bounces, and look like an asshole myself.
That one Russian spammer who was savagely murdered... it's hard to drum up sufficient sympathy for that.
If all the world is bending over backwards to find new ways of plugging their ears, stop yelling.
SO YOU'RE GOING TO DIE: The Comic for Dealing with Death
Then drop Hotmail; that is a no brainer.
I don't know why people don't mod up mod up posts like this one (#16452955) since it is not a ploy for being modded up.
I like the way the parent pokes fun at the GP and GGP because they are consistent in bad grammar.
GP - "Damn, why no mod points right now."
PARENT - "(read the following in 'HULK SMASH' voice)"
Jeen-yus!
Maybe you should just try and ignore it.
William of Ockham had no beard. The most likely explanation is that it was chewed off by squirrels every morning.
...that we're not the only ones. I've seen the rate of blocked spam messages on our spam firewall increase from 75% to 97% in the past few months. That means only 3% of our total message stream is allowed through as "legit" and our users are STILL seeing about 20 spam messages a day. So this, is apparently normal e-mail in this day and age? Sad.
-"...bad old ideas look confusingly fresh when they are packaged as technology" - Jaron Lanier (Digital Maoism on Edge.o
The bottom line is that the law means something or it doesn't. The decision may not have been the one most sysadmins (or even users) hoped for (and God knows it's not the one I would have wanted), but it was decided within the rules of the law and in accordance of the law as written now. I would hate to think that a judge would make a decision based on what his friends and neighbors might think. This is supposed to be a country of laws. Should it ever not be, that would be a very bad thing.
So stop the judge-bashing. Cases are not supposed to be decided on pragmatic issues when the pragma directly violates previous jurisprudence - legislation is the solution to pragmatics not matching current judicial findings. The bottom line is that Spamhaus f*cked up by not appearing in court. They should have. And, because of that, the judge rendered judgement in a proper fashion. If Spamhaus didn't understand the impact that not showing up in court would have on them (especially if they already had the wherewithal to hire a lawyer to file motions with said court), then they have no one to blame but themselves.
Spamhaus is now free to ignore the court's ruling (they are, of course, based in another country with servers in a third and can do so with relative impunity). The court is also now free to attempt to enforce its judgement in any way it sees fit within the bounds of the law. That's the way the system works. If you don't like it, change the system. Don't bitch at the actors who are merely doing their jobs (and, in fact, appearing to be doing so in an relatively competant way).
That is all.
I am serious: If any politican would seek to introduce the death penalty for spammers, he'd have my vote. I have lived with this nonsense now for ten years, and my patience is wearing thin.
I agree that spam email is about 90% of traffic. In my case the ratio is probably even higher. I get a lot of spam. Most of it gets filtered out by spamcop.
If RBLs suddenly became unavailable, the only - and I do mean only - option for me would be to reject any email that doesn't come with correct sender verification of some sort, say, SPF. Then, once spammers start using those systems too I'd have to start whitelisting senders.
I really can't believe that the US is putting up with that. I think only judges who have no email account could even agree to hear such a case.
I'd love to see all of the spam-fighting services go on strike for a week. DNS blackholes, spam filters, the works. Let spam flow uninterrupted. Let every user on the internet see just how bad spam really is. THAT would get some useful laws in place, and some criminals behind bars.
Unfortunately, too much of the IT economy is closely tied to fighting spam, and they can't afford to let that happen.
"People who do stupid things with hazardous materials often die." -- Jim Davidson on alt.folklore.urban