Slashdot Mirror
University Capitulates, Switches Off Spam Filters
Heraklit writes "As reported on German news site Heise, the system administrators of the Technical University of Braunschweig have temporarily given up the fight against spam. Because of the legal obligation to deliver all mail and of the delay time exceeding critical 5 days(!), they decided to switch off all filter mechanisms. Before, the 20 servers dedicated to processing e-mail alone had been breaking down under a load of 100000 unprocessed mail messages, ca. 98% of which had been spam or viruses. ... A similar e-mail jam occurred recently at the IT central of the German Federal Government.
Is this the beginning of the end of e-mail?" (The Fish may be useful.)
Does anybody know the filtering methods they were using before they decided to toss everything to wind?
As Seen On TV's? Come back!!!
If you don't filter out any of the spam, then some mail server somewhere is gonna have to store all that junk mail. Even with a quota system I'd expect that there'd be a whole bunch of people just "giving up" on e-mail.
what sort of awful sound the servers made as soon as the filters were turned off? ...I imagine it would be akin to someone who 'just' made it in a mad dash to the bathroom.
Perhaps just disabling spam filters and leaving virus blocks in place would be a less drastic approach. Detecting spam is non-trivial, but detecting viruses is not. They are easily found and the email should be blocked. This is implemented by my ISP (Road Runner NYC). Emails containing viruses are replaced by a text message warning that a virus was sent to the email address.
...is to inform the students how to install their own software, like Spam Assassin. That would distribute the processing to the people who actually would use it.
Read journal when you are not understand
> Is this the end of email?
:), so I think that responibility is the key to keeping email working. Adding some numbers (*sigh*) helps guard against random address guessing.
Yes. When one university decides to stop filtering SPAM the entire world's infrastructure has effectively been shut down. Oh wait... no.
My UIC account gets NO spam (because I don't give it to anyone
Anyway I don't see anyone stopping you from using your own SPAM filter. Let's not blow this out of proportion, please.
My other car is first.
Excuse me? One university gives up on spam filtering for questionable reasons and you declare death of email? Weird, I still do most of my communication via email. My servers all run spam marking services and my client filters out the junk as soon as it's retrieved.
:)
Of course more bandwidth is wasted on spam mails, but since I don't see much of it, it doesn't bother me so much.
What do you propose to use instead of email? instant messaging? Talk about waste of time
-- shortcut - the longest distance between two points.
Here at the university where I am a sysadmin, we get approx. 100K emails per day and we have no problem pushing them through spamassassin on a single server with dual 2.8 xeon processors. How in the world could this place possibly need 20 servers to process this much mail?!
Wait, don't tell me.
1: They refused to use blacklists to cut the load.
2: They refused to publish SPF records and use SPF to block all the email forged to look like it's from their domain, significantly cutting the spam load.
3: They used one of those "commercial-grade" virus/spam mail scanners that's designed to use entirely Bayesian scanning without ever setting time-outs on the generated rules, and which was written for "completeness", not speed.
4: They forgot to set up a honeypot machine to auto-block spam domains.
6: They underbudgeted for the servers to actually do the mail handling, forgetting to set up up appropriate MX records with good fallover behavior, so when any of their served domain's MX record listed machine blinked that entire domain went offline.
7: They're using MS Exchange SMTP servers, which bog down incredibly under load, especially if you run any separate service such as spam processing.
Personally, if it were my universtiry, I would prefer they started to use a RTBL. The fact of the matter is, if the likely spam isn't sorted out first, I have to try to discern the stuff entirely by hand. And although I can easily pick out Viagra ads, I have relatives and the occasional acquaintence who send mail that looks awfully like spam. Didn't want to type a subject. Used "hello" as the subject. Didn't configure their mail client properly, so their "replyto" looks crazy. Without some initialy spam filtering, I would miss at least some of these -- in fact, I'd probably miss more mail with no filtering than with a judicious blackhole in front of me.
Love or hate SPEWS and other kinder, gentler RTBLs, they're better than the present choice. It would certainly reduce the load of these email servers to where it could be more easily handled. And, if nothing else, they couldbe used to prioritize mail. Use Spam Assassin or something else to do some initial tag and filter so that mail coming from Asian IPs or originating from mail servers on cable/ADSL networks gets put into the "slow" processing queue while everything else gets sent down the faster pipe.
</spouting with little to no knowledge>
You like splinters in your crotch? -Jon Caldara
www.spamgourmet.com has always worked well for me. Give your adress to whom you want, receive just as much mail from them as you want.
No, but its one more nail in the coffin..
Something has to be done soon or email just wont be practical to have. Between Spam and viruii its overloading a lot of comanines network feed and servers..
And don't forget the cost of having to maintain antispam and antiviral solutions..
I know personally where I'm at, we are hitting over 2/3 of all email is spam/virus. ( i hear we drop 10k a day from the black hole list alone )
At home its 98%...
---- Booth was a patriot ----
I run Exim with an ACL extension called Exiscan, which runs SpamAssassin and virus checker during the SMTP dialogue.
Rejected mails thus don't generate any undeliverable bounce messages to fill up the local mail queue, and the sender gets an immediate response.
Why dont these people start using reverse DNS to MX record verification? It checks to make sure the machine sending you email has a real reverse DNS that matches their MX record. If not, it disconnects. Combine that with the real time black hole list and you'll never see spam again! This mail package does it: Icewarp
It is a common misunderstanding. While most web server these spams are pointing to may be located overseas, most of spams are originated from US. Mostly likely from hijacked fast cable/DSL connected home machines.
You may think it is okay to block email from China or even the whole Asia because you don't know some Asians in person, but please check again where your RAM, mobo, anime etc come from... A lot of companies and university have collaborations overseas as well...
We don't really have much options left... Basically, you will have to blacklist all the high boardband provider's IP range (rr, earthlink etc)... Sorry, geeks, your email server will no longer work... It is not really an ideal solution. The other idea is kind of similar to secured DNS, ie, mail server retrieves "good IPs" from a central server. Email originated elsewhere are assigned with very low priority or filtered out altogether.
Everyone needs to be registered with their mail server with the governing body (similar to the domain name idea), say for $100 per IP. It is not that expensive if you really need that... But, prohibitive for spammer... Yes, it makes home run email server more expensive... But, you cannot get a domain name for free anyway. Why should we expect email server to be free? It may be the solution to get the economy of spamming right again.
I go to Penn State, but since the university feels it has to protect dumb windows users from themselves, I cannot even send or recieve email with the subject, "Hi such-and-such"(Try explaining to a friend overseas who has almost never in her life touched a computer, in her language, why she can't send you mail with that subject) because it might contain the bagle virus. This is the same university that put in a firewall because supposedly too many people on campus had a butt-load of viruses and spyware.
Yet this same university loves to publish my email address on the web; ensuring I get tons of spam(some even in Chinese!)
I hate when the community at large has to pay for the transgressions of a few slimeballs and the idiocy of some(not even most) gullible windows users.
Some university departments run on email. If you don't deliver reliably, you could create chaos in some classes.
7: They're using MS Exchange SMTP servers, which bog down incredibly under load, especially if you run any separate service such as spam processing.
Nah, it's sendmail:$ dig -t MX tu-bs.de
[...]
tu-bs.de. 172738 IN MX 10 rzcomm5.rz.tu-bs.de.
$ telnet rzcomm5.rz.tu-bs.de smtp
Trying 134.169.9.40...
Connected to rzcomm5.rz.tu-bs.de.
Escape character is '^]'.
220 rzcomm5.rz.tu-bs.de ESMTP Sendmail 8.11.1/8.11.1; Mon, 24 May 2004 04:00:51 +0200 (METDST)
GROGGS: alive and well and living in
As someone who lives in China I get more than a little tired of being filtered out because of the continent I live in. (Especially since the vast majority of spam I get is selling products from America, regardless of what server they're sending them through.) And in this particular case, being a university it's very likely that they have a sizeable number of students from China, and many staff with academic links.
I would say this is probably not the end of email, nor is it the end of the Internet as a whole. However, it is probably the end of the protocols currently used to send and receive email.
I believe that spam is ultimately a security issue, because it slows down systems and creates problems for users and system administrators. Sometimes, security problems are caused by buffer overruns and other programming errors. However, in this case, I think the entire protocol is faulty. It may have worked wonderfully before spammers, but it's time to introduce something new that will make it extremely difficult to send spam.
I don't know exactly how the new protocol needs to look. But I have some ideas. Paying for "postage" is not one of them, as I think it is a very bad idea. Unless some payment system could be set up whereby the recipient of the mail receives the payment, not some 3rd party, like Microsoft, which would profit incredibly from garbage spam mails going all over the place. In fact, if that were the setup, then each recipient could state a price per email and/or per kilobyte of the mail message for receiving an email from a source, which the source would pay to the recipient as postage. A whitelist could be set up to allow certain senders, like one's friends, family, coworkers, etc., to send emails without paying the recipient. A blacklist could be set up to disallow all emails from specific senders and/or domains, as we have today, and if you read further in this post, you'll see my ideas for making sure that addresses are not spoofed. But I digress...
Perhaps first of all, the mail headers need to include digital signatures based on the source and destination domain names, email addresses, and other identifying information that is unique to each email sent. To avoid address spoofing, for example, people sending junk with a 'yahoo' or 'hotmail' address, when in fact it originates elsewhere, each such domain would have a private key, which upon sending, would be used in the computation. A valid signature could not be computed when the address is spoofed, and so all spammers would need to use their own valid domain name. Further, the need to make computations would make it more costly for spammers to send mail in high volumes. The algorithm should be designed so that recipients of email will have a much lower cost to verify the key. Further, the signature system could, should, and would be used to verify that each bit of the contents of the email, including all attachments, arrived correctly and without being tampered with or corrupted in transit.
the numbers dont add up, Loads of people have already raised the issue about the fact that 20 servers ( even decently mid spec single CPU machines ) will handle 100k emails an hour ( about 80 emails per min per machine is very achieveable ... ).
But there are some other issues you need to look at, with these emails not being scanned - do you know how much of storage you need to have online to have a mailstore this size and developing by the hour at 100k msgs ? not everyonce will use pop3 to get their emails, and not all the users will check email every day. Were talking about a very very large and very well setup Mail Store for this kind of volume. What about network bandwidth ?
A few basic things can reduce the work of those servers : Duplicacy level across these emails is going to be very high - all 100k emails per hour cannot be unique, there are going to be loads and loads of dupes, that dont even need to be scanned.
Creating a small database in-house with bad MailSender's list ( kind of like an in house RBL ), and flushing that list on an 6 hour interval will slow the inflow as well to quite an extent - in some tests done, i have seen it go down by almost 15 - 18% when there is a heavy load. Since most 'real' mailservers tend to retry, even if a genuine mailserver is blacklisted for 6 hours - it wont make much of a difference, however most 'hijacked PC's sending spam' dont have any retry or resending mechanism - and will just not be able to send into your server.
Another issue that helps stem the tide of bad email is to check for Virus infections before checking for spam. A lot of cases the tides of mail coming in can be virus infections ( which are easier and faster to check against - compared to rules + logic based spam checkers ).
However, all this is said and done without knowing of what system and what kind of a setup they use, there is no way anyone can really know what happened and why.
In the end, classic case for Linux and Unix based technologies to come into the frame I think.
Call your elected representatives! Get them to outlaw spam!
Napster-to-go says "Fill and refill your compatible MP3 player", which is a lie. It's not MP3. It's WMA with DRM.
I would agree, but only on a few stipulations. E-mail as we know it will almost certainly die sooner or later, to be replaced with something else that better fits our future needs. Like gopher and http, smtp, pop, and imap will all sooner or later be replaced by another set of protocols. Perhaps they will require something like SPF to reduce spoofed "From" headers. Perhaps they will support or even require encryption? Face it. Sooner or later, e-mail as we know it will die, but only when something else is able to take its place.
Slackware, what else when it must be secure, stable, and easy?
Joking aside, it boils down to economics. Spam is profitable. If something is profitable, people will do it. Selling drugs is profitable, and the war on some drugs hasn't changed that. The answer to spam (and drugs) is not to try and stop them, but to make doing them unprofitable.
What makes spam profitable is the presence of people on the internet who are SO incredibly stupid that they fall for it. (See Junkie loves his spam) Remove them, and you shoot spam through it's purtid heart. I can think of several methods of doing do:
- If you respond to spam, you've probably got shitloads of viruses on your computer. Beyond any shadow of a doubt some of them are spamming people. If you ISP detects lots of mystery traffic from you on known virus ports, you're given one warning. Then you get kicked off without ceremony and not allowed to reconnect until you can prove to them that all computers using your connection are malware-free. No more malware, no more spam zombies.
- (You, the ISP) Send test-spams. Specify in the header that it is NOT a real spam so you don't get blacklisted. Anyone who responds to them loses all services except port 80 until you prove to a professional who visits your house that you know enough not to buy from spam. Do it again and you will never be allowed to use your ISP's mail servers again.
Neither of these can possibly be routed around or hacked by spammers, because they are not involved in any part of the process. If you are not in the habit of perpetuating malware with your computer, you needn't worry of getting caught up in it all. Neither of these requires a major invasion of your privacyspamd is a new approach to blocking spam. Its called greylisting. It rejects all email with a temporary failure notice in the hopes that the large volume spam senders don't have the resources to wait 30mins and send the same email again. Apperently this method works quite well and uses little resources.
The MTA's work is relatively light compared to what anti-spam software must do. This is especially true of SpamAssassin. While it does have some advantages over its competition, SpamAssassin is extremely resource intensive. Firstly, SpamAssassin is not written in fast C/C++ but Perl. Every email is sent through zillions of Perl regex rules. Then there is the Perl implementation of the Bayesian test, which really bogs down when an email auto-learned. Then there are the various (optional) network lookup tests: several RBLs, Pyzor/Razor/DCC ... each email can eat up a lot of resources even if you bypass the startup overhead by running spamd.
I have also seen situations where SpamAssassin was not correctly respecting the maximum child spawn limit. Since spamd is a fairly heavyweight process, the server started swapping and throughput plunged.
Such heavy overhead is not a essential part of anti-spam software. Something NOT written in Perl nor any "interpreted" language, something with a smaller footprint, will be much, much faster. I wonder how many people have switched to dspam for this reason?
Does Germany have a law that I'm not familiar with? Email is free not a paid service, why is there some obligation to deliver? Snail mail is normally Govt. run and delivery is what you pay for with a stamp.
No one has to or could guarantee anything for email. With the amount flowing because of SPAM the dropped packets must be astronomical.
Professional Politicians are not the solution, they ARE the problem.
Being the person that blocks spam is a lose/lose situation. They don't understand how bad the problem is when you do your job right. They complain when spam gets through and complain when legit email gets blocked, but don't want you wasting all your time on it.
I predict that this school will be forced back into filtering spam by their students (customers).
[rant]See, 3 years ago, as spam was beginning to get bad, I began filtering spam on the email system I manage. Over 2.5 years, I developed a rather intensive filter, but since I knew I was not perfect, I had to scan blocked email for false positives. It got to the point I was spending 25% of my time scanning for false positives and the boss didn't like that. He also didn't want me to spend time trying to figure out how to set up Spam Assassin. (I'm not a Linux guru. Sorry!) The board didn't want to spend the money on a purchased system and didn't want me wasting my time with spam. They didn't think it was a problem so they told me to just stop blocking spam. My boss told them that spam was a BIG problem, but they never saw it so they didn't believe him. I asked my boss 10X "Are you sure you want me to stop blocking spam? They won't like the results." He confirmed. I stopped blocking spam and about 50,000 additional spams per week came flooding into the system. The 50,000 were what was being blocked previously. I was flooded with phone calls until everyone realized what was happening. Then, just 2 weeks ago, I was instructed by the board to go back to my filtering, but only spend 30 minutes a day on it. RIIIIGGGHHHHHTTTTT! Ever try scanning for legit email among the trash, adjusting filters to make it better and taking calls and emails from people that want you to be sure an email is blocked and only spend 30 minutes a day on it? I managed to put together a Spam Assassin box and it blocks 10,000 per week, but there's a lot that doesn't get blocked. I don't know enough about it to make it better.[/rant]
But why is the rum gone?
Your faith is touching. Was it Nixon who started the first "war on drugs"? How's that going?
The measured UBEs over a 3 moth period were 172,887 - only for their top-25 most spammed employees!
Notepad specialist & FAT administrator, group training available
How can you know you've had no false positives.
Have you personally reviewed the 2.9M messages which were filtered out... if you have then i'd question the value of your filtering.
I know i've occasionally had false positives and i get nowhere near your message volume. My personal favorite is the UK paypal-esque service NoChex which sends emails with the subject line "YOU'VE GOT CASH!!"...
I've seen this happen in my local University too.
Take a university that has thousands of people actively using email, and thousands of computers, probably a hundred of which function as mail server. Now, decide that "we need a central mail server to filter viruses and spam". Take a few useless machines lying in the computer center, and make them the mail server that's supposed to replace the hundred you had previously. Then slow down the new mail server by applying every concievable virus and spam filtering.
What do you get? Incredibly slow service (sometimes mails get stuck for hours or more in the queue), single point of failure, and officially-mandated false positives (noone in the university can avoid them). AND, you still get a lot of spam.
Computer centers must know that if they want to centralize a service that was previously decentralized (different departments and individual running their own mail servers and filters), they must be prepared. Prepared to handle the load (Google had to buy 100,000 machines to handle their load!), prepared to handle the humans who use their service, and prepared to handle exceptions (a person or department that doesn't want the centralized filtering). Often, these computer centers don't think of these issues in advance, causing things like described in this article.
Spamd and other means for "tarpitting" the calling SMTP are another great tool to be used in combination with RBLs and bayesian filters.
It's a strategy in layers:
To quote the post directly above you ...
No, sendmail (Score:5, Informative)
by marnanel (98063) on Monday May 24, @12:04PM (#9234290)
(http://marnanel.org/)
7: They're using MS Exchange SMTP servers, which bog down incredibly under load, especially if you run any separate service such as spam processing.
Nah, it's sendmail:
$ dig -t MX tu-bs.de
[...]
tu-bs.de. 172738 IN MX 10 rzcomm5.rz.tu-bs.de.
$ telnet rzcomm5.rz.tu-bs.de smtp
Trying 134.169.9.40...
Connected to rzcomm5.rz.tu-bs.de.
Escape character is '^]'.
220 rzcomm5.rz.tu-bs.de ESMTP Sendmail 8.11.1/8.11.1; Mon, 24 May 2004 04:00:51 +0200 (METDST)
Because once a solution becomes commercial, the spammers get hold of it and work out how to modify their spam so that it gets through.