Slashdot Mirror
University Capitulates, Switches Off Spam Filters
Heraklit writes "As reported on German news site Heise, the system administrators of the Technical University of Braunschweig have temporarily given up the fight against spam. Because of the legal obligation to deliver all mail and of the delay time exceeding critical 5 days(!), they decided to switch off all filter mechanisms. Before, the 20 servers dedicated to processing e-mail alone had been breaking down under a load of 100000 unprocessed mail messages, ca. 98% of which had been spam or viruses. ... A similar e-mail jam occurred recently at the IT central of the German Federal Government.
Is this the beginning of the end of e-mail?" (The Fish may be useful.)
Site's a little slow -
Akamai Mirror.
Spam wave?rrollt DOES Braunschweig
The system administrators at the computing centre of the technical Universit?Braunschweig kapituliert on Friday of yesterday before the effects of an unknown Spam load and the Spam and virus filters DO deactivated. After "quite controversially gef?ten service discussion" the responsible persons decided that their setting obligation, all enamels within f? To deliver days to the Empf?er, priority before the Sch?ingsschutz genius?. For the description of the situation the Admins submitted the following numbers:
"our server park of approx.. 20 systems alone for enamels is overloaded. A "basic load" of zun?st 30,000 rough enamels (Di) on approximately 100,000 enamels expanded (Fr). [... ] Per hour up to 10.000 enamels by the Spam and virus scanners are worked on and set. 98% of it are "unerw?chte" Mails (Spam, viruses) for which we nevertheless a legal setting obligation have."
W?end Mails within the Braunschweiger of computer network to be still normally set, m?en themselves external Mail Empfanger with the advice to manage, them should e-Mail-Anh?e after M?ichkeit only after R?sprache with the sender?nen and otherwise a local virus scanner use and this at least once t?ich update.
The University of Braunschweig is not threatened as only ones of the Spam oversupply: Already since past week the computing centre of the free Universit?Berlin blocks perforce all Mails with potenziell gef?lichen Attachments, and as reported the E-Mail supply of the Federal Government in the digital M sags?.
It d?te indisputablely its that the Admins of the?rfluteten computing centres does not act differently k?en, but a deichbruch as in the Braunschweiger IT landscape tr? surely not to the Abschwellen of the Spam wave, rather to their further growth with (hps/c't)
Wait, don't tell me.
1: They refused to use blacklists to cut the load.
2: They refused to publish SPF records and use SPF to block all the email forged to look like it's from their domain, significantly cutting the spam load.
3: They used one of those "commercial-grade" virus/spam mail scanners that's designed to use entirely Bayesian scanning without ever setting time-outs on the generated rules, and which was written for "completeness", not speed.
4: They forgot to set up a honeypot machine to auto-block spam domains.
6: They underbudgeted for the servers to actually do the mail handling, forgetting to set up up appropriate MX records with good fallover behavior, so when any of their served domain's MX record listed machine blinked that entire domain went offline.
7: They're using MS Exchange SMTP servers, which bog down incredibly under load, especially if you run any separate service such as spam processing.
translation to en
www.spamgourmet.com has always worked well for me. Give your adress to whom you want, receive just as much mail from them as you want.
Having gone through German, I find that WorldLingo.com returns a much more accurate translation than Altavista.
"All you have to do is be fragile and grateful. So stay the underdog." Chuck Palahniuk, Choke
Yeah, who needs those stupid spam countries?
You, for example, if you live in the US or Canada, or Europe... or.... you get the picture.
Certainly, nobody likes the current situation, but suggesting that we send spammers (or people whos boxes have been hijacked by spammers) to prison camps without charge or bomb their countries (How'd you fix the economy? Bomb it?) is clearly stupid.
Why dont these people start using reverse DNS to MX record verification? It checks to make sure the machine sending you email has a real reverse DNS that matches their MX record. If not, it disconnects. Combine that with the real time black hole list and you'll never see spam again! This mail package does it: Icewarp
Some university departments run on email. If you don't deliver reliably, you could create chaos in some classes.
7: They're using MS Exchange SMTP servers, which bog down incredibly under load, especially if you run any separate service such as spam processing.
Nah, it's sendmail:$ dig -t MX tu-bs.de
[...]
tu-bs.de. 172738 IN MX 10 rzcomm5.rz.tu-bs.de.
$ telnet rzcomm5.rz.tu-bs.de smtp
Trying 134.169.9.40...
Connected to rzcomm5.rz.tu-bs.de.
Escape character is '^]'.
220 rzcomm5.rz.tu-bs.de ESMTP Sendmail 8.11.1/8.11.1; Mon, 24 May 2004 04:00:51 +0200 (METDST)
GROGGS: alive and well and living in
It seems that they've decided to provision potentially 50x their existing disc space for email (as 98% of the email is currently spam, which is presumably filtered out at the moment), instead of deploying additional resources for filtering before it gets to the users.
Good luck with that approach! If their primary constraint is budgetary, as it would seem, it would make more sense to invest *more* in filtering so that the crap didn't get to users' mailboxes where it will doubtless stay indefinitely in some cases.
Note: I'm assuming that, because they have some apparent requirement that all mail gets delivered, that they cannot effectively enforce email quotas that would result in non-delivery of email.
Permanant Failure (5xx SMTP) codes are not safe either.
There are many cases where email is relayed before being sent to a system that does virus scanning. (Consider what happens when you use sendmail aliases and virtual domain entries that contain somthing on the order of "user: user@someotherhost.com".)
Your SMTP 5xx error will cause the relaying server to generate a bounce. The bounce will go to the person listed by the forged "To" headers, and will even include a copy of the Virus.
The proper way to deal with email worms is to quietly delete them.
I seem to recall the whitepaper about this method being posted on slashdot a while back. My free email provider (softhome.net) implements this and it works ok but it still lets quite a bit through. It seemed like a great idea in theory though.
"It is not how things are in the world that is mystical, but that it exists." -Ludwig Wittgenstein
I use Cashette for my email server. It's free, allows POP access, gives you the ability to activate its systems on other email accounts, and it works by using an authorization system. Basically, if someone isn't on your "authorized" list, then their mail gets put into a special folder. You can either review what's in that folder, or just forget about it. Here's the nifty part... If a spammer REALLY wants to get their message to you, they can pay you for delivery. You set the price, up to $300 for them to get their message to you.
You can get your own account at http://www.cashette.com/
Have fun!
-Phyre
"When the people fear their government, there is tyranny; when the government fears the people, there is liberty." -Thom
Thats why you use the spamd for Spam Assassain.. significantly alieviates the problems associated with running a perl interpreter for each email being processed
Filter on custom header Content-Type. multipart/mixed implies a message with attachments
Simple problems have simple solutions.
You can increase the threshhold at which you declare spam to be spam. Allows for more misses, but reduces the false positives to, essentially, nothing.
Or, you can just tag likely spam with ***SPAM*** in the subject and let the user deal with it.
Or even better, you can direct likely spam into a specific IMAP folder on the server that the user's client can subscribe to and they can glance at their personal SPAM folder on the server whenever they want without having to download all the bodies.
As someone who personally uses postfix+procmail+spamassassin+razor and recieves 4,000 emails per day, I am currently filtering out 98% of the spam on the server and have had ZERO false positives in two years and 2.9million messages.
Statistically, you will eventually get some false positives - especially if you have a large userbase (as opposed to just one or two accounts). But if one out of every few million messages isn't acceptable, you can just use one of the previously suggested methods.
The worst you can do is nothing at all.
No, Sendmail:
220 rzcomm5.rz.tu-bs.de ESMTP Sendmail 8.11.1/8.11.1; Mon, 24 May 2004 06:46:39 +0200 (METDST)
base64 bloats 1/3, not 1/2.. i agree it's not great though. (Makes me wonder why newsgroups are so popular for leet file sharing.)
This isn't even the beginning of the end of email. It's simply becomming less and less workable to run a single mail server system with a large amount of users. Small time mail servers aren't targeted by spammers. Universities are heavily targeted because there are lots of users all going to a common domain.
It's the same reason users of major ISPs are more likely to be probed for vulerabilities.
I've found the method of filtering based on the "Click-Me" domains to be the most effective with virtually no false positives (zero is a realistic number).
I've found that setting up a secure public mail system is cake. Mercury Mail is free and handles well. A single check box set by default is all it takes to keep it from being an open relay. Students of the university could probably do rather well offering their own e-mail services to students. Mercury Mail's filtering system is quite robust.
MM supports IMAP/POP3/SMTP and alternate ports as well as SSL on all them. Adding a web-based front end also isn't that difficult if you know what you're doing. There's actually one built in and a more robust version coming.
I already have a few hundred users on Indie-Mail and the amount of bandwidth used per day is pretty negligable.
Ben
Work Safe Porn
Make your boss happy, and block on these three DNS based lists: dsbl.org, spamhaus.org, dnsbl.org. Everything coming from IP addresses in these range is basically garantueed not to contain false positives. It'll clear your inbox quite effectively. (I'm one of the volunteers helping out dsbl.org, so feel free to mail me with questions)
-John
The measured UBEs over a 3 moth period were 172,887 - only for their top-25 most spammed employees!
Notepad specialist & FAT administrator, group training available
Two things:
The approach that we take is the following: We mark virus messages with a special header and deliver them in a dedicated folder in the user's mailbox. Most users simply delete all messages in this folder, but then it is their choice, we abide to all laws and do not generate bounce messages.
Sebastian
They're probably thinking of StGB 303a, Datenveränderung, which prohibits unlawful deletion, suppression, disabling and alteration of data. Mail can always be lost, but spam and worm filtering is a deliberate act, so this law probably applies.
All it takes to avoid this is to have every user sign an agreement that the admins are allowed to apply automatic filtering which deletes mail without notice and, despite diligent configuration, can result in false positives.
A very useful and free mail email scanning tool that is fast & GPL. (Please visit the sourceforge link if at all possible).
Mailscaner at Sourceforge
Mailscanner website
I run my own webservers, with mail service etc. /dev/null as most spam is sent to madeupname@domain.com
1 good thing was to make sure every user has a defined email address, or alias to their username. That means I can send a good 85% of mail straight to
The rest of the spam is due to people leaving their addresses in plain site (on web pages etc) and not having virus free computers.
I also run MailScanner to remove viruses, before the user can get to them, but I don't use spam assassin, because thats not my problem. The users are, to a large extent, to blame for the amount of spam going through the server, (see above) and I don't see why I should deprive them of their ill-gotten gains !
My spam count in my inbox is virtually zero, the few I do get are forwarded from other servers, but are trivial to delete.
If only people would use personal certificates to identify themselves, then spam filtering would be so much easier.
Strip all attachments.
All of them. Don't process them, just ban them.
If you want to send a file, use ftp or send a link to a read-only http or smb/nfs share.
Love to... but not gonna happen with our users.
We settled on blocking all executable attachments (VBS, EXE, SCR, etc.).
You know, the extensions that 99.999% of users have no business reason to be sending to each other, but which are used by the viruses/worms to spread. Blocking those put a good damper on the amount of virus/worm mails that were getting through and was cheap CPU-wise.
Wolde you bothe eate your cake, and have your cake?
I don't know what kind of machine you are running but we have SA running on it's own machine, serving two mail servers. It handles over 300K messages a day with network tests enabled, and the standard scan time is sub 1 second.
If you are going to be running SA with any kind of volume you need to keep in mind...
1 - Run a local DNS caching server. dnscache works well, give it lot's of memory to play with
2 - Rsync and run as many of the RBL's locally as you can.
3 - Set the max number of children that SA is allowed to spawn, on our hardware that number is about 12.
4 - Lot's of memory! Depending on the number of max children, you might want 1 gig or even 2 gigs of memory
5 - Off load SA on to it's own dedicated machine, so if need be you can easily inject another server using hardware or dns round robin load sharing.
I don't know what kind of volume the Uni was handling but with 20 machines I think I could easily handle upwards of 20 million deliveries per day.