Slashdot Mirror
Revolutionary Spam Firewall Developed
psy writes "physorg has a story on a new spam firewall developed at The University of Queensland.
The new technology is the only true spam firewall in existence, according to co-developer Matthew Sullivan.
"Existing anti-spam software filters out spam whereas ours puts up a firewall, stopping all email traffic and only allowing real mail through," said Mr Sullivan.
"In addition, our technology is accurate and fast. We recently completed a successful trial of a key layer of the spam firewall and it processed the emails at 90 messages per second, misclassifying only one out of 25,000 emails."
"It turned out that the software was even better than us, picking up spam we'd incorrectly classified as legitimate emails."
I have a simple algorithm to reject spam: spelling.
If you can't spell correctly, then I don't want your v1agr4.
Life is the leading cause of death in America.
I think Barracuda Networks would rather disagree with the idea that this is the "only true spam firewall in existence," considering that Barracuda's entire product line consists of spam firewalls.
Damn fine spam firewalls, too, I might add. They handle around 115 messages per second, and can run up to eight filtering steps (including Bayesian analysis, which is similarly efficient to SVM, which the one in the article uses). Plus Barracuda's can do virus scanning.
I'm not sure how this is revolutionary.
This isn't a firewall as it doesn't filter based on addressing. Furthermore, the use of SVMs (support vector machines) to classify text is not new...
It's easy to produce these kind of results in trials - you just tune the spam filter to handle a certain set of emails, then you feed it those emails again and you get a near 100% success rate.
Heck, why not do it with a million emails? Makes better headlines that way.
I don't see how this is any different to SpamAssassin (the term 'Mail Firewall' is pure marketing bullshit. It's a spam filter. Get over it.) except I bet it costs a hell of a lot more...
1 out of 25k is impressive, but what happens to these spam mails? Are they bounced back as an error "no user account found"? Or done like a blackhole where the spammer doesnt know if it reeached its intended recipiant? I like my SpamBayes :)
I know! Ciphertrust's Ironmail works the same way... It stops ALL mail inbound, runs it through about a dozen different detection queues, only letting legitimate stuff through. I'd really like to see how this new one is otherwise unique.
Ed R.Zahurak
You know, oblivion keeps looking better every day.
Although this is a great new technology, for a business setting, I don't know if even missing one e-mail is acceptable...
That's what everybody says but what's the other option? Letting all the SPAM come in? Do you really think that fed-up employee who gets hundreds of SPAMs a day is really going to do a better job of just mashing down the delete key then a SPAM filter with a 1/25000 error rate?
Of course I doubt this technology would perform that well but the point still stands -- if you don't have a computer flagging them then chances are you have a human flagging them. Who do you trust more?
I want peace on earth and goodwill toward man.
We are the United States Government! We don't do that sort of thing.
Most users of email are now treating it as a lossy messaging system, and the users themselves accept that some messages simply don't make it. Critical business is always followed up with a call.
-Adam
If you are sending something so critical then you shouldn't be using email. FedEx with signature required delivery and certified/return-receipt USPS mail exist for a reason.
"...companies losing valuable employee time to deleting spam..."
/.ed, here:3
Maybe they should be working on a Slashdot-Firewall. Damn, I really should get back to work.
Oh, and since the linked article got
http://www.uq.edu.au/news/index.phtml?article=583
Unconsciously Desired Email Industry (Our slogan: You opted in in your heart!), I'd like to strongly protest the continuing escalation of technology against us. We provide the opportunity for hundreds of thousands of people to spend freely on products unburdened by simple heuristics of "they work" or "they won't make you ill" or "we'll actually send them". Why are you so intent on interfering with the consumer ethos?
One of two conditions exists in this case.
1) The e-mail is vitally important and your business will be seriously damaged by its failed delivery.
2) The e-mail was somewhat important, but not something large enough to materially change your revenue/profits.
If the first is the case, you probably shouldn't be using e-mail in the first place and/or whoever sent it is probably going to follow up with a FedEx or phone call.
In the case of number 2 (ha ha, number two), you've saved so much time not having to wade through spam that the losses are negated.
The idea is that the mail server keeps a whitelist of "allowed" addresses which are always accepted. If a mail comes from an address which is not known, the mail server will reply with a "server unavailable, try later" error message. All real mail servers will try to send the message a little later (I don't know the exact time, but it's probably less than an hour. Someone else might know better).
The second time the remote mail server tries to connect, the server accepts the mail and adds the address to the whitelist.
However, mass mailers for spam don't do this but simply go on to the next address in the list if this happens. This way the spam message is filtered out.
Note that this method doesn't require any analysis of the actual content of the messgae, nor does it involve any manual actions from neither the sender nor the receiever. Currently it's porbably the best spam blocking method that exists.
amidoacetic platymyoid granomerite nonacceptant dorsoposteriad uninclined unshocked zibet intercity lornness
1/25000 is significantly better than a human being. If you use no automatic spam filtering at all, and you get a typical geek's email load (about 100 spam a day with 10 legitimate emails a day), you will still delete mail as spam when it wasn't spam.
That's why I use SpamAssassin - it does a good job, and is no worse at making false positives than I am. If I'm just as liable to make a false positive than an automatic filter, I'm better off saving my time.
Oolite: Elite-like game. For Mac, Linux and Windows
For example, Mail Avenger allows you to filter spam based on network characteristics like SYN fingerprints and routes. It even integrates with the kernel firewall to filter out aggressive spammers and mail bombers. However, because it runs as an ordinary user-level process, it also has much more flexibility, for example allowing individual users to set different policies on different email addresses. What can a spam "firewall" do that you can't do with a system like Mail Avenger.
(Presuming that wasn't a troll) That's a horrible, horrible solution. Viruses fake sender addresses, which means the faked address gets *loads* of these 'Please confirm' emails, clogging up another innocent mail server. Get it wrong, and you'll have two servers sending 'Please confirm' messages to each other until one screws up into a little ball and dies. I'm all for the War Against Spam, but this isn't the way - it just doubles the amount of emails.
This didn't make it through my bullshit filter. Oh - sorry, I mean bullshit firewall. It's like this new technology that rejects bullshit from the evil internet, so I never have to read it. Thank god, because if I'd read about this "revolutionary spam firewall" I would be forced to make a childish comment on slashdot and burn some karma.
Then you're stuffed anyway, because internet e-mail is not guaranteed.
It is difficult. We're swatting away a million of the damn things a week and still our users complain. They also complain when we get false positives. And when, next week, we turn on the system that lets them see what we have blocked that was addressed to them, they'll complain too.
I think the one solution they would find acceptable is for me to personally read every one of those million messages and mark it as good or bad. I hope our VP doens't read slashdot....
~~~~~ BigLig2? You mean there's another one of me?