Revolutionary Spam Firewall Developed

psy writes "physorg has a story on a new spam firewall developed at The University of Queensland. The new technology is the only true spam firewall in existence, according to co-developer Matthew Sullivan. "Existing anti-spam software filters out spam whereas ours puts up a firewall, stopping all email traffic and only allowing real mail through," said Mr Sullivan. "In addition, our technology is accurate and fast. We recently completed a successful trial of a key layer of the spam firewall and it processed the emails at 90 messages per second, misclassifying only one out of 25,000 emails." "It turned out that the software was even better than us, picking up spam we'd incorrectly classified as legitimate emails."

Comments

Not the first; not revolutionary

[Anonymous Coward]

I think Barracuda Networks would rather disagree with the idea that this is the "only true spam firewall in existence," considering that Barracuda's entire product line consists of spam firewalls.

Damn fine spam firewalls, too, I might add. They handle around 115 messages per second, and can run up to eight filtering steps (including Bayesian analysis, which is similarly efficient to SVM, which the one in the article uses). Plus Barracuda's can do virus scanning.

I'm not sure how this is revolutionary.

Re:Not the first; not revolutionary

[Greyfox]

I believe the distinction is when the filtering takes place. If you wait for the spam to be placed on your hard drive and filter it out when you start your mail client, then it's filtering. If you reject the spam before the remote MTA drops the connection, then it's a firewall.

I'm using Postfix at home and it's got some nifty features to allow you to do this sort of thing. You can write a simple SMTP server that listens on some port of 127.0.0.1 and configure postfix to send the mail though that. Your server scans the E-Mail and sends a reject or accept message back to postfix, which sends it on to the remote MTA. Your SMTP server then feeds the mail into another postfix server which listens on an odd port of 127.0.0.1 and doesn't have the restrictions that your publically accessable postix server does. There are packages available for all sorts of scanning based on this ability. Since you reject the message at MTA time, you don't have to bother with sending a bounce message, either.

Re:Not the first; not revolutionary

[naelurec]

I do multi-layered protection. At the MTA level, I utilize some DNSRBL lists to block from known spam servers. In addition, I require HELO and reject people who are claiming to be my server. In addition, I will reject invalid recipient domains, etc.

From here I run accepted emails through AMaViS / SpamAssassin / ClamAV / Sophos Sweep (I have yet had Sophos catch a virus that ClamAV did not detect.. though ClamAV caught two that Sophos did not..) and will not deliver (but notify postmaster) of spams over a set value (ie 8), deliver spam between 5-8 tagged and items under a certain value get passed without tagging. Viruses are always blocked and reported.

Overall this has reduced unwanted email significantly. On networks of 40-60 users, between 35-50% of email is rejected at the SMTP level, about another 10% or so is quarantined (either viruses/spam), another 10% or so is tagged but delivered and the rest is legit.

I have yet had any compliants of false positives (granted there is a risk that they do not know) but have had a lot of priase for reduction in spam levels. I am not aware of any viruses penetrating.

Check out http://jimsun.linxnet.com/misc/postfix-anti-UCE.tx t for more info (this is postfix centric, but the ideas could be applied to other setups)

Not a firewall

[BarryNorton]

This isn't a firewall as it doesn't filter based on addressing. Furthermore, the use of SVMs (support vector machines) to classify text is not new...

Ciphertrust, too...

[TrebleJunkie]

I know! Ciphertrust's Ironmail works the same way... It stops ALL mail inbound, runs it through about a dozen different detection queues, only letting legitimate stuff through. I'd really like to see how this new one is otherwise unique.

Re:Spelling

[random_culchie]

Yes and aparently there are 600,426,974,379,824,381,951 different ways to spell viagra!

Will your algorithm do it with polynomial complexity ;)

Re:Here's how it probably works

[Santana]

That's how spamd works, and yes, it works tremendously well. I used to get 300 spam messages daily. I receive now one or two every week.

Re:Spelling

[CommanderData]

His algorithm doesn't need to. All it needs to do is check against an existing dictionary of words. If the word is not on the list, it is assumed to be misspelled. (If the good spelling of Viagra is in the dictionary, simply remove it so that any correctly spelled reference to Viagra counts as a misspelling too). If there are greater than X% misspellings in the e-mail it gets trashed. X can be a smaller percentage if the e-mail has any hyperlinks in it, because it is virtually guaranteed that someone is trying to sell you something...

Re:Here's how it probably works

[hedronist]

I think you're trying to describe greylisting. Although greylisting is amazingly effective, I don't believe that's what is being discussed here (the site is slashdotted).

Our experience with greylisting has been (1) an 90%+ reduction in passed-through email (with no complaints from users about lost mail (yet)), (2) a dramatic decrease in server load because SpamAssassin doesn't see the message until after it gets past greylisting, and (3) people rediscover how useful email is once you get all of the crap out of their inbox.

Marketing Guy: What's the worst that could happen?
Dilbert: Our beta product could turn into an evil robot that annihilates the galaxy.