Slashdot Mirror

On second thought, perhaps "package" isn't the best choice of words.. but anyway -

SpamAssassin uses Perl, which adds a couple megs of overhead to the connection. Most spammers slam a server with a billion connections, so Theo's package would be more efficient - it's very small and has low overhead.

I've heard various horror stories about SPEWS though -- mostly about them being indiscriminate when blacklisting whole subnets.. so although I won't be using this tool myself, I'm sure some people will find it useful.

You can setup SpamAssassin in a site-wide configuration. You could also put it together with MimeDefang and integrate it with Sendmail.

I'm going to take advantage of a duplicate article, shamelessly grab a place near the top of the replies, and tell y'all how to fight back against spam.

1. Get a cheap discarded PC and install Linux on it. Get one of those 'always-on' net connections to your home, like DSL or a cable modem. You'll need a service plan that gives you a static IP address. Register a domain name of your very own, and use dyndns.org to point your domain name at your PC. This has the added benefit of letting you host your own web site on your own domain name if you want to.

2. Download the Exim mail server and install it on your PC, and set it up to accept email for you. You'll also want to set up an IMAP server so that you can fetch your email from the PC. Now you can make up any address you want on your new domain, and have mail sent to it reach you. This is great for when you need a one-time throwaway address for something.

3. Install SpamAssassin, and also install SA-Exim to link SpamAssassin with the Exim mail server. This will let the mail server identify and reject spam instead of only dealing with it after it's been accepted.

Once you run this for a while to make sure it's doing a good job of identifying spam, turn on Sa-Exim's teergrube ('tarpit') feature. Now, when someone tries to send you spam, your mail server will hold the spammer's connection open indefinitely by sending it occasional 'keepalive' messages without ever sending an accept or a reject. Once the spammer stumbles across enough teergrubes, the mail relay he's using will hit a process limit and be unable to continue sending spam until the spammer notices and resets it or moves on to another relay.

Teergrubing is a passive way of tying up a spammer's resources, or the resources of an open relay that's being abused by spammers. It has a negligible hit on your own resources. The more teergrubes (and honeypot web pages which feed spamtrap addresses to address harvesters) pop up out there, the harder it will be for a spammer to simply spam millions of people with the touch of a button.

If you use a Macintosh, get iCab. It prevents JavaScript from opening windows that are not requested by the user, and can filter out banner ads by size or URL.

Windows users might wish to try Opera. While not quite as flexible as iCab, stopping popups is easy with Opera.

FInally, ask your ISP to start using Spamassassin. My ISP recently began using SpamAssassin and now, I can go for days without getting any spam, and the two or three that do slip through are helpfully labeled with a (SPAM?) tag by SpamAssassin.

You'd be wasting your money, since excellent email and web filtering software are available for free.

These are the two I personally use, and they are very effective. There are many others available. There's even some that apparantly work pretty well for windows users.

I've complained about the spam situation more times than I can remember, mentioned how email addresses are being harvested off of our email servers, and described how much resource is probably being wasted processing and storing the spam, all to no avail. I mentioned anti-spam options, such as SpamAssassin, but nothing ever came of that, either. Perhaps I'm expecting too much from an IT department that's so heavily entrenched in Windows and Exchange...

We use spamassassin at work, which we've just setup recently.. We've been using pegasus email for years and years. Since the machines were ps/2s running dos 3.x. pegasus doesn't cut down on spam, but there is *no* excuse for email clients such as outlook that can infect your computer with such apparent ease. If you use a client like this, or force it on your users, you are an irresponsible net citizen.

We use spamcop.net at work. It's gets 95% of the spam. The thing which made us move on it was female employees complaining of sexually explicit spam from porn sites--with an HTML enabled mail reader, sometimes the first thing they saw was some pornographic picture.

Unless a company makes a best effort to protect people from exposure to offensive material (as defined by them, within reason), the company could be sued by the employee for creating a hostile workplace. While I haven't heard of cases of this yet, it's only a matter of time. (I hope I didn't give anyone any ideas here...)

We've been experimenting with spamassassin, and it's roughly as good as spamcop (as to how much spam gets through to the end user), but it's free. Note: spamcop and spamassassin have to completely different approaches to determining what is spam.

No need to get violent. No need to kill. The solution is simple, cheap, and pretty easy.

Just start using SpamAssassin. It's free and installs easily on modern unix systems using either sendmail or procmail. If you're stuck with Outlook on Windows, there's a company selling an installshield-based version for only $30 (considerably less that even the cheapest of murder plots). They claim to be working on support for other windows based clients, so if you're windows based and using another program, relief is probably on the way. They have a 2 week free trial version.

Spamassassin really works. They claim it filters about 95%, which should put your spam level between 12.5 to 15 messages per day.... very close to the desired goal of 10 (and nobody needs to die).

With SpamAssassin, every message gets a spam rating. Legitimate messages usually score under 3 points, and SpamAssassin's default threshold is 5.0 points. You can adjust the threshold where messages get filtered... I personally set mine to 7.0 because I'm a bit paranoid of losing any legit messages. But even 7.0 works great... most spam scores well over 10 points. If all your legit messages are scoring very low (quite likely), you might be able to safely lower the threshold a bit and get under that magical 10 per day. Personally, I find it filters nearly all spams even at 7.0.

Be sure to turn on all the "network" tests including the blacklists and razor. By default, these might be set to 0.0 points each, so they won't get used. They do take some time because they involve communication with other sites (very large ISPs with one mail server for thousands of uses don't want to spend that much time per message, but as an individual you almost certainly do). The blacklists often block legit messages, so give them low scores, but it's safe to set Razor (a database of known spam messages, with "fuzzy" matching) to a high value like 4.0 or even 5.0.

There's been a lot of hype lately about Bayesian filtering... and maybe someday lots of email clients will have it built in. And maybe large numbers of users will go to the trouble to sort their messages properly so the filters on each machine "learn". Maybe.

But right now, you can download SpamAssassin for free (or pay just a bit for a commercial much-easier-to-install-on-windows version), and instantly 95% of your incoming spam will be gone. Well, most people just have SpamAssassin modify the message and then they use their mail client or procmail to deliver the message to a "spam folder" (so you can occasionally look through it and remember the bad-old-days before you finally broke down and went through the not-really-that-difficult process of installing SpamAssassin).

It really works, it's free (or cheap), and it doesn't involve killing anyone.

this one works

personally i won't be in any more trouble than before, because i use spamassassin.

but what comes more and more to my attention:
spam is traffic, traffic costs money. who pays for all this?

http://spamassassin.org/

Really, all you need to do is manage your address properly from the beginning, don't do obvious spam-lure tactics with it, use sneakemail/other aliasing and you're set.

Ah... I remember the days when I tried to avoid spam.... Now I collect it!

SpamAssassin

Guys, in these pages I've many times read about the benefits of Spam Assassin to get rid of SPAM.

I can vouch for it working, getting rid of some 99% of the SPAM I *used* to get.

How is this any different? I understand that using S/A still means I get one or two SPAMs per day, and I know that I shouldn't "delete" them, but set them aside and check periodically in case something legit got filtered.

It's give and take, guys. Rules based web filtering works rather well. I've been using Dan's Guardian (free for noncommercial use!) and after a bit o' tweaking, it's working rather well for me.

I know, I can't look up "tits" in an online thesaurus, but it's rules+scoring method, similar to Spam Assassin, does give me > 99% just fine.

-Ben

You seem to think there's an unlimited capacity and market to send to. Wrong.

Spammers are already effectively targeting as much of the email-accessible population as they can. I've run stats from multiple, widely seperated addresses and domains, and have seen loads of largely identical patterns, trends, and mails received.

As several more critical articles have revealed (the WSJ one referenced as history in this article), spam is marginally profitable. Where it is profitable, it can be lucrative -- at least sufficiently so to leverage the ill-gotten gains to some impressive electronics and real estate. But raising costs will impact the bottom line

And that means:

• Going after the ISPs.
• Reducing TTL for a given relay.
• Utilizing SPEWS and other blacklists to put the muscle on pink-contract ISPs.
• Utilizing SpamAssassin and other adaptive filtering methods to reduce the crud flowing into mailboxes.
• Using heuristic throttles at major gateways to slow down major spews of email.
• Teergrubbing.
• Consider per-mail charges. The rate need not be high to be effective -- on the order of $0.01 / 100 mails would add $100 to a million mail spam dump, but only $1 per message for a mailing list with 10,000 subscribers. At these rates, membership dues or donations could float legitimate organizations, and legitimate commercial marketers would swallow the cost without blinking (legitimate email marketing has response rates in the 1% - 25%+ range -- thousands of times higher than spam).
• Leveraging political tactics in the effort...

Spam is economic activity. Attack it on economics. You'll see success.

Junk snailmail costs on the order of $1-$5 per item, with items such as circulars and flyers being considerably less, though there's an implied geographical targeting occuring. Yes. I've worked for outfits which considered a large campaign to be 30k pieces, and a large part of the effort was selecting the target group (blanketing the US or any other country is not an option), and measuring the results.

The result is that you receive a limited amount of such mail. Note too that payment methods (the USPS, in the US, is taking payment) means that there are audit trails available. And there are legal means, operating through the USPS, for blocking junk postal mail (including the pornography exclusion method). Very useful for, say, keeping a PO Box useful w/o requiring daily checks.

I personally use both Razor and Spamassain, and between them I get very little left over spam, and no false positives.

I am one of the developers of SpamAssassin and I'm going to agree; non-spam is far harder to collect and it is needed in just as high a quantity as spam.

The biggest problem with non-spam is that it's private and often sensitive. It would be impossible to collect a giant corpus of non-spam representative of the business world that could be used to tweak spam filters.

For SpamAssassin, we get users/developers to submit the results (tests hit) for each message when run through spamassassin, and plug spam and non-spam results into some sort of a Genetic Algorithm. This way, users only need to submit results of rules, not full messages to us for scoring. For the most recent score set, we had 169k non-spam messages and 29k spam messages. (The scores are very good!)

For testing individual rules, we have a similar mechanism in place, with a smaller volume of results.

I'd say the best testing you can do involves the user with the mail running the test, and sending you the results, rather than sending you the mail.

One problem with public corpuses is that they tend to get dated, and generally aren't representative of the messages you want to filter. Filters based on a Bayesian type mechanism will find this sort of an archive entirely useless, and there are clearly better methods for rules-based filters.

NOTE: Cloudmark is Razor which can be used standalone or as a part of Spamassassin.

Another doomsayer, give me a break, the Internet is going to fall apart in $random years, we'll be swimming in spam and popup ads, hackers will wage "cyberwar" on our "infostructure" unless we do something about it. Whatever. Use the proper tools. By now if you're still swamped in spam/popups/adware, then you're an idiot.

The moron who cut me off on the road this morning is a danger to motorists, highways are doomed to failure!

Sorry in advance if this is redundant. I know
it's a bit lame filtering spam with spamassassin
after having downloaded it, but if you don't
have any other valid option, this would be
good as well.

Verio is listed on blackholes.us, which make it us easier to set it up on
spamassassin

For instructions click here:

http://www.blackholes.us/docs/usage.html

I already use it with china.blackholes.us, nigeria.blackholes.us and korea.blackholes.us, and
I must say I'm very happy of this setup, even if
idiots like "merrynhappy" still are out from
the filters. Notice that I don't filter all the
foreign encodings, since I want to allow my .jp friends as well.

Ciao.

Install the latest Mozilla (nightly build) and spam will become a non-issue...

Accessing my mail through IMAP with evolution I'm a big fan of doing exactly what you said, basically testing for the spam header and displaying the mail in a different color or moving it to an alternate folder (I'm super paraoid about false-positives although I've never seen one with spamassasin).

1) Decentralized database servers that communicates P2P-like to track and exchange statistics about what is spam and what is not....

Like Vipul's Razor...

2) Mail Server Plug-In/Filter that uses (1) to decide whether to deliver/mark/throw out mail based on a....

Like SpamAssassin...

3) Mail Client Plug-In/Filter that receives mail from (2) according to a level of filtering you specify. Oh, and you can also vote on the mail that does get through to ID it as spam so the rest of the system gets it's statistics updated from your misfortune.

Although this takes more effort due to the need to support a number of different mail clients it appears that this may be doable on some platforms using software that supports SpamAssassin.

1) Decentralized database servers that communicates P2P-like to track and exchange statistics about what is spam and what is not....

Like Vipul's Razor...

2) Mail Server Plug-In/Filter that uses (1) to decide whether to deliver/mark/throw out mail based on a....

Like SpamAssassin...

3) Mail Client Plug-In/Filter that receives mail from (2) according to a level of filtering you specify. Oh, and you can also vote on the mail that does get through to ID it as spam so the rest of the system gets it's statistics updated from your misfortune.

Although this takes more effort due to the need to support a number of different mail clients it appears that this may be doable on some platforms using software that supports SpamAssassin.

Spam Assassin in combination with procmail has worked well on the server side for me. You can tune the sensitivity to how much spam it catches, but my informal assessment is that it catches about 95% of the spam, with only 1 false positive in about 3 weeks of use (the false positive's and any other email address can be put in a whitelist of email addresses that are let through automatically). Great stuff. Saves me from having to constantly update my ~/.procmailrc for new spammers.

SpamAssassin should soon include its own Bayesian filters, and Perl support.

SpamAssassin should soon include its own Bayesian filters, and Perl support.

SpamAssassin should soon include its own Bayesian filters, and Perl support.

Well, most of my spam is already sent to /dev/null by the SpamAssassin ninja.

But, for those that make it past the email shadow warrior, I guess Bayesian filters are a double whammy they'll never survive... Mwahahahaha!

Kudos to the Mozilla programmers!

I think you mean spamassassin rather than spamassasin. The misspelled-domain squatters are at it again.

I'll second the nomination of SpamAssassin. In the last 30 days, it tagged 427 messages to me as spam. No false positives, and probably about 30 or so false negatives (I use the standard threshold). I could probably tweak it to do even better.

SpamAssassin now includes Bayesian filter which you can add to the many other rulesets. This is in the CVS 2.50 version though.

I started blocking off all Asian Pacific networks about 6 months ago. I wrote a quick Sendmail tutorial about it right here.

How well does this work? Extremely well. I've gone from receiving 20 pieces of SPAM a day to only 1 or 2 (which Spamassassin typically catches. I realize that this method won't work for everyone, but it has worked out quite well for me.

Ms. Betterly says she refuses to send e-mails about adult fare, because it "disgraces society."

Yeah whatever - spammers claiming moral superiority over pornographers. What's next, the RIAA claiming it supports artists?

Thankfullly, Spamassassin means I don't have to deal with her garbage. Unfortunately it just hides the problem, but at least I get the satisfaction of a "fuck you" when it redirects to /dev/null.

If you've got an unfortunate friend stuck in Outlook, Cloudmark does a decent job of cleaning up the mess, and Mozilla's soon-to-be turned on anti-spam features are looking nice.

I've been running qmail forever at my place of employment, so when the bosses told me it was finally time to get an anti-Microsoft virus solution on my mail server, I dug around. Everyone seems to be using Sophos, so we went with that. Having used it for just half a month, I am really impressed with it. Easy to update. Fairly quick. I highly recommend it. However, if you do go with it I urge you to look into Sophie.

I'm also using Clam Anti-Virus as a backup. Out of the 3000+ viruses my server has caught so far, only 4 have been caught by ClamAV. Probably don't need it, but hey... anything free is worth keeping around.

I threw spamassassin in there because I was already wasting time scanning -- might as well tag spam. It helps my users filter spam, and they're happier for it. Plus, it gave me stats to throw out there -- nearly 50% of our incoming email that originates off-site email is spam. Scary.

Okay, so here's my setup:

• qmail-scanner
• Sophos (SAVI) + Sophie
• ClamAV (I need to write/find a client like Sophie for it -- it has the daemon, just no client)
• Spamassassin

Good luck.

Shayne

This seems to be about using strange approaches to spam filtering, but really...a bayesian network seems to be a natural step for a system that henceforth was composed of a series of heuristics with no knowledge of which is more important.

(Why hasn't it been done? Bayesian networks are only taught in AI and statistics classes).

What really interests me is that Spamassasin claims to use a genetic algorithm to rate how likely an e-mail is to be spam.

Bogofilter has been out since august, and does this bayesian spam-stuff in C, which probably will run a bit faster than the perl or python versions just because of it's compiled-ness. I've never run it myself, but people on debian lists say it works better or not as good as spamassassin.

Get Mailscanner and set the virus notices to off, and you'll NEVER get another klez/sircam/et al bug/notice to your inbox ever again.

Team it up with SpamAssassin and watch your spam counts plummit!

Vipul's Razor is the protocol that Cloudmark uses.

SpamAssassin is an implementation using perl that might also be useful. I believe this has more to it than just the Razor checking (it checks blacklists and headers etc.) - I have heard of some false-positives using it though, (but perhaps it just wasn't configured correctly.)

Deersoft have a Windows product based on SpamAssassin.

Until everything is right in the world of Internet email again, I will just continue to use Spam Assassin I am using it on my own mailing lists and myself here at home via procmail, and we just launched it into production at work using milter on our main external SMTP servers. The nice thing is, we don't delete the stuff, we just make it easier for our users to filter it themselves. No real legal issues that way.

This is also currently being look at being added to SpamAssassin, the idea is;

Its not required, but if you use it, you can avoid being flagged by the filter. In effect it provides a backdoor around the filter, without the potential for abuse by spammers.

I have also being trying to get Microsoft to add this to Hotmail, as a means to 'highlight' messages which have valid tokens to avoid accidental deletion. If anyone has a good contact address for them, please reach me at;
shird :at: dstc.edu.au

And to most people, it makes total sense then to ban those anonymous, crime-friendly pc's. I suppose the silver lining is, we could at least free ourselves of spammers. So it's a tough call ;-)

It isn't a tough call at all, as there are already ways of freeing yourself from SPAMMERs that don't require you to give up your basic freedoms.

Besides, do you really think Palladium is going to 'free' you from SPAM. Given the track record of Microsoft's email services (hotmail, etc) I think it is more likely you are going to be receiving SPAM adverts from Microsoft "strategic" partners, and perhaps anyone who pays the piper appropriately, and with 'trusted' computing, maybe what won't be trusted won't be the SPAM, it will be the SPAM assissin software that otherwise would have let you filter the crap out of your inbox.

One thing is absolutely certain. Whichever way that particular battle on your Palladium Trusted Computer goes, it won't be your choice. It will be Microsoft's choice.

AmaVis: Antivirus filtering daemon; packaged by most linux distros; multi-threaded (recognized multiple CPU's); sends out email alerts; very configurable; supports many antivirus scanners; works well with postfix; written in Perl; GPL

Clam Antivirus (clamav): virus scanner; written in C; fast; virus definition update tool included; uses virus definitions from the Open Antivirus project; (does not disinfect, just identifies); GPL

SpamAssassin: Perl-based Spam filter; use with Procmail; client-server architecture (one daemon); Perl Artistic License

Our application of the above software seems to work quite well. We server about a thousand users (about 100 "heavy users"), and the average server load rarely gets above 0.21 with a Dual AMD 1500+ MP that provides SMTP, IMAP, and POP all w/SSL enabled.

I use mailscanner with sendmail to scan mail for viruses . It has a number of nice features such as the ability to block certain types of attachments (e.g. exe's) - this can be configured to block/ allow any attachment based on regular expressions. It relies on third party virus engines - I use Sophos at work and f-prot on my home network, but others work too. It also integrates well with spamassassin to effectively tag spam.

If you have a mixed network with samba shares you might also like to have a look at Rainer Link's samba-vscan VFS module for samba at the openantivirus site.

I'd like to see mail browsers add a nice big "SPAM" button that will can do a number of configurable actions, and has a useful default. I suggest as the default that it forge and send back a "no such user" message, save the message in a "past spam" folder, and occasionally invokes a naive Bayesian statistical analysis program (as Graham describes) to create a filter for the future (then filter out email with a high probability of being spam). Perhaps it could optionally do other things, such as forward a copy to a list of email addresses (e.g., your local "abuse" account, the newsgroup news.admin.net-abuse.sightings, and email addresses of well-known spam killers), or calling on other spam killers to check it like SpamAssassin.

Perhaps there could be checkbox beside each action like "don't do it when you press SPAM", "do it when you press SPAM", or "confirm before doing it when you press SPAM" - that way, you could get rid of chain letters without sending them to net-abuse.

By building easily-invoked SPAM-handling capabilities right into the mail browsers, people will be able to fight back more easily.

I know the Mozilla folks are considering anti-SPAM measures; I hope they're willing to build in this kind of functionality, so that it's enabled by default.

I'd rather have a software package that has 50% filtering and 0 false positives then 100% filtering and 1 false positive. I _never_ want to miss an actual email directed at me.

I have to respectfully disagree here. First, you should NEVER trust an automated mechanism to delete e-mail before you open it (I'm not say you are, just saying it should never be done). When e-mail comes in to my inbox generally it's a user problem or network down situation.. Mozilla beeps at me, and I drop what I'm doing to see what e-mail has just arrived. If it's spam, I've wasted the effort in loosing my train of thought on whatever I was working on, plus whatever amount of time it takes me to refile it in my spam folder and adjust my filters so it doesn't happen again.

Using spamassassin, I filter all e-mails marked as spam off into a "spam" folder which I browse through about once a day at the end of the day just to be sure no legit e-mail has been filed over there. Takes only a second, and generally if the e-mail is "spammish" enough for spamassassin to file it over there it's not an important e-mail, but maybe a package ship notice from UPS, or an order update from amazon.com (though with effective whitelisting you can reduce how often this happens).

Not trying to change your opinion, just wanted to offer an alternate viewpoint. IMHO this is one of the things that makes spamassassin so good is that you can alter your threshold, so that if you can live with some false-positives but hate spam, you can use a lower threshold. If you can live with some spam and never want to miss "legitimate" e-mail, you can use a higher threshold.

Shayne

The statistical approach is not usually the first one people try when they write spam filters. Most hackers' first instinct is to try to write software that recognizes individual properties of spam.

BUT, now, the best spam filters out there already use statistical properties. Spamassassin does this, for example, and it works *extremely* well. Before I found Spamassassin, I had a huge procmial recipe that used it's scoring mechanism to do basically the same thing -- but of course spamassassin does it better, so I switched :)

Spam Assassin

Eudora Spam Filter

Mail Jail

Turn on your Sendmail antispam features!

Happy spamming, morons. I hope someone breaks your kneecaps. Repeatedly.

Yes, try Spamassassin for that.

After installing that, i've had maybe 3 spam mails come through, out of, i don't know, trillion maybe?