SpamAssassin 3.0 Released

SURBL by alatesystems · 2004-09-22 01:29 · Score: 5, Informative

For those not in the know, SURBL is really cool. It actually lets you scan the message(well, SA does that) and then look for urls that it links to. It compares this to a realtime BL of other people getting spam like you and if it is a known spam TARGET url then it blocks the message based on that.

It makes it really hard for them unless they just register countless domains.

Excellent technology, and I will be upgrading to the newest stable.

Chris

Re:SURBL by virtualone · 2004-09-22 01:32 · Score: 2, Interesting

this sounds nice, but what if the url is put together with javascript?

--
Only morons moderate based on a sig.
Re:SURBL by Mr+Guy · 2004-09-22 01:34 · Score: 3, Insightful

I would imagine the javascript would flag it highly suspicious on that basis alone.

--
Never confuse volume with power.
Re:SURBL by PhotoBoy · 2004-09-22 01:42 · Score: 2, Insightful

Can you run javascript inside emails? I thought even Outlook and Outlook Express would be blocking that by now?
Re:SURBL by thing12 · 2004-09-22 01:47 · Score: 3, Insightful

I'm not aware of any (modern) mail clients that execute javascript, so what would be the point of sending an email with js links?
Re:SURBL by slaad · 2004-09-22 01:47 · Score: 3, Funny

Ahh, the same principal as MT Blacklist. Good stuff, works half way descent and you're pretty much sure to not get a false possitive, unless...

From: Mom
Hi, honey, I just wanted to see how you were doing. We're doing great around here. Write back soon. I love you.

Do you yahoo? Enlarge your penis [penis-enlarge.com] today!

You never know I guess...

--

~Warning!~ The above is encrypted using rot676!
Re:SURBL by hey · 2004-09-22 02:02 · Score: 4, Interesting

I suppose this will driver spam-advertizers to obviscate their URLs in the spam mails. Eg use javaScript to build the URL so the real URL can't be detected -- like we do with our mail addresses on webpages so they won't be harvested by spammers!
Re:SURBL by BlowChunx · 2004-09-22 02:13 · Score: 3, Insightful

Probably some argument along the lines that made e-mail clients render HTML...

<rant>
You know, if we just dropped the whole notion of using HTML in e-mails, and only allowed plain text most of this would never have happened in the first place.
</rant>
Re:SURBL by Darby · 2004-09-22 05:42 · Score: 2, Informative

I'm not aware of any (modern) mail clients that execute javascript,

Well, Kmail and thunderbird to name 2 off the top of my head.
Granted it's off by default, but the functionality is there.
Re:SURBL by ChaosDiscord · 2004-09-22 06:03 · Score: 3, Insightful

I suppose this will driver spam-advertizers to obviscate their URLs in the spam mails. Eg use javaScript to build the URL so the real URL can't be detected...

Which is fine. There are two defenses, both of which work now: 1. Javascript in a message is a big spam flag; legit mail almost never uses it. SpamAssassin and most other hybrid systems assign mail a score, more points means more likely to be spam. HTML typically adds a small penalty, javascript adds a bit penalty. Bayesian systems that see the Javascript will quickly learn to penalize any javascript tags. 2. The filter can filter based on what the user sees, not the raw feed. SpamAssassin already does this to catch people using HTML to try and break up words.
All in all, this isn't worrying to me at all.

--
Search 2010 Gen Con events

Comment removed by account_deleted · 2004-09-22 01:29 · Score: 4, Interesting

Comment removed based on user account deletion

Plugin Architecture by CleverFox · 2004-09-22 01:32 · Score: 5, Interesting

The real news here is not Bayes filtering or SURBL, but the totally rebuilt plug-in architecture of SA 3.0. Plug-ins for the 2.x version were quite a bit harder to write.

Version 3.0 will result in a proliferation of good third party plug-ins that are going to put SA into more direct competition with some of the commercial vendors out there.

actually i've always felt their name's not right by Build6 · 2004-09-22 01:33 · Score: 4, Funny

You know, *assassins* are the type to take out single, lone, "high value" targets, right?

Sneaking into fortresses/castles, creeping up and then offing the bad guy, or else maybe using some nice long-distance sniper rifle to take out the bad guy, or maybe choice application of poisons at the right bottle of wine, etc.

This is not appropriate for *spam*, where we're talking about waves upon waves upon unending waves in what we would call a "target rich environment". Assassination? No, more like machine-gunning, or artillery, or, I dunno, nukes.

Assassination would take too long.

anto-spam by Outsider_99 · 2004-09-22 01:35 · Score: 4, Informative

Ive been playing with DSPAM which seems very good. They claim a 99.991% accuracy. Apparently this is 10 times more accurate then a human. But Ive heard that most anti-spam solutions are very good.

Re:anto-spam by Skuto · 2004-09-22 01:40 · Score: 5, Interesting

There was a good scientific test linked on slashdot a while ago, comparing spamfilters and including DSPAM and SpamAssassin.

Contrary to DSPAM author's claims, both it and and CRM-114 (another package which likes to self-hype) performed quite a bit worse than SpamAssassin.

Then again, I've heard people being happy with DSPAM that were not happy with SA.

Guess it depends on the mailfeed you get.
Re:anto-spam by MartinB · 2004-09-22 02:26 · Score: 2, Interesting

SpamAssassin 2.x with well trained (>1 year of spam @ 100+ spams/day) Bayes:
~5% false negative (~95% spam filtering accuracy, 1 in 20 spams let through).

DSPAM with large training corpus (~10k spams from a honeypot) plus 6 weeks of real mail at same spam rate:
0.45% false negative (99.55% spam filtering accuracy, 1 in 222 spams let through).

I now publicise an inoculation honeypot address: yumyum@easyweb.co.uk for spammers to harvest, which adds super-strength training.

I'm very happy with my move to DSPAM.

Further, I don't believe heuristic filtering works any more, particularly if you're using published heuristics/shared rules. Spammers adapt too quickly, and test their spam against known rulebases. The solution is I believe to go entirely statistical, allowing each user to have their own definition of spam that is untestable by spammers.

(Incidentally, ever seen the SpamAssassin header forgery spam now being used?

--
The only thing you can accurately describe as "Scotch" is a sticky tape made by 3M. And it's

Release notes? by degradas · 2004-09-22 01:36 · Score: 2, Interesting

Anybody have a link to the changes compared to the last stable version?

Re:Release notes? by AnotherScratchMonkey · 2004-09-22 01:52 · Score: 5, Informative

You can browse the version 3.0.0 Subversion repository. I'd suggest looking at the files UPGRADE and Changes.

Re:Artificial intelligence was born... by Duke+Thomas · 2004-09-22 01:36 · Score: 5, Funny

"Here I am, brain the size of a planet and they ask me to filter your spam."

Buzzword use detection alert. by Grumpy+Troll · 2004-09-22 01:37 · Score: 3, Funny

Snazzy!

This word has triggered the filter of the buzzword use control, as a level 5 'useless/idiotic buzzword'. This is the last time we would like to have to alert, severe sanctions will prevail on next use of such a buzzword.

Thank you for your coordination,

the Buzzword Police.

Purple Bayes... by jav1231 · 2004-09-22 01:37 · Score: 3, Interesting

I use SA and like it. I only get about 75% reduction because SA-Learn doesn't seem to work very well. I've been told it takes a lot of mail to get it to learn. Though I would think, "If you see this again kill it" wouldn't take but once. hehe

A spam arms race? by zaxios · 2004-09-22 01:39 · Score: 4, Insightful

And will SpamAssassin's effectiveness erode as spammers adopt smarter methods in response? Escalation is not a long-term solution to any arms race or conflict. We can continue to fight spam, but the only way we will decisively defeat it is by acknowledging it as a social problem and legislating against it, with an common sense certainty and determination no one in Western goverments seems to be providing.

Re:A spam arms race? by joshtimmons · 2004-09-22 01:46 · Score: 3, Interesting

Well, I'm using spamassassin on my server (and have been for the past 2 years). Unfiltered, I get around 200 spam per day. 1 or 2 get through.

It's been that way since the day I installed it. and it doesn't appear that the spammers are using any substantially "smarter methods"

Maybe it really is easier to write a filter than it is to write filter-proof spam.
Re:A spam arms race? by garcia · 2004-09-22 02:05 · Score: 3, Funny

The only way to stop drugs is to realize that it is a social problem and legislate against it. Use enforcement to stop drugs in their tracks.
Re:A spam arms race? by celerityfm · 2004-09-22 03:04 · Score: 2, Insightful

We use SpamAssassin in between our post office and our smtp server and its been working great at filtering spam. But I always wonder why the spammers don't first try running their spam through spamassassin before sending it us. I mean sure they can't beat the bayes filter, but really theres no excuse for sending out mail that trips any of SA's other filters! WTF are they thinking sending out messages that trip even the simplest of filters like malformed headers or whatever? Goes to show you that spammers aren't interested in spamming people with spam filters if they don't even bother to see if their spam can make it past them in the first place.

Its kinda like The Club(tm). Its just enough to make the spammer go to the other car, so to speak.

--
...unfortunately no one can be told what The Mat^H^H^HGoatse is...they must experience it for themselves...

Re:Improved Performance? by xcomputer_man · 2004-09-22 01:40 · Score: 5, Interesting

I've been using RC1 for over a month now, and I'll tell you confidently that

-- Performance is MUCH better than it used to be. It scans messages much faster than I've ever seen SA 2.x do, and doesn't hog my server's resources anymore.

-- THIS THING ROCKS. For almost two weeks after I installed it I kept instinctively sending myself test emails to make sure I hadn't broken my mail system, because my volume of incoming mail had reduced so drastically. I was used to getting at least a new spam every 2 minutes. After installing SA 3.0 I got one false negative in a 72 hour period. It is *that* good. To date I still have not recorded a single false positive. I really had to convince myself that this thing was real.

This spamfilter rocks. I'd award it product of the year if I could.

--
Am I a hipster-doofus?

New logo ... by YetAnotherName · 2004-09-22 01:40 · Score: 4, Interesting

Didja notice the Apache feathers on the arrow in the new logo? Nice touch!

New logo by slaad · 2004-09-22 01:40 · Score: 2, Funny

I feel like that new logo should be subtitled "Message for you, sir!"
Perhaps a good "thwooop!" sound effect would go well with it too.

--

~Warning!~ The above is encrypted using rot676!

Performance by smooc · 2004-09-22 01:44 · Score: 4, Interesting

What I would like to know, how does SA scale? About a year ago a talked to my ISP about it and they said they could not use it as it did not scale well and could not handle big loads.

It would be nice if it could be implemented now as I personally receive about 1000 spam messages a week.

--
- In Memoriam: Jeroen de Bruin (1972-2004), bye bro

Re:Performance by SCHecklerX · 2004-09-22 03:59 · Score: 2, Informative

It scales just fine. We are using it in comination with Mimedefang's multiplexor here. Mimedefang is great, since you can totally reject stuff that fails basic tests without having to even bother invoking your spam filters.
We have 3 linux dual-processor mail servers, and have basically maxed them out with memory so that we can use ramdisks for Mimedefang processing. CPU Utilization is currently <10%.
Some stats from yesterday:
Spam stats: Total Mail In and out: 39320 Processed by MimeDefang: 24210 Spam: 1519 Non-spam: 22691 Rejected stuff: Completely discarded: 6453 Reject 554 total: 31343 Reject 550 total: 18151 Reject 501 total: 4376 Rejected pre-greeting total: 1273 Suspicious Header total: 1 Partial MIME type total: 0 Non-multipart total: Forbidden File attachment total: 60

Does it use IP's or URI's ? by NKJensen · 2004-09-22 01:46 · Score: 4, Insightful

From the SURBL site: "parse URIs in message bodies, extract their domains, and check those against a SURBL...."

I would rather extract the domain, look up the IP, and check the IP.

That way the server will have to move to a new IP - not just get a new bogus domain name.

Yes, I know that servers many host many domains:

This will only increase pressure on the spamheaven server admins to get rid of the people who use spam to spamvertize their sites.

--
-- From Denmark

Re:Does it use IP's or URI's ? by jlrobins_uncc · 2004-09-22 01:58 · Score: 2, Interesting

I would hope that it would use IPs also. Our site is currently receiving mortgage spam that slips past our current amavisd-new + spamassasin setup, as well as client-side Apple Mail.app baysean filtering. But one thing is consistent between all of em -- the hostnames in the single link within each of the mails resolve to the same IP address.

So, would either SA 3.0 take care of this naturally, or allow me to easily write a plugin to resolve the addresses in links and apply my own IP address based blacklist?
Re:Does it use IP's or URI's ? by Ewan · 2004-09-22 02:05 · Score: 3, Informative

our Spamassassin 3 release candidate seems to filter on both IP addresses and URIs, seems very effective - our spamassassin now marks over 50% of incoming email as spam.

Ewan
Re:Does it use IP's or URI's ? by NKJensen · 2004-09-22 02:20 · Score: 4, Informative

Sorry, I've found the question and some pros and cons here:

http://www.surbl.org/faq.html#numbered

--
-- From Denmark
Re:Does it use IP's or URI's ? by krunk7 · 2004-09-22 02:22 · Score: 3, Informative

Are you using Rules Du Jour
Re:Does it use IP's or URI's ? by Anonymous Coward · 2004-09-22 02:22 · Score: 3, Informative

This is just a small nitpick, Apple's Mail.app uses latent semantic analysis, not baysean filtering.
Re:Does it use IP's or URI's ? by platipusrc · 2004-09-22 02:22 · Score: 5, Interesting

One of the problems with using IPs is the massive amount of Virtual Hosting being used. Say I'm a 1&1 customer, and there are 400 other domains going to the same IP as one of my domains, and I send you an email with a link to something on my site, but one spammer has managed to get an account with 1&1 for now. If they're on the same box as me, you just blacklisted 399 other domains that shouldn't have been blacklisted.

--
And the muscular cyborg German dudes dance with sexy French Canadians
Re:Does it use IP's or URI's ? by Anonymous Coward · 2004-09-22 02:33 · Score: 2, Funny

I can beat that. I wrote a little perl scrpt to use with procmail that marks 100% of incoming email as spam. 100% vs 50%. My way is better :)
Re:Does it use IP's or URI's ? by AKnightCowboy · 2004-09-22 02:41 · Score: 3, Interesting

If they're on the same box as me, you just blacklisted 399 other domains that shouldn't have been blacklisted.
Extreme spamfighters don't care though. You're guilty by association in their eyes and deserve to feel the same wrath that the spammers do. It's so that you'll bitch to your provider and in turn your provider will shut down the spam site because all their other customers are complaining vs. some random guys on the Internet complaining they're receiving that URL in spam.
Re:Does it use IP's or URI's ? by Kainaw · 2004-09-22 02:47 · Score: 3, Informative

I would rather extract the domain, look up the IP, and check the IP.

I wanted that a long time ago. At the time, I couldn't find a program written by anyone else, so I wrote my own. It works well for me and anyone who wants the script is free to use it. It is at my homepage.

--
The previous comment is purposely vague and generalized, but all of the facts are completely true.
Re:Does it use IP's or URI's ? by NKJensen · 2004-09-22 03:06 · Score: 2, Insightful

If the admins are too slow to take down the spamvertized site and report back to the reporter, yes, and only then.

Which is fine with me.

--
-- From Denmark
Re:Does it use IP's or URI's ? by marmoset · 2004-09-22 03:08 · Score: 3, Informative

Best human-readable discussion of the techniques I've read is here.
Re:Does it use IP's or URI's ? by Ignignot · 2004-09-22 03:19 · Score: 2, Interesting

And so what happens if I decide to send out spam that links to URL http://www.dina.kvl.dk? It looks up the IP address, and blocks it. Then everything that you ever send will be blacklisted. Go get a new host.

Your suggested technique would be exploited by script kiddies everywhere (who already have access to large zombie networks) to basically ban someone from the internet. What a fantastic idea.

--
I submitted this story last night, and it didn't get posted.
Re:Does it use IP's or URI's ? by ePhil_One · 2004-09-22 03:41 · Score: 2, Interesting

Extreme spamfighters don't care though.
May fvorite was a Washington DC news company that had implemented extreme spamfighting measures. Since our outgoing mail server doesn't receive incoming mail, its not in the MX records. This guy was bouncing our mail because of that. God hopes that the next Deep throught doesn't try to contact his news organization...

--
You are in a maze of twisted little posts, all alike.
Re:Does it use IP's or URI's ? by pqdave · 2004-09-22 04:29 · Score: 4, Informative

SpamAssasin is for email, and won't affect anyone trying to browse to your site. At worst, a properly-configured SpamAssasin would see a mention of your URL in an email, resolve it to the same IP as a spammer, and give it a few more points towards the spam threshhold. SpamAssasin (at least as used by my mail admin) scores messages based on various factors rather than giving pass/fail tests, so a suspicious URL in an otherwise non-spammy message wouldn't necessarily send it over the spam threshold.
Re:Does it use IP's or URI's ? by pjrc · 2004-09-22 04:53 · Score: 3, Informative

I would rather extract the domain, look up the IP, and check the IP.
That won't help against "bulletproof hosting", commonly used by spammers, where a nameserver in a country like Russia or Poland resolves the name to one of thousands of zombie machines hosting the site.
The SURBL approach does.
Yes, I know that servers many host many domains: ... This will only increase pressure on the spamheaven server admins to get rid of the people who use spam to spamvertize their sites.
Spammers don't use $10/month shared virtual hosting for their websites.

--
PJRC: Electronic Projects, 8051 Microcontroller Tools
Re:Does it use IP's or URI's ? by ChaosDiscord · 2004-09-22 06:10 · Score: 4, Insightful

If they're on the same box as me, you just blacklisted 399 other domains that shouldn't have been blacklisted.

You're not blacklisting; you're marking as "more likely spam". In practice the damage will be minimal. First, legit email from the other 399 domains will in general be non-spam-like. The positive hit on the IP address won't be enough to push them over the edge. The penalties for being found in the SURBL at the moment are all relatively small, all less than 1 (5 points are needed in the default configuration to mark a message as spam). The only exception is data from the Spam Cop database, which is fairly small and more carefully vetted. If they broaden from hostnames to IPs, you might have to tweak the scores down, but that's it. Second, what's the realistic chance of your getting email containing a URL linking to that IP? There are millions of web sites. The Big Important Web Sites aren't on the sort of massive shared server you describe. The chances that you'll get an email mentioning one of those smaller sites is pretty small. There is a risk, but it's small enought that I won't lose any sleep over it.

--
Search 2010 Gen Con events

3.0 New Features by CleverFox · 2004-09-22 01:46 · Score: 5, Informative

Major feature list:

- SpamAssassin is now part of the Apache Software Foundation and has an
improved software license, the 2.0 version of the Apache License.

- SpamAssassin now includes support for SPF (the Sender Policy
Framework, http://spf.pobox.com/).

- Web site links contained in the message are checked against SURBL and
SBL. SURBL and SBL track sites that advertise with spam, known spam
sources, and spam services.

- The new 3.0 architecture allows third-parties to easily add plugin
modules.

- There is now SQL database support for both the Bayes and
auto-whitelist modules, allowing more large sites to easily deploy
SpamAssassin.

- A more accurate simulation of email client handling of MIME and HTML
improves our accuracy. In addition, there is better detection and
handling of spammer techniques that try to trick anti-spam software.

Important installation notes:

- The SpamAssassin 2.6x release series was the last set of releases to
officially support perl versions earlier than perl 5.6.1. If you are
using an earlier version of perl, you will need to upgrade before you
can use the 3.0.0 version of SpamAssassin.

- SpamAssassin 3.0.0 has a significantly different API (Application
Program Interface) from the 2.x series of code. This means that if
you use SpamAssassin through a third-party utility (milter, etc,) you
need to make sure you have an updated version which supports 3.0.0.

- The --auto-whitelist and -a options for "spamd" and "spamassassin" to
turn on the auto-whitelist have been removed and replaced by the
"use_auto_whitelist" configuration option which is also now turned on
by default.

- The "rewrite_subject" and "subject_tag" configuration options were
deprecated and are now removed. Instead, using "rewrite_header Subject
[your desired setting]". e.g.

rewrite_subject 1
subject_tag ****SPAM(_SCORE_)****

becomes

rewrite_header Subject ****SPAM(_SCORE_)****

- The Bayesian storage modules have been completely re-written and now
include Berkeley DB (DBM) storage as well as SQL based storage (see
sql/README.bayes for more information). In addition, a new format has
been introduced for the bayes database that stores tokens in fixed
length hashes. All DBM databases should be automatically converted to
this new format the first time they are opened for write. You can
manually perform the upgrade by running "sa-learn --sync" from the
command line.

The "sa-learn --rebuild" command has been deprecated; please use
"sa-learn --sync" instead. The --rebuild option will remain
temporarily for backwards compatibility.

- "spamd" now has a default max-children setting of 5; no more than 5
child scanner processes will be run in parallel. Previously, there
was no default limit unless you specified the "-m" switch when
starting spamd.

- If you are using a UNIX machine with all database files on local
disks, and no sharing of those databases across NFS filesystems, you
can use a more efficient, but non-NFS-safe, locking mechanism. Do
this by adding the line "lock_method flock" to the /etc/mail/spamassassin/local.cf file. This is strongly recommended if
you're not using NFS, as it is much faster than the NFS-safe locker.

- Please note that the use of the following command line parameters for
spamassassin and spamd have been deprecated and are now removed. If
you currently use these flags, please remove them:

in the 2.6x series: --add-from, --pipe, -F, -P, --stop-at-threshold, -S
in the 3.0.x series: --auto-whitelist, -a

- The following flags are de

Re:Artificial intelligence was born... by Scarblac · 2004-09-22 01:48 · Score: 4, Interesting

Artificial intelligence was born... Filtering spam.

In Greg Egan's _Permutation City_, spam filters and spam become ever more intelligent. Your spam filter runs the interactive video mail in a sandbox trying to detect whether it's spam, the spam tries to detect that it is in a sandbox or that it is talking to an AI construct, so that it can hide its commercial intent. Your filter tries to mimic you (and you review its reactions now and then, try to get its facial expressions ever more like yours, etc), the spammers try to get more information about you so they can try to fool your filter by making the spam look like on of your friends, etc.

This is an obvious arms race and in that book, AI and uploaded individuals etc exist - but the trick is to make your AI spam filters as good as possible without making them actually self-conscious, since using self-conscious AI software for spam filtering would be torture.

I rather liked that idea.

--
I believe posters are recognized by their sig. So I made one.

Re:Improved Performance? by slobbargoat · 2004-09-22 01:49 · Score: 2, Funny

does anyone know of any GPL win32 anti-spam utilities capable of working with exchange?

Curse you ISP! by Anonymous Coward · 2004-09-22 01:53 · Score: 2, Insightful

"Perl v5.6.1 required"

Sigh. Now I have to fight with my ISP to get a semimodern version of Perl installed.

For those who may have forgotten by numbski · 2004-09-22 01:55 · Score: 5, Informative

I'm building the latest on all of my clients' mail exchangers and our primary boxen. ;)

Here's the command to install/upgrade 3.0 via CPAN:

# perl -MCPAN -e shell;
cpan > install Mail::SpamAssassin

(many lines, type in the administrator's e-mail address, say no to network tests)

exit

#

Very difficult stuff. :) Keep up the good work.

Oh! Some link whoring as well:

SpamAssassin Milter for Sendmail - Filters everyone without procmail

SpamAssassin Milter Quarantine - Quarantines spam messages and sends summaries in digest for 1 or more times daily rather than simply delivering to the end user.

--

Karma: Chameleon (mostly due to the fact that you come and go).

Re:For those who may have forgotten by davemabe · 2004-09-22 02:29 · Score: 3, Informative

There's also sa-exim which is a local scan plugin for Exim that does site wide spam filtering at SMTP time. It is quite nice.

Dave

Re:Improved Performance? by eddy · 2004-09-22 01:58 · Score: 3, Interesting

[...] and doesn't hog my server's resources anymore.

Got any numbers on memory use? I would love to run SA on my home server, but it has "only" got 80MB of RAM. I tried running 2.x, but it seriously brought the system to its knees (swapping)

I must say, Python might be a nice language and all, but as it's making inroads everywhere it's also wrecking havoc on ones ability to convert older hardware into a competent server. YMMV (mailman + bittorrent + (apache + exim + samba) and you're pretty much down to the last few megabytes )

--
Belief is the currency of delusion.

Damn... by Anonymous Coward · 2004-09-22 02:02 · Score: 2, Interesting

Am I the only one that loved those cheesy little plastic ninja dudes in the old logo?
In fact, I thought their logo contest rules suggested that they would prefer the new one to contain those guys still, in some way or another.

Better names? by Da+Twink+Daddy · 2004-09-22 02:03 · Score: 5, Funny

Well, since it's capable of removing a certain caste of emails entirely how about SpamGenocide or SpamacialCleansing?

Perhaps we should identify it with (im)famous person(s) to drive up hits like SpamHitler, SpamNazi, or SpamlobodanMilosevic?

Maybe something that has an associated coolness factor, instead of being (almost) universaly hated, like Dr. Spamibal Lecter?

Well, there's still the problem of overwhelming evil there. It's not really evil, just heartless and calculating. Hmm, heartless, calculating, killer... I got it! How about SpamAssassin? Oh, wait...

Spam is a technical problem, not political by ftzdomino · 2004-09-22 02:04 · Score: 2, Interesting

Email was designed to trust everyone, making it hard to reject email from people you don't want it from. We must get everyone to move to a better architecture which can force sender authentication if desired by the receiver. My own personal preference would be to have the sending MTA sign outgoing mails with a public key. Any scheme be much easier than getting 100% of governments to outlaw spam, which is what is needed to be effective. Legislation is not the answer to a technical problem.

Re:actually i've always felt their name's not righ by shadowkoder · 2004-09-22 02:05 · Score: 2, Insightful

Well, though there may be a large amount of spam, doesn't like 90% of it get sent by the top 10% of spammers, or something to the same effect? If you could whack off a couple of the top guys ...

fillters vs. stallers by Anonymous Coward · 2004-09-22 02:07 · Score: 4, Insightful

When do people learn that
what we need is not spam filters but spam stallers.

With spam filters your just precipitating in a arms race.

The spammers will send more and more spam
and your spam filters will use more and more
of your processor time to filter the spam.
It is a uphill battle against the spammer.

With spam stallers like sa-exim and tarproxy
your are stalling the spammers smtp connection
and the effect is that the spammer can't send
as much spam or that they drop you email from there email database.

Re:fillters vs. stallers by Anonymous Coward · 2004-09-22 02:31 · Score: 2, Insightful

This is not exactly true. If you can manage to get fewer eyeballs viewing the spam, you should get fewer people responding to spam. This is removing the economic incentive while keeping the costs the same. In other words it becomes less profitable.
Re:fillters vs. stallers by SnarfQuest · 2004-09-22 02:43 · Score: 2, Insightful

Are you actually stalling the spam producer, or are you stalling someone who is just forwarding the mail? Most mail does not go directly from the producer to your machine, but goes through numerous forwarders before it gets to you.

What you are likely doing is pissing off the people who are willing to forward your mail to you, and they may decide to just drop all of your mail instead of being screwed with.

--
Who would win this election: Andrew Weiner vs Andrew Weiner's weiner.
Re:fillters vs. stallers by Coppit · 2004-09-22 06:24 · Score: 2, Insightful

With spam filters your just precipitating in a arms race.
True, but the purpose of a spam filter is to only let legitimate email through. If that encourages spammers to start writing legitimate emails, great! You might argue that they are writing legitimate looking emails, but SpamAssassin has always been 1 step ahead of them.
With spam stallers like sa-exim and tarproxy your are stalling the spammers smtp connection and the effect is that the spammer can't send as much spam or that they drop you email from there email database.
I agree. However, you still need to first figure out they are a spammer, right?

New logo. by suso · 2004-09-22 02:08 · Score: 3, Funny

The new logo is nice, but I was kinda partial to the nunchaku wielding ninjas knocking the crap out of spam.

still waiting for spammerassassin by Daniel+Ellard · 2004-09-22 02:10 · Score: 4, Funny

This looks great, and I look forward to using it, but it doesn't address the root of the problem. Anyone working on spammerassassin yet?

--
Disclaimer: I work for a company, but I don't speak for them.

Re:still waiting for spammerassassin by geeklawyer · 2004-09-22 02:51 · Score: 4, Funny

Yes,
The spammerassassin team is active, but on my legal advice they are not documenting their work: it could, technically, be argued to be murder.

--
-he who laughs last, is a bit slow.
journal
Re:still waiting for spammerassassin by jeffguy · 2004-09-22 04:51 · Score: 3, Insightful

from a legal dictionary.

murder
n. the killing of a human being by a sane person, with intent, malice aforethought (prior intention to kill the particular victim or anyone who gets in the way) and with no legal excuse or authority...

No worries, spammers don't qualify as human.

Debian installation by Eater · 2004-09-22 02:36 · Score: 2, Interesting

Any word of a .deb for this? I am so tired of v2.64.

McAfee SpamKiller based on SpamAssassin no good by aardwolf204 · 2004-09-22 02:43 · Score: 3, Informative

A lot of closed source software has open source counterparts, (i.e. MS and Open Office) but its always interesting to see closed source commercial software based on an open source project.

McAfee has a product for Exchange servers that is based on Spam Assassin called Spam Killer. I found out about it from the Spam Assassin site when I was looking for a windows version. Spam Killer isnt free yet its not as expensive as some of the other solutions out there.

The major problem I've been having with it is it creating zero byte emails which cannot be downloaded via pop3. When a user gets 30 messages, and message 10 is a zero byte email the client will constantly download the first 10 over and over, creating duplicates, until the user logs into outlook web access (webmail) and deletes the zero byte message. This doesnt happen to the MAPI users but we have quite a few POP3 users.

The support people are useless, I'm about to try out Microsoft Intelligent Message Filter for exchange, and hopefully with some good RBLs it should be ok.

--
Im dreaming ofa big bndwdth, That can resist the /.crowd.May ur days b merry & bright & may al

I no longer get spam by Anonymous Coward · 2004-09-22 02:53 · Score: 2, Informative

The solution is extremely simple if you use OpenBSD.

rdr on $ext_if from any os "Windows" to any port smtp -> 127.0.0.1 port 8025

99.9% of all spam comes from compromised Windows boxen, and nobody with a clue would run a mail server on windows.

Turbo Smorgreff

Great Book on it by Erwos · 2004-09-22 02:54 · Score: 2, Interesting

I recently read an excellent book on SpamAssassin by Alan Schwartz, published by O'Reilly and Associates, Inc. My views might be biased since he's my first cousin, but if you're a mail server admin, it's probably a must-have. I don't think it covers desktop usage as well, but then again, Evolution's getting that integrated anyways.

The sections on rules are extremely nice, and I found them pretty informative as to how the software works underneath. It covers version 3, too, so it's damned timely.

-Erwos

--
Plausible conjecture should not be misrepresented as proof positive.

Spaminator by KaiBeezy · 2004-09-22 02:56 · Score: 2, Interesting

Earthlink Spaminator(TM)

Seems like they're kind of wasting a name that would work pretty well in the market.

Antispam Gateway Distribution? by Darth_Burrito · 2004-09-22 02:56 · Score: 2, Interesting

This may seem a tid bit lazy but...
It seems like there are linux distributions for just about anything you might want: routers, pvrs, etc. Are there any linux distributions designed to be a mail anti-spam/anti-virus (or just anti=spam) gateway? ... something that would install and configure postfix, spam assassin, etc to receive mail and forward it to another server after filtering it.

The reason I think this would be cool is because configuring mail apps on linux can be hard and because this would be a great linux foot-in-the-door distribution for Exchange admins who didn't want to pay thousands of dollars for antispam gateways.

Re:Antispam Gateway Distribution? by mortonda · 2004-09-22 04:17 · Score: 2, Informative

This is exactly the direction we are planning to go with Maia Mailguard, plus features such as tarpitting, network reporting, and p2p associations. It's going to take a while to get there, though.

Exciting! by theamarand · 2004-09-22 03:07 · Score: 2, Interesting

I've been using the 2.63 version of Spamassassin for a few months now, and it's surprising how well it works, especially when you use the "spam/ham" folder saving feedback system. I've noticed a lot fewer miscategorizations, which gives me a better feeling about using the app site-wide (I'm just using it for three users right now). I'm really excited about the potential for a major release like this having significant and noticable improvements in key features like heuristics and integration. The logic improvements will help end-users feel better about setting things up a certain way, then forgetting about it. Integration (it's great that it's under the Apache S.F. umbrella now) means that more people will get behind supporting it, which follows with increased feature richness, improved algorythms and rapid filter development. In the end, though, myself and my users just love seeing the spam marked out in an increasingly accurate way...so it becomes second-nature to just rapidly press the delete key without much thought....

Re:Improved Performance? by Tim+Macinta · 2004-09-22 03:08 · Score: 4, Informative

I must say, Python might be a nice language and all, but as it's making inroads everywhere it's also wrecking havoc on ones ability to convert older hardware into a competent server.

Spamassassin is actually written in Perl, not Python. I'm not saying your point about certain languages making it difficult to maintain older machines isn't valid, I'm just clarifying what Spamassassin uses.

--

-----

Free P2P Backup, Windows & Linux

You'd be amazed by CmdrGravy · 2004-09-22 03:09 · Score: 3, Interesting

Someone in the place I used to work at had an e-mail of someone else which had a signature which scrolled in from the right of the page and flashed and stuff and from there in around 2 months more than 90% of everyone else in the office had the same thing. I believe this relied on Javascript and Outlook was more than happy to comply.

An invitation to fellow spam-fighters by bigberk · 2004-09-22 03:37 · Score: 2, Interesting

SpamAssassin, when properly configured, has spectacular spam detection accuracy. For your account or for a small domain, you should be able to see SA yield "near perfect" filtering (i.e., probably as good as a human could pull off).

That's the point at which we become interested in SpamAssassin users joining WPBL, an automated spam reporting system. Powered by scripts living in procmail and cron, participating systems send WPBL lists of IP addresses sending spam and ham. The central server crunches this data hourly to produce a list [rsync://rsync.pc9.org/wpbl/wpbl-blocks.cidr] of blocked IP addresses that are spam sources.

If your site uses SA and you have verified your spam detection accuracy as nearly-perfect, you might be interested in contributing your spam/ham sighting stats to WPBL. The resulting block list can be used by anyone (and is used by some ISPs for spam scoring). The way I think of it is, after you've taken care of the spam problem at your site why not help tell the rest of the world where spam is coming from.

TMDA by Gudlyf · 2004-09-22 04:23 · Score: 2, Informative

I've used SpamAssassin for quite some time now, but I was still getting spam through, mostly because I'm a paranoid freak and figured I'd be missing out on mails that it mistakenly tagged as spam.

What I use now (alongside SpamAssassin) is TMDA. This is basically an "approval queue" for messages. If someone not in your approved list send you mail, they get a reply telling them they need to send mail to a specificly generated address in order to allow the mail to pass through to me. Eventually mails that don't get approved time-out and get added to a blacklist for the future. I also quickly review the queued items every morning in case someone didn't see the approval mail (it has a tool that allows you to easily peruse the list with just subject and sender info). So far I've gotten NO spam through this method -- NONE. I used to get hundreds a day, and now I have a spam-free INBOX because of TMDA.

While I highly recommend using TMDA, it may not be for people running businesses or waiting for mail from clients. The auto-reply message can perhaps strike some as inconvenient, even though they only have to do it once (once they've sent mail to the approval address, they're added to the whitelist for all future mails). So far spammers haven't found a way around TMDA it seems...so far.

--
Trolls lurk everywhere. Mod them down.

Installing on Windows....you're kidding, right? by Chris+Carollo · 2004-09-22 05:29 · Score: 4, Insightful

So I've heard good things about SpamAssassin and headed over the webpage to figure out what I needed to do to install, and I found this.

I'm probably going to flamed for this, but that install process is ridiculous. I'm not even close to being a newbie, but there's no way I'd go through that much hassle to install a spamblocker compared to something like SpamBayes that does a standard windows install and hooks right into Outlook. Does anyone thing that these things are reasonable?

1. I'm supposed to extract it to the root of my drive. Sorry, my root is sacrosanct. If the /. crowd is going to complain about RealPlayer dumping shortcuts in my desktop, quickstart bar, and main start menu, how is SpamAssassin making directories in my root any better? At least I can delete the stuff RealPlayer litters around.

2. I've got to install Perl modules? And it doesn't work with certain versions of Perl? The install should include whatever it needs to run. Don't make me track down some particular version of outside software.

3. I've got to generate a batch file and run it to generate the documentation? Why not just include the generated documentation?

4. Step 10 of the install FAQ mentions a D drive. I don't have a D drive. Does SpamAssassin really require TWO drives to run/test properly?

5. The whole install process includes 13 steps, some of which are fairly complicated.

This is one of the reasons why the whole open-source initiative has such a bad, pointy-headed reputation. Where is the focus on usability and user-friendliness? I often get the impression that it's "not cool" to actually put time and energy into making your software anything other that esoteric in its usage. I realy would like to try SpamAssassin, but dealing with the minor annoyances of SpamBayes for the next six months is clearly less work than installing SpamAssassin today. Why doesn't that bother anyone?

I'm probably going get either flamed or ignored for this post, but I would appreciate a reasonable response if there is one. We'll see I guess.

Re:Installing on Windows....you're kidding, right? by Clover_Kicker · 2004-09-22 06:45 · Score: 2, Informative

SpamAssassin on Win32 is an afterthought at best.

The Win32 stuff is provided as a courtesy. I don't think they really expect anyone to use it, since it is so much easier to install on *nix.
Re:Installing on Windows....you're kidding, right? by slappyjack · 2004-09-22 07:06 · Score: 2, Interesting

I think the point you're missing is that these instructions are very complete and fills in every step of the process. Far different from doubleclicking a big fat executable and watching the pretty progress bar.

I for one prefer this kind of install when loading up geeky type things like this. You learn more about your machine and the application, what its doing, and where it is in case you want to modify or otherwise play with it. And really, how can you NOT want to konw this? Your computer is a tool. The more you konw about it the more powerful it becomes.

If you don't want to do all of that, then suffer with what is probably an inferior product. Not all freeware is entirely "free".

On the flip side: when I'm installing games, I'm more than happy to just sit back, drink my beer, and watch the pretty little installation graphics twirl and dance for me. I just wanna get to the killin'.

--
s'wut i sed.
Re:Installing on Windows....you're kidding, right? by Christopher_G_Lewis · 2004-09-22 08:41 · Score: 3, Informative

First of all, there is no install. This is a pure source release. Quite common, and after a little bit of testing, (you wouldn't blindly put this on a production box, would you :-) it's quite easy.

Your points...
1. Extract it where ever you want.
2. So? PPM and CPAN are simple.
3. or you could use the docs on the web site you were looking at.

4. Step 10 does *not* require a D drive, the -D is for Debug mode. It spits out everything that SpamAssassin is doing, i.e. what config files, what db's what tests are being run. Actually quite usefull.

5.

--
www.christopherlewis.com
Re:Installing on Windows....you're kidding, right? by sidney · 2004-09-22 13:18 · Score: 4, Informative

I did a lot of the work of getting SpamAssassin to build and run on Windows. My goal was to have SpamAssassin build and install on Windows using the unmodified sources before version 3.0 was released. It does that now.

SpamAssassin was written in Perl on Unix and Gnu/Linux, for use in high volume server environments. The installation for an ISP or for anyone running a *nix mail server is a piece of cake. Their users get their mail filtered without having to install anything on their own PCs.

The fact that it works on Windows at all is a bonus. It is an open source project. Would anyone like to volunteer to help with the next steps of getting the server daemon, spamd, working properly in Windows as a service; writing or adapting an existing mail proxy that would integrate SpamAssassin with mail clients such as Thunderbird, Mozilla Mail, Eudora, Outlook Express; packaging it up in a standard Windows install package?

Addressing the 5 points in the parent post:

1. Nothing has to go in the root directory. The instructions show an example of Perl having been installed in C:\perl and configuration going in directories underneath a C:\etc\mail directory.

2. Yes you have to install Perl. And a recent enough version that doesn't have certain bugs. And the required modules. SpamAssassin was written in Perl, which makes it useful on systems that have Perl, such as most Unix and GNU/Linux systems. If you install Perl and the modules on your Windows system then you have a system that meets the minimum requirements. If you have a Palm Pilot or or an Xbox or Windows without Perl then your system does not meet the minimum requirements and you are not going to even try to run SpamAssassin on it. In that case install SpamBayes, or get an ISP who uses SpamAssassin for your mail, or any of many other alternatives.

3. Making the doc files is easier in *nix. I'll file a request for enhancement suggesting that generating the HTML be made part of the Makefile and that it be made to work under Windows. The doc files are generated from the sources as part of the build, so they are not included in a source distribution, which is what we are talking about here. If someone built a binary distribution they would include the doc files.

4. That -D command line option stands for Debug, not D drive

5. The whole install proces consists of 13 steps, some of which are things like "download SpamAssassin", some of which are "if you are installing the old version 2.6x do this extra step", and some of which have to do with getting the required Perl and Perl modules. The actual installation pretty much happens in three lines of step 7. It really is quite easy for a build and installation starting from source files. A binary installation package would be a lot easier. Does anyone know how to package perl plus modules plus a built SpamAssassin into a Windows install package? If you do, feel free to volunteer.

The focus on usability and user friendliness is where it should be in this particular project, on the sysadmin who installs SpamAssassin on a server and on their end users who don't have to install anything at all.

If you have the ideas and the expertise to also make SpamAssassin more useful and friendly to the end user owner of a PC running Windows, please volunteer to help.

Want to learn more about SA 3.0? by felicity · 2004-09-22 05:59 · Score: 2, Informative

At ApacheCon 2004, there will be 3 presentations about SpamAssassin: a 3-hour tutorial covering SpamAssassin as a whole, a 1-hour talk about the new features in 3.0, and a 1-hour talk about using the new SQL features.

Sorry for the plug, but I thought may be interested. :)

Your ISP sucks then. :P by EvilStein · 2004-09-22 07:25 · Score: 3, Informative

I'm using it on a dual 1.6ghz Xeon box with Gentoo here in the office - the box processes over 70,000 emails per day (spamassassin, amavisd-new and clamav/f-prot) and the load average barely goes above 0.02.

Your ISP just didn't want to take any time to actually learn about it. :)

85 of 335 comments (clear)