SpamAssassin 2010 Bug

SEWilco writes "You might want to check your spam folder, as SpamAssassin has a rule which is tending to mark email sent in 2010 as spam. There is some discussion in a bug report. The SpamAssassin Wiki FH_DATE_PAST_20XX page doesn't have discussion, but it was updated today with a different date rule."

Comments

crapola

[DNS-and-BIND]

My provider runs spamassassin, and given their track record in updating their other software, I rather doubt that they'll update spamassassin anytime soon. Is there any way around this that doesn't involve root access? (I love helpful responses from idiots that start with "first, edit the /etc/spamassassin.conf file" or whatever.)

Oh yeah, the other wonderfully helpful stock response "stop using the software if you don't like it". Sure, I'd love to go back to getting 500 spams a day.

Re:crapola

[godrik]

I love helpful responses from idiots that start with "first, edit the /etc/spamassassin.conf file"

Yes. That's easy! First edit the /etc/spamassassin.conf file!
Wait a sec...

Re:crapola

[DaMattster]

Well, I am not sure what OS your running, but you can use OpenBSD Spamd which works very well. Rather than taking a defensive approach, Spamd goes on the offense by allowing known spam-sending IP addresses to attempt to send to you but throttling the connection down to 1 byte per second. This shakes most people off with no perceivable impact on your part. Even if the spam bot decides to wait the entire time to complete the connection, Spamd ends up dropping the message anyway. I use this solution in my business and I've gone from getting 500+ per day to maybe 2 per week. It is delightfully elegant.

Re:crapola

[ngc5194]

"Is there a way to work around this that doesn't involve root access?"

Yes, but it isn't a good way. Check your scores file for the scores associated with the FH_DATE_PAST_20XX. This indicates the number of points added to the spam score of every message that fails this test. Basically, increase your spam threshold by this amount until you can apply this patch.

Good for a quick-n-dirty fix.

Re:crapola

[karnal]

I think you missed the "My provider runs spamassassin" part of the parent post.

Re:crapola

[smartaleckkill]

depends--i have a cheap n cheerful shared hosting account with the same issue, but i do have cpanel access which allows me to override the score for any rule--check out the last link in the summary basically if you have access to local config files (even through a frontend like cpanel) you can do it without root access

Re:crapola

[mhrivnak]

The new rule gets picked up when "sa-update" is run. spamassassin deployments should run sa-update automatically on a regular basis, for example every day via a cronjob. Thus, most deployments will pick up the update automatically tonight if a sysadmin doesn't do it first.

Re:crapola

[Mana Mana]

OpenBSD spamd(8) is wholly unrelated from spamassassin spamd. FYI.

OpenBSD spamd(8) has no code from any other project. Its similarity in appellation is name deep.

OpenBSD spamd(8) approach is different and was created by deraadt@.

Re:crapola

[nabsltd]

My provider runs spamassassin, and given their track record in updating their other software, I rather doubt that they'll update spamassassin anytime soon. Is there any way around this that doesn't involve root access?

If you have shell access, it should be trivial, although you do have to edit a file.

Add the following to ~/.spamassassin/user_prefs:

score FH_DATE_PAST_20XX 0.0

Re:crapola

[Penguinshit]

Not to rub it in but I want to give a shout to my provider (Cruzio) who also use Spamassassin and were apparently on top of this (as they usualLY are0. i haven't noticed anything wrong with my email today. Thanks Cruzio!

Re:crapola

[Anonymous Coward]

You really do have a hard time reading and comprehending what people post, don't you?

Re:Millenium bug, how I have missed thee

[fuzzyfuzzyfungus]

Given what memory cost in 1970, I suspect that using 64-bit time would have been an expensive decision.

A lot of gross little hacks look like (and are) great ideas when hardware costs a fortune and you don't yet know how persistent legacy stuff is going to be.

FIX details:

[drDugan]

this is also happening on Ubuntu server, running Spamassassin 3.2.5

The linked article references a workaround:
add this line to the "local.cf" spamassassin config file, on this system is was /etc/spamassassin/local.cf

score FH_DATE_PAST_20XX 0.0

If you're running spamassassin as a daemon, you *may* also want to restart spamd
with something like:

sudo /etc/init.d/spamassassin restart

This solution simply removes the rule by setting the score for that rule to 0.
You'll want to undo this once a solution is deployed.

Re:FIX details:

[KiloByte]

Since nearly 14 hours ago, you can simply run "sa-update".
It is in cron.daily in the default install, too.

Re:"I'll just use a regex!"

[Briareos]

Last I checked a timezone was mandatory in mail dates, allowing you to handle this correctly - for example by converting your current date and the date specified in the email to GMT...

One hack replaced by another

[tomp]

From the "fix"

> FH_DATE_PAST_20XX
> change '/20[1-9][0-9]/' to '/20[2-9][0-9]/'

That's no fix, it just puts the problem off for another 10 years. Why call the rule FH_DATE_PAST_20XX, shouldn't it be FH_DATE_PAST_201X? At least then the hack would be documented.

Re:One hack replaced by another

[JWSmythe]

    I just fixed a mail server for someone (after reading the article), and told them to remind me in 10 years to fix it again. :) Hopefully by then, they'll have rewritten the rule so it behaves better.

Re:One hack replaced by another

[6Yankee]

Exactly. This is the right "fix" right now - the bug is out there causing real problems, and the fix (while definitely a filthy hack) is well-understood and can be pushed out immediately. If the same thing were to happen ten years from now (or the threshold quietly got pushed back to 2030), that would be nothing short of negligent.

Nice....

[kramer]

[url]https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5852[/url]

Noticed 14 months ago. Fixed 5 months ago. Released today.

Re:Nice....

[Bigjeff5]

Hey! Nice to see open source software gets fixed so ultra fast! :P

To be fair though, at least they released it the day it broke things, why didn't they release it by yesterday? Then the default cron job would have picked it up on most servers and nobody would have noticed.

Re:"I'll just use a regex!"

[ubrgeek]

It depends. If the email from San Francisco was traveling east going 400 miles per hour and the email from NZ is going west at .... ;)

Re:Fixed in spamassassin 3.2.5-7 in Debian/Unstabl

[John Hasler]

toncho/~ sudo apt-get install spamassassin
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
    libmail-dkim-perl
Recommended packages:
    re2c
The following packages will be upgraded:
    spamassassin
1 upgraded, 0 newly installed, 0 to remove and 1086 not upgraded.
Need to get 1097kB of archives.
After this operation, 0B of additional disk space will be used.
Get:1 http://ftp.us.debian.org/ unstable/main spamassassin 3.2.5-7 [1097kB]
Fetched 1097kB in 13s (84.2kB/s)
Reading changelogs... Done
apt-listchanges: Mailing root: apt-listchanges: news for toncho.dhh.gt.org
(Reading database ... 163295 files and directories currently installed.)
Preparing to replace spamassassin 3.2.5-6 (using .../spamassassin_3.2.5-7_all.deb) ...
Stopping SpamAssassin Mail Filter Daemon: spamd.
Unpacking replacement spamassassin ...
Setting up spamassassin (3.2.5-7) ...
Starting SpamAssassin Mail Filter Daemon: spamd.

Here is the apt-listchanges message:

spamassassin (3.2.5-7) unstable; urgency=high

      This version of SpamAssassin fixes a bug which caused mails sent
      in 2010 to be flagged as suspiciously spammy. If upgrading to this
      version, you are recommended to update any per-user caches previously
      created by sa-compile, and to check mail already in your spam folder
      for false positives more carefully than usual.

  -- Joey Hess Fri, 01 Jan 2010 12:03:40 -0500

holy crap

[Anonymous Coward]

Thanks for the heads up, my kid's birthday party is next weekend and when I look at the spam folder it turns out 3 more people have replied that I hadn't seen.

Great workaround

[xororand]

The suggested fix is just silly... They postpone the problem to 2020-01-01:
3) change '/20[1-9][0-9]/' to '/20[2-9][0-9]/'

Re:Great workaround

[doshea]

We're likely going to drop the rule all-together in the future. In the interest of getting the fix out fast, with the least confusion of users, we decided to just make the change to 2020 for now.

Although... maybe a little visibility for the project every 10 years wouldn't hurt. :)

End User

[Kenshin]

As an end user, with no real ability to configure SpamAssassin other than through cPanel, what can I do about this? (Unless my webhost is right on top of it.)

I've disabled it, for now.

Re:End User

[6Yankee]

Making sure they're aware of the issue might be a good place to start.

Re:What do we call this?

[andreyvul]

Y2KX has a better ring to it.

I almost missed some important mail!

[darthwader]

"You might want to check your spam folder, as SpamAssassin has a rule ...

Thanks for the heads-up. There was a very important e-mail from the Internet Lottery people telling me my e-mail address had been picked as the winner of the EUR 20,000 prize. All I have to do is send them $200 by Western Union to cover the processing fees. And to think I almost missed it!

It's terrible that SpamAssassin flags such important messages as spam.

Re:"I'll just use a regex!"

[Hal The Computer]

Your solution doesn't work.
It fails on new years eve if someone is in a different time zone or if their clock is slightly off.

I'd suggest that any message sent more than seven (pick your favorite number) days in the future is spam.

Note from the VP, Apache SpamAssassin

[doshea]

Clearly we dropped the ball on this one. As far as I know it's our first big rule screw up in the project's 10 years. If you're going to screw up you might as well do it well.

I posted the following note to the Apache SpamAssassin website (http://spamassassin.apache.org/). Updates are available via sa-update, please run sa-update immediately. It's included in all versions of 3.2.x (the affected version of SpamAssassin). Alternatively zero the rule's score in your local.cf file if you have access to it. If you don't, increase your spam threshold by 3.6 points if your mail provider allows you to do that.

Y2K10 Rule Bug - Update Your Rules Now!

2010-01-01:

Versions of the FH_DATE_PAST_20XX rule released with versions of Apache SpamAssassin 3.2.0 thru 3.2.5 will trigger on most mail with a Date header that includes the year 2010 or later. The rule will add a score of up to 3.6 towards the spam classification of all email. You should take corrective action immediately; there are two easy ways to correct the problem:

* If your system is configured to use sa-update run sa-update now. An update is available that will correct the rule. No further action is necessary (other than restarting spamd or any service that uses SpamAssassin directly).

* Add "score FH_DATE_PAST_20XX 0" without the quotes to the end of your local.cf file to disable the rule.

If you require help updating your rules to correct this issue you are encouraged to ask for assistance on the Apache SpamAssassin Users' list. Users' mailing list info is here.

On behalf of the Apache SpamAssassin project I apologize for this error and the grief it may have caused you.

Regards,

Daryl C. W. O'Shea

VP, Apache SpamAssassin

Re:Note from the VP, Apache SpamAssassin

[Antiocheian]

Thanks. SpamAssassin has saved a lot of time on my life.

I was using Spamcop and got a bit zealous on reporting so spammers flooded my email address out of vengeance. It was then when I learned that reporting spam to admins only works in a few cases while the majority of spam is sent either through botnets or from countries with relaxed or no spam laws.

SpamAssassin removed 99% of the spam without ever giving me problems of blocking wanted messages...

Respectfully, thanks.

Re:Note from the VP, Apache SpamAssassin

[Old Sparky]

Working great over here - all I had to do was run sa-update.

Thanks for the hard work!

Re:Fixed in spamassassin 3.2.5-7 in Debian/Unstabl

[doshea]

It was an oversight. The rule fix got committed but never added to the update channel. Nobody noticed before it was too late.

Re:"I'll just use a regex!"

[jamesh]

Your solution doesn't work.
It fails on new years eve if someone is in a different time zone or if their clock is slightly off.

You need to talk to Microsoft - they could have used someone like you when they were building their 'stampinf' tool that I can't use until 10am in the morning (11am during DST) because it stamps UTC which then confuses another tool in the build chain...

Re:2010

[BronsCon]

First, it learned that 90% of mail is spam. Then, it got lazy. Now, it just stamps everything.

FX_DATE_20XX rule due to be obsoleted

[jonathan1979]

I think 2038 is a nonissue.

In this case it really is as this rule is due to be removed in future releases of SpamAssassin, for details see: https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6271