SpamAssassin 2010 Bug

SEWilco writes "You might want to check your spam folder, as SpamAssassin has a rule which is tending to mark email sent in 2010 as spam. There is some discussion in a bug report. The SpamAssassin Wiki FH_DATE_PAST_20XX page doesn't have discussion, but it was updated today with a different date rule."

Comments

First post to the bit bucket

[Shikaku]

This post has been removed due to spam.

2010

[Anonymous Coward]

Maybe SpamAssassin's AI is trying to tell us something... 2010, the year of spam?

Re:2010

[BronsCon]

First, it learned that 90% of mail is spam. Then, it got lazy. Now, it just stamps everything.

Millenium bug, how I have missed thee

[ickleberry]

Instead of having one millenium bug, letting it do it's thing and get it over with we get similar bugs every year. The 2007 zune issue, the impending 2038 issue and this. Of course many more similar bugs that don't deserve their own slashdot article. If they had decided to start using 64-bit time on the 1st of January, 1970 none of these problems would have happened

Re:Millenium bug, how I have missed thee

[fuzzyfuzzyfungus]

Given what memory cost in 1970, I suspect that using 64-bit time would have been an expensive decision.

A lot of gross little hacks look like (and are) great ideas when hardware costs a fortune and you don't yet know how persistent legacy stuff is going to be.

Re:Millenium bug, how I have missed thee

[JWSmythe]

    I'm actually thankful that they ran this one as a story. After seeing it, I remembered that rule. I just hopped in and fixed my servers.

    The better rule would have been to watch for dates more than X days from today. But, they're using regular expressions, so that makes it a bit harder.

Re:Millenium bug, how I have missed thee

[ls671]

> I'm actually thankful that they ran this one as a story.

Yep, not that /. is a security list, but it occurred to me a few times that I first learn about similar issues on /.

   

Re:Millenium bug, how I have missed thee

[Sparr0]

I think 2038 is a nonissue. Technological progress is accelerating. It took us 30 years to upgrade from 2 decimal digits to 4 for storing years. It will not take 30 years to upgrade from 32 bits to 256 bits for time.

Re:Millenium bug, how I have missed thee

[snaz555]

If they had decided to start using 64-bit time on the 1st of January, 1970 none of these problems would have happened

The pattern:

##{ FH_DATE_PAST_20XX
header FH_DATE_PAST_20XX Date =~ /20[1-9][0-9]/ [if-unset: 2006]
describe FH_DATE_PAST_20XX The date is grossly in the future.
##} FH_DATE_PAST_20XX

Will match 2010 no matter how many bits are used to keep time. This is a problem that has been know for years; it's a total embarrassment to OSS practices that it wasn't fixed before 1/1/10, before becoming critical. It's not based on a bad architectural decision, or even particularly bad code - it's just a typo where the fix wasn't pushed in a timely manner. I suspect a problem here is that open source development doesn't separate bug fixes from feature additions, which means non-critical fixes get backed up with features to be considered stable enough to ship. A critical fix can be spun separately (being backported to the current stable branch) - but this problem wasn't critical... yet...

Re:Millenium bug, how I have missed thee

[TheRaven64]

You're almost right, but actually you miss the point. Back in 1970, RAM cost around $1/byte, so using 32 bits instead of 64 saved you $4/record. However, that wasn't what caused the millennium bug, it was choosing a silly representation to use to store the date. If you wanted to, you could have stored dates as a 32-bit quantity in seconds since some fixed time and been able to handle any date in a 140 year period. You could also have performed calculations on these dates very quickly (just simple arithmetic) and not had to worry about things like leap seconds or different calendars (you only convert to calendar time for output).

Storing the date in decimal two digits only makes sense if you have a computer that supports BCD and not binary digits (e.g. an IBM 1620, although these were mostly decommissioned by 1970 and I can't think of any post-1970 architectures that made such silly decisions), in which case you're using 12 bits just for the year. You then need another 18 bits for the day (you could store the month and day-of-the-month as two 12-bit quantites, using 24 bits instead). You then need to store the time, either as a 24-bit (4 decimal digit) number of seconds or a 12-bit hour and a 12-bit minute and a 12-bit second (36 bits in total). If you use a natural human representation then you end up with 12 bits each for the year, month, day, hour, minute and second, giving 72 bits, or 40 more (costing $5 more) per date. Or maybe you only need to be accurate to the day, in which case it's only 4 bits ($0.50) more than you'd need for a 32-bit number of seconds.

By 1970, a lot of computers had 8-bit bytes and so used 4-bit BCD digits, meaning that your BCD year only took 48 bits, only costing $2 more than the binary version. If you'd only wanted dates to be accurate to the day, then it would have only been six BCD digits so would have been 24 bits, so $1 cheaper than a 32-bit time. However, if you only wanted that level of accuracy then storing a binary number of days in a 16-bit value would have given you a range of 179 years and saved $1 per date. Alternatively, a 24-bit number of hours would have made your code work for over a thousand years and cost the same amount.

In all cases, the binary version is cheaper, has a greater range, and is faster for computation. There was no excuse for using the BCD version. The reason for it was COBOL, which had DECIMAL types and encouraged their use. If the code had been written in FORTRAN instead, then this would not have been a problem.

Re:Millenium bug, how I have missed thee

[Anonymous Coward]

Yes, we absolutely need time support for 3671743063080802746815416825491118336290905145409708398004109081935347 years. Using 64 bits for time, would only give us support for 584942417355 years.

Re:Millenium bug, how I have missed thee

[Anonymous Coward]

Great comment bro.

Now actually read the fucking article, and explain to me how using 64 bit time since 1970 would have prevented this problem (somebody hardwiring 2010 into a config file at a time when 2010 seemed like far in the future, and not noticing it was no longer far in the future until we got here) from happening.

And this drivel is moderated insightful. Blah.

crapola

[DNS-and-BIND]

My provider runs spamassassin, and given their track record in updating their other software, I rather doubt that they'll update spamassassin anytime soon. Is there any way around this that doesn't involve root access? (I love helpful responses from idiots that start with "first, edit the /etc/spamassassin.conf file" or whatever.)

Oh yeah, the other wonderfully helpful stock response "stop using the software if you don't like it". Sure, I'd love to go back to getting 500 spams a day.

Re:crapola

[Shikaku]

Is there any way around this that doesn't involve root access?

No.

Re:crapola

[godrik]

I love helpful responses from idiots that start with "first, edit the /etc/spamassassin.conf file"

Yes. That's easy! First edit the /etc/spamassassin.conf file!
Wait a sec...

Re:crapola

[DaMattster]

Well, I am not sure what OS your running, but you can use OpenBSD Spamd which works very well. Rather than taking a defensive approach, Spamd goes on the offense by allowing known spam-sending IP addresses to attempt to send to you but throttling the connection down to 1 byte per second. This shakes most people off with no perceivable impact on your part. Even if the spam bot decides to wait the entire time to complete the connection, Spamd ends up dropping the message anyway. I use this solution in my business and I've gone from getting 500+ per day to maybe 2 per week. It is delightfully elegant.

Re:crapola

[ngc5194]

"Is there a way to work around this that doesn't involve root access?"

Yes, but it isn't a good way. Check your scores file for the scores associated with the FH_DATE_PAST_20XX. This indicates the number of points added to the spam score of every message that fails this test. Basically, increase your spam threshold by this amount until you can apply this patch.

Good for a quick-n-dirty fix.

Re:crapola

[karnal]

I think you missed the "My provider runs spamassassin" part of the parent post.

Re:crapola

[smartaleckkill]

depends--i have a cheap n cheerful shared hosting account with the same issue, but i do have cpanel access which allows me to override the score for any rule--check out the last link in the summary basically if you have access to local config files (even through a frontend like cpanel) you can do it without root access

Re:crapola

[darkpixel2k]

My provider runs spamassassin, and given their track record in updating their other software, I rather doubt that they'll update spamassassin anytime soon. Is there any way around this that doesn't involve root access? (I love helpful responses from idiots that start with "first, edit the /etc/spamassassin.conf file" or whatever.)

Oh yeah, the other wonderfully helpful stock response "stop using the software if you don't like it". Sure, I'd love to go back to getting 500 spams a day.

My car is so old it doesn't have an airbag. I heard about this glitch where I have a higher chance of dying because my car doesn't have an airbag. Given my track record in buying new cards, I doubt I'll update my gremlin anytime soon. Is there a way around this that doesn't involve require spending money? (I love helpful responses from idiots that start with "first go to NAPA" or whatever.) Oh yeah, the other wonderfully helpful stock response "stop driving the car if you're worried about dying in a wreck". Sure, I'd love to go back to the horse and buggy.

Re:crapola

[Anonymous Coward]

...my gremlin...

If you are/have contemplated/have ever driven/driving a Gremlin, then you should[need] to die a horrible death! Crash not required.

Re:crapola

[mhrivnak]

The new rule gets picked up when "sa-update" is run. spamassassin deployments should run sa-update automatically on a regular basis, for example every day via a cronjob. Thus, most deployments will pick up the update automatically tonight if a sysadmin doesn't do it first.

Re:crapola

[Mana Mana]

OpenBSD spamd(8) is wholly unrelated from spamassassin spamd. FYI.

OpenBSD spamd(8) has no code from any other project. Its similarity in appellation is name deep.

OpenBSD spamd(8) approach is different and was created by deraadt@.

Re:crapola

[Daniel_Staal]

Simplest non-root way (assuming they have it allowed): Edit the ~/.spamassassin/user_prefs file.

Re:crapola

[nabsltd]

My provider runs spamassassin, and given their track record in updating their other software, I rather doubt that they'll update spamassassin anytime soon. Is there any way around this that doesn't involve root access?

If you have shell access, it should be trivial, although you do have to edit a file.

Add the following to ~/.spamassassin/user_prefs:

score FH_DATE_PAST_20XX 0.0

Re:crapola

[Penguinshit]

Not to rub it in but I want to give a shout to my provider (Cruzio) who also use Spamassassin and were apparently on top of this (as they usualLY are0. i haven't noticed anything wrong with my email today. Thanks Cruzio!

Re:crapola

[Anonymous Coward]

Is there any way around this that doesn't involve root access? (I love helpful responses from idiots that start with "first, edit the /etc/spamassassin.conf file" or whatever.)

Well, that should be the /etc/mail/spamassassin/local.cf file. Local customizations shouldn't be in the main config file since the main config file will get overwritten when spamassassin is upgraded :)

But seriously, assuming your provider allows per-user customization, edit your ~/.spamassassin/user_prefs file, and add a line like this:

score FH_DATE_PAST_20XX 0 0 0 0

That sets the rule to add zero points to the spam score in all situations.

Re:crapola

[harlows_monkeys]

My provider runs spamassassin, and given their track record in updating their other software, I rather doubt that they'll update spamassassin anytime soon

If they are slow enough updating, it's possible you don't have the bug. I haven't bothered to update my home mail server past Ubuntu 6.06 LTS, since it works fine. The version of Spamassassin it uses, 3.1.7, does not appear to have this bug.

Re:crapola

[Anonymous Coward]

Sounds like a bad idea to me. Potentially too many open connections wasting memory, which is PRECIOUS and CRUCIAL on servers.

Re:crapola

[TheRaven64]

OpenBSD spamd(8) is wholly unrelated from spamassassin spamd

Not entirely. You can combine the two quite well. The spamdb tool can be configured quite easily to take input from SpamAssassin, so if someone sends you spam they then get blacklisted or more aggressively greylisted.

Re:crapola

[TheRaven64]

Spamd is very low overhead. It can tarpit a few thousand connections without you noticing and each of these connections is tying up a zombie and preventing it from hitting your smptd (which uses a lot more memory per connection). Spamd sets the window size to 1 and replies at one character per second, so it takes a few minutes for the SMTP handshake to complete. The University of Alberta saw a huge drop in their system load when they deployed spamd in front of their mail servers.

Re:crapola

[Sleepy]

> (I love helpful responses from idiots that start with "first, edit the /etc/spamassassin.conf file" or whatever.)

>Oh yeah, the other wonderfully helpful stock response "stop using the software if you don't like it". Sure, I'd love to go back to getting 500 spams a day.

"Idiots?"

You might have a point if the ONLY step began with "first". But here you see a bunch of people answered you.. and yes, ALL of them began with editing the "whatever" file.

You know... paid support for SpamAssassin IS available.
I suppose it could even work for someone who doesn't have TIME to "edit the /etc/spamassassin.conf file".

Even during the holidays there are folks like you who have the audacity to demand FREE free support... while pre-emptively demeaning those who would provide it. Unbelievable.

Re:crapola

[Anonymous Coward]

You really do have a hard time reading and comprehending what people post, don't you?

Re:crapola

[Anonymous Coward]

edit .spamassassin/user_prefs

or substract 3.6 from spam-score by an own filter program.

Re:crapola

[Pigskin-Referee]

Oh yeah, the other wonderfully helpful stock response "stop using the software if you don't like it". Sure, I'd love to go back to getting 500 spams a day.

Is the alternative of receiving no mail more palatable?

Re:crapola

[jonathan1979]

My provider runs spamassassin, and given their track record in updating their other software, I rather doubt that they'll update spamassassin anytime soon. Is there any way around this that doesn't involve root access?

Not that I know of, but perhaps you might have access to a local.cf file in which you can set the score of the rule to a lower value or ignore it by setting the score to 0. Before the update I did this (in my case) by adding the following line to /etc/mail/spamassassin/local.cf: score FH_DATE_PAST_20XX 0.0 The new rule however is not released in a new spamassassin release yet, but the rules are pushed through the udpate channels, so perhaps your provider might have received the update. Othwerwise it is wise to inform your provider (as this would affect all the users in a heavy way due to the high score assigned to this test) or switch provider.

"I'll just use a regex!"

[Inf0phreak]

What the... must all SpamAssassin rule be regexes? Because I can't see any (non brain dead) reason why not to implement this by just checking if $current_year < $header_year if that's not the case.

Re:"I'll just use a regex!"

[Anonymous Coward]

So what happens when an email server in say, San Francisco receieves email from someone in say, New Zealand where in NZ it's 2011 (New years) but still 2010 in SF?

Re:"I'll just use a regex!"

[Briareos]

Last I checked a timezone was mandatory in mail dates, allowing you to handle this correctly - for example by converting your current date and the date specified in the email to GMT...

Re:"I'll just use a regex!"

[ubrgeek]

It depends. If the email from San Francisco was traveling east going 400 miles per hour and the email from NZ is going west at .... ;)

Re:"I'll just use a regex!"

[Anonymous Coward]

You'll still run into the same problem when the sender's clock is just marginally wrong. What you should do is calculate the difference between the full sent date and the full current date, not just the year, and make the decision based on that. You won't be doing it with a regex though.

Re:"I'll just use a regex!"

[SanityInAnarchy]

I'm guessing the obvious solution is to make sure $current_year >= $header_year + 1

That would only hurt you when receiving mail from a machine with a clock wildly out of sync, and as all major OSes do NTP out of the box by default, I don't see how that's an excuse.

Re:"I'll just use a regex!"

[JWSmythe]

    I agree, that would be the better solution.

    I've seen lots of people's machines that have their dates wrong though. NTP is a wonderful thing, if it works. It's possible that corp firewalls can block unexpected traffic, including NTP though.

    I just got an embedded machine, which doesn't keep it's time right. I have to force an update at boot time, which makes the logs look real funny.

    There was a Y2K bug on some of my servers (like, a decade ago), so if they rebooted after Dec 31, 1999, the BIOS couldn't understand the date, and rolled it back to Jan 1, 1998. Why 1998? I don't know. Everything that happened until the time updated was timestamped wrong though. That wasn't a big deal, it was just entertaining to see the beginning of the log saying 1998, and as soon as it fixed the time, it would switch to 2000. And no, it didn't take 2 years for the machine to boot. :)

Re:"I'll just use a regex!"

[Anonymous Coward]

Why 1998? I don't know.

That's when the BIOS was configured. The current date can not be before the date the BIOS was configured, so that's the most recent date which can not be in the future. I prefer an obviously wrong date though, something like the UNIX epoch. Suppose you want to check if the system is running on default time: If your software runs on a system that is newer than your software, then there is no way to tell without reaching an external reference. If the system used a standard epoch as default time, then software could easily determine that the system has no current time information.

Re:"I'll just use a regex!"

[Bigjeff5]

...(like, a decade ago)...

Hopefully nobody had trouble inferring that, since it is now 2010, Y2k would have been ten years ago...

Re:"I'll just use a regex!"

[Anonymous Coward]

Great Scott! Now how am I going to email Marty from the year 2015?

I don't want to jump in the Delorian to go see him everytime, the fluxcapacitor is starting to wear out.

Dr Emmett Brown

Re:"I'll just use a regex!"

[JWSmythe]

    I agree totally.

    When I set up those servers, ntpd didn't cooperate very well for huge changes, it would just try to skew the clock. I wrote my own little program to check in with our local time server. Our time server would hit a real time server once a day, and the servers would all update hourly. In the case of those, I put it in at boot time, before any important services started that needed to know the time.

Re:"I'll just use a regex!"

[Hal The Computer]

Your solution doesn't work.
It fails on new years eve if someone is in a different time zone or if their clock is slightly off.

I'd suggest that any message sent more than seven (pick your favorite number) days in the future is spam.

Re:"I'll just use a regex!"

[jamesh]

Your solution doesn't work.
It fails on new years eve if someone is in a different time zone or if their clock is slightly off.

You need to talk to Microsoft - they could have used someone like you when they were building their 'stampinf' tool that I can't use until 10am in the morning (11am during DST) because it stamps UTC which then confuses another tool in the build chain...

Re:"I'll just use a regex!"

[jandoedel]

... or it could be an important message from your future self, containing instructions for making a time machine that send mails backwards in time.

Re:"I'll just use a regex!"

[Anonymous Coward]

ntpd didn't cooperate very well for huge changes, it would just try to skew the clock.

Right. ntpd is very careful to always move time forward and not at a crazy speed, so fixing the time shouldn't break anything. What you wanted to do was a one-time:
ntpdate pool.ntp.org
and *then* start ntpd.

Re:"I'll just use a regex!"

[JWSmythe]

    That's actually what I've been doing at boot time for several years now. When I did my own program, it was quick and dirty to run my own.

FIX details:

[drDugan]

this is also happening on Ubuntu server, running Spamassassin 3.2.5

The linked article references a workaround:
add this line to the "local.cf" spamassassin config file, on this system is was /etc/spamassassin/local.cf

score FH_DATE_PAST_20XX 0.0

If you're running spamassassin as a daemon, you *may* also want to restart spamd
with something like:

sudo /etc/init.d/spamassassin restart

This solution simply removes the rule by setting the score for that rule to 0.
You'll want to undo this once a solution is deployed.

Re:FIX details:

[KiloByte]

Since nearly 14 hours ago, you can simply run "sa-update".
It is in cron.daily in the default install, too.

Re:FIX details:

[nmb3000]

score FH_DATE_PAST_20XX 0.0

You'd probably be better off changing the rule instead of the score. Putting something like:

# Fixes bug: https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6269
header FH_DATE_PAST_20XX Date =~ /20[2-9][0-9]/ [if-unset: 2006]
describe FH_DATE_PAST_20XX The date is grossly in the future.

in your local SA config would probably be better (unless you really want to just drop the rule entirely).

That said, you'd probably be better off updating everything with sa-update, but this fix works in case you cannot or don't want to do that.

Re:FIX details:

[rduke15]

If you are running amavis, sa-update alone is not enough. You also need to restart amavis so that it picks up the new rules. On a Debian system, that would be something like:

sa-update && /etc/init.d/amavis restart

The spamassassin daily cron job does sa-update, and reloads spamassassin, but doesn't reload amavis.

What do we call this?

[Anonymous Coward]

Y2.01k?

Re:What do we call this?

[carlhaagen]

I suppose, yeah. It is after all just a minor revision.

Re:What do we call this?

[andreyvul]

Y2KX has a better ring to it.

One hack replaced by another

[tomp]

From the "fix"

> FH_DATE_PAST_20XX
> change '/20[1-9][0-9]/' to '/20[2-9][0-9]/'

That's no fix, it just puts the problem off for another 10 years. Why call the rule FH_DATE_PAST_20XX, shouldn't it be FH_DATE_PAST_201X? At least then the hack would be documented.

Re:One hack replaced by another

[JWSmythe]

    I just fixed a mail server for someone (after reading the article), and told them to remind me in 10 years to fix it again. :) Hopefully by then, they'll have rewritten the rule so it behaves better.

Re:One hack replaced by another

[Anonymous Coward]

my favorite response ever:

Not really much of a "fix" - more like a work-around that'll come back and bite
again in 10 years. "grossly in the future" is directly related to the current
time, so shouldn't this rule take the current time into account?

Comment 4 Henrik Krohns 2010-01-01 05:27:17 UTC

Right. But we have years of time to fix it.

Re:One hack replaced by another

[6Yankee]

Exactly. This is the right "fix" right now - the bug is out there causing real problems, and the fix (while definitely a filthy hack) is well-understood and can be pushed out immediately. If the same thing were to happen ten years from now (or the threshold quietly got pushed back to 2030), that would be nothing short of negligent.

Re:One hack replaced by another

[Anonymous Coward]

Yay, I've been mentioned on slashdot! May google archive my name forever!

Re:One hack replaced by another

[jamesh]

That's no fix, it just puts the problem off for another 10 years.

I had that thought too. Then I had another thought which was 'Meh'.

Re:One hack replaced by another

[techdoc70]

This won't help

Re:One hack replaced by another

[tendays]

Why call the rule FH_DATE_PAST_20XX, shouldn't it be FH_DATE_PAST_201X? At least then the hack would be documented.

Changing the rule name would break existing configuration files changing the rule score and/or description (like in that comment).

Re:One hack replaced by another

[Anonymous Coward]

You don't get it. The XX part is Roman.

Implicit whitelisting?

[YouDoNotWantToKnow]

That is so last decade ;P

Nice....

[kramer]

[url]https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5852[/url]

Noticed 14 months ago. Fixed 5 months ago. Released today.

Re:Nice....

[Bigjeff5]

Hey! Nice to see open source software gets fixed so ultra fast! :P

To be fair though, at least they released it the day it broke things, why didn't they release it by yesterday? Then the default cron job would have picked it up on most servers and nobody would have noticed.

Re:Nice....

[Anders]

Hey! Nice to see open source software gets fixed so ultra fast! :P

My sarcasm-o-meter is broken, so I cannot tell whether you are kidding. But it is indeed impressive to have a fix within hours, on a holiday.

To be fair though, at least they released it the day it broke things, why didn't they release it by yesterday? Then the default cron job would have picked it up on most servers and nobody would have noticed.

Why didn't they release it before someone noticed? Well, that's a good question. Maybe it was because nobody had noticed?

Re:Nice....

[Anonymous Coward]

Why didn't they release this when it was fixed.

Fixed in spamassassin 3.2.5-7 in Debian/Unstable

[John Hasler]

n/t

Aha!

[/dev/trash]

So this is why my Schwab Stock Alert was marked as Spam this morning.

Re:Fixed in spamassassin 3.2.5-7 in Debian/Unstabl

[Pretzalzz]

spamassassin:
    Installed: 3.2.5-6
    Candidate: 3.2.5-6
    Version table:
  *** 3.2.5-6 0
                701 http://ftp.us.debian.org/ testing/main Packages
                  70 http://ftp.us.debian.org/ unstable/main Packages
                100 /var/lib/dpkg/status

apt-get updated 30 seconds ago.

Re:Fixed in spamassassin 3.2.5-7 in Debian/Unstabl

[John Hasler]

toncho/~ sudo apt-get install spamassassin
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
    libmail-dkim-perl
Recommended packages:
    re2c
The following packages will be upgraded:
    spamassassin
1 upgraded, 0 newly installed, 0 to remove and 1086 not upgraded.
Need to get 1097kB of archives.
After this operation, 0B of additional disk space will be used.
Get:1 http://ftp.us.debian.org/ unstable/main spamassassin 3.2.5-7 [1097kB]
Fetched 1097kB in 13s (84.2kB/s)
Reading changelogs... Done
apt-listchanges: Mailing root: apt-listchanges: news for toncho.dhh.gt.org
(Reading database ... 163295 files and directories currently installed.)
Preparing to replace spamassassin 3.2.5-6 (using .../spamassassin_3.2.5-7_all.deb) ...
Stopping SpamAssassin Mail Filter Daemon: spamd.
Unpacking replacement spamassassin ...
Setting up spamassassin (3.2.5-7) ...
Starting SpamAssassin Mail Filter Daemon: spamd.

Here is the apt-listchanges message:

spamassassin (3.2.5-7) unstable; urgency=high

      This version of SpamAssassin fixes a bug which caused mails sent
      in 2010 to be flagged as suspiciously spammy. If upgrading to this
      version, you are recommended to update any per-user caches previously
      created by sa-compile, and to check mail already in your spam folder
      for false positives more carefully than usual.

  -- Joey Hess Fri, 01 Jan 2010 12:03:40 -0500

Now it shows up

[Anonymous Coward]

Ahh, the y2k.01 bug, naturally.

holy crap

[Anonymous Coward]

Thanks for the heads up, my kid's birthday party is next weekend and when I look at the spam folder it turns out 3 more people have replied that I hadn't seen.

Great workaround

[xororand]

The suggested fix is just silly... They postpone the problem to 2020-01-01:
3) change '/20[1-9][0-9]/' to '/20[2-9][0-9]/'

Re:Great workaround

[doshea]

We're likely going to drop the rule all-together in the future. In the interest of getting the fix out fast, with the least confusion of users, we decided to just make the change to 2020 for now.

Although... maybe a little visibility for the project every 10 years wouldn't hurt. :)

Re:Great workaround

[Anonymous Coward]

Have you ever done production operations before?
Goal 1: Get People Working Again.
Goal much-farther-down-the-list: Document, Debug, and get people on the issue to properly patch/ensure it never happens again.

Re:Great workaround

[Phroggy]

Ideally it should use an offset from the current date (which assumes the system clock is correct, but on a mail server it really REALLY should be).

Re:Bug report...

[Anonymous Coward]

Bill?

End User

[Kenshin]

As an end user, with no real ability to configure SpamAssassin other than through cPanel, what can I do about this? (Unless my webhost is right on top of it.)

I've disabled it, for now.

Re:End User

[Bigjeff5]

sa-update, assuming you can run that through cpanel.

Re:End User

[6Yankee]

Making sure they're aware of the issue might be a good place to start.

Re:End User

[doshea]

If you can't get the rule to stop firing (by changing the rule or zeroing the score) you could increase your spam threshold by the amount of the rule's score (anywhere from 2 to 3.6 or more depending on your config). This would essentially cancel out the rule.

Re:End User

[smartaleckkill]

as pointed out in a couple of places above, without root access you may still be able to edit your own user prefs for spamassassin, either by editing the /.spamassassin/user_prefs file, or through cpanel dpending on your provider's setup--in the configure options (where you can set required_score for example) you add the FH_DATE_PAST_20XX rule to one of the 'score' boxes with a value of 0--but note that this disables the rule entirely--i did it last night and my inbox was full of spam this morning, currently trying to figure out a +ve value/required_score combination that'll minimize the false +ve risk but still keep the spam down--by default the rule has a score of 3.4 i think, so with a threshold of 5 it might be ok to crank it back a little rather than zero it--you might still get some false +ves tho, i've disabled auto-delete meantime

Re:Fixed in spamassassin 3.2.5-7 in Debian/Unstabl

[Bigjeff5]

I know y'all are all proud about how fast it got fixed, but in reality it was discovered over a year ago, fixed half a year ago, and was only released today.

Why? Why not at least release it yesterday so most people's daily update would pick it up and make the whole thing invisible?

I almost missed some important mail!

[darthwader]

"You might want to check your spam folder, as SpamAssassin has a rule ...

Thanks for the heads-up. There was a very important e-mail from the Internet Lottery people telling me my e-mail address had been picked as the winner of the EUR 20,000 prize. All I have to do is send them $200 by Western Union to cover the processing fees. And to think I almost missed it!

It's terrible that SpamAssassin flags such important messages as spam.

Re:I almost missed some important mail!

[cdfh]

I hate it when I make a joke and I get modded "+5 insightful". Mod the stupid comments "funny", not "insightful", pleas

Moderators, please note: he's not being funny --- he's being insightful.

It would be 90% right!

[SgtXaos]

Only a small fraction of email I get is legit, so if it dumps all messages into the spam folder, it is pretty much doing the right thing

Shouldn't the real solution

[Anonymous Coward]

be something like
bad_date == date() + 1 yr
and then compare against bad_date ?

Note from the VP, Apache SpamAssassin

[doshea]

Clearly we dropped the ball on this one. As far as I know it's our first big rule screw up in the project's 10 years. If you're going to screw up you might as well do it well.

I posted the following note to the Apache SpamAssassin website (http://spamassassin.apache.org/). Updates are available via sa-update, please run sa-update immediately. It's included in all versions of 3.2.x (the affected version of SpamAssassin). Alternatively zero the rule's score in your local.cf file if you have access to it. If you don't, increase your spam threshold by 3.6 points if your mail provider allows you to do that.

Y2K10 Rule Bug - Update Your Rules Now!

2010-01-01:

Versions of the FH_DATE_PAST_20XX rule released with versions of Apache SpamAssassin 3.2.0 thru 3.2.5 will trigger on most mail with a Date header that includes the year 2010 or later. The rule will add a score of up to 3.6 towards the spam classification of all email. You should take corrective action immediately; there are two easy ways to correct the problem:

* If your system is configured to use sa-update run sa-update now. An update is available that will correct the rule. No further action is necessary (other than restarting spamd or any service that uses SpamAssassin directly).

* Add "score FH_DATE_PAST_20XX 0" without the quotes to the end of your local.cf file to disable the rule.

If you require help updating your rules to correct this issue you are encouraged to ask for assistance on the Apache SpamAssassin Users' list. Users' mailing list info is here.

On behalf of the Apache SpamAssassin project I apologize for this error and the grief it may have caused you.

Regards,

Daryl C. W. O'Shea

VP, Apache SpamAssassin

Re:Note from the VP, Apache SpamAssassin

[Antiocheian]

Thanks. SpamAssassin has saved a lot of time on my life.

I was using Spamcop and got a bit zealous on reporting so spammers flooded my email address out of vengeance. It was then when I learned that reporting spam to admins only works in a few cases while the majority of spam is sent either through botnets or from countries with relaxed or no spam laws.

SpamAssassin removed 99% of the spam without ever giving me problems of blocking wanted messages...

Respectfully, thanks.

Re:Note from the VP, Apache SpamAssassin

[Old Sparky]

Working great over here - all I had to do was run sa-update.

Thanks for the hard work!

Re:Fixed in spamassassin 3.2.5-7 in Debian/Unstabl

[doshea]

It was an oversight. The rule fix got committed but never added to the update channel. Nobody noticed before it was too late.

Why does this rule exist anyway?

[Animaether]

I can only guess that it's a rule to flag e-mails as being spammy (all of my new years' wishes to brits and americans I got replies to with the subject tagged "{Spam?}") when they are 'sent from the future'.
    I can also only guess that this is done via a regex on the date with the date being somewhere far into the future (but that future kinda crept up on us the other day, right?) so as not to burden CPUs with actual date conversions to universal time and doing comparisons on that, and flagging e-mails sent greater than a 24 hour period or so into the future.

But given that
    1. such e-mails hardly exist in the first place (In my archives, I just found 2*.. out of 19,836 and counting)
    2. spammers will be readily aware of this rule and really why would they make an effort to inject a future date anyway when their mail daemons will happily use the -current date-
    3. in case of an actual entity from the future informing us of impending doom with instructions on how to stave it off, we'll have seriously fucked ourselves?

I just don't see why this rule exists at all, and why it needs to be 'fixed'. Maybe somebody can inform me of a very good reason for the rule existing. If not: I say the rule doesn't need fixing.. it needs removing.

*
E-mail number 1: A travel itinerary auto-sent from expedia back in May 2004. It has a date of February 7th, 2101.
E-mail number 2: An e-mail from a friend back in June 2005. It inexplicably has a date of June 26th, 2165. ThunderBird oddly enough reports a date in the search results reading May 21st, 2029 - though that's nowhere to be found in the mail's source. Display bug?
Neither are spam, regardless. (yes, I know, there's no telling how many (spam) e-mails from the future I never received because they were spam-flagged in the first place. I think I'd rather take my chances.)

Re:Why does this rule exist anyway?

[Anders]

2. spammers will be readily aware of this rule and really why would they make an effort to inject a future date anyway when their mail daemons will happily use the -current date-

They will use dates from the future to have their offerings shown at the top of the mailbox.

Re:Why does this rule exist anyway?

[mikael_j]

Then there's the other date-mangling tactic, setting the date weeks or months in the past, I'm guessing they're doing this to make recepients think that the spam is some old legit mail they forgot about, it's annoying when one of those slip through though...

/Mikael

Re:Fixed in spamassassin 3.2.5-7 in Debian/Unstabl

[John Hasler]

> I know y'all are all proud about how fast it got fixed...

Where do you get that? I posted to inform people running Debian that they need not apply the patch. That's all. I expressed no opinion about the speed of the fix.

2K10 is not 2010

[Ivan Stepaniuk]

It is 2100... even if you don't like how 2k01 sounds.

Re:2K10 is not 2010

[jlehtira]

No. 2K10 reads two thousand and ten, and means just that.

Re:2K10 is not 2010

[CyberDragon777]

Unless you are used to the electronics way:

        1K2 = 1.2 k = 1,200
        M47 = 0.47 M = 470,000
        68R = 68

Fine grain rule to stop anything after 2014

[freaker_TuC]

This rule is a little bit more fine-grained. It will not allow anything to be sent after 2014.

edit the file /usr/share/spamassassin/72_active.cf

##{ FH_DATE_PAST_20XX
header FH_DATE_PAST_20XX Date =~ /20[1-9][4-9]/ [if-unset: 2006]
describe FH_DATE_PAST_20XX The date is grossly in the future.
##} FH_DATE_PAST_20XX

I wish variables could be used in regexps like these, like $year + 2 would be a nice start....

Re:Fine grain rule to stop anything after 2014

[doshea]

I recommend against the above suggestion for a number of reasons:

- /usr/share/spamassassin is not the place to edit your rules. Edit your local.cf file.

- We will be dropping this rule or changing it to a non-regex rule that actually takes the current date into account. We'll do whichever our statistical data suggests would be most beneficial. Once we've done this, we'll push another update via sa-update. I wouldn't risk forgetting that you've modified a rule that's going to hit you again down the road.

BTW, the suggested rule is only going to hit on years 2014-2019, 2024-2029, etc...

Re:Fine grain rule to stop anything after 2014

[freaker_TuC]

Actually a better rule taking that in account is:

##{ FH_DATE_PAST_20XX
header FH_DATE_PAST_20XX Date =~ /20(1[4-9]|[2-9][0-9])/ [if-unset: 2006]
describe FH_DATE_PAST_20XX The date is grossly in the future.
##} FH_DATE_PAST_20XX

I've made a post entry about this on my blog at http://development.artistblog.me/2010/01/spamassassin-2010-bugfix/ ; although I've heard cases where sa-update didn't work, so I wouldn't know which way would be the best since there are more ways to skin a cat ...

Many thanks to the poster!

[admiraljayce]

Thanks for the timely tip (no pun intended) :-)

FX_DATE_20XX rule due to be obsoleted

[jonathan1979]

I think 2038 is a nonissue.

In this case it really is as this rule is due to be removed in future releases of SpamAssassin, for details see: https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6271