ClamAV Forced Upgrade Breaks Email Servers

An anonymous reader writes "A couple of weeks ago Sourcefire announced end-of-life for version 0.94 of its free ClamAV antivirus package (and in fact has been talking about it for six months). The method that Sourcefire chose to retire 0.94 was to shut down the server that provided its service. Those who had failed to upgrade are scrambling now. Many systems have no choice but to disable virus checking in order to continue to process email. I am very glad I saw the announcement last week!"

Comments

Alternative

[InsertWittyNameHere]

The alternative was them not doing anything and then months later we see a story about how "ClamAV silently stops support. Virus outbreaks ensue."

Re:Alternative

[Anonymous Coward]

It's kind of an inflammatory article:

Rather than simply phase this geriatric version out (it was at least one year old, revised to versions .95 and .96 since release, and announcements about the need to upgrade had been made for six months) the development team put to halt instances of V0.94 in production

So, it's a year and two versions out of date AND they'd been saying for 6 months to move off it.. Yet still it's their fault for shutting down the server!? I'm sorry, but how much support do you want for something that's free?

Re:Alternative

[compro01]

It's quite a bit more extreme than just shutting down one of their servers. They issued a final "signature" update that literally caused each installation of that version to stop functioning.

From the announcement :

Starting from 15 April 2010 our CVD will contain a special signature which disables all clamd installations older than 0.95 - that is to say older than 1 year.

Re:Alternative

[HarrySquatter]

Would you trust an email server that is running a virus scanner that is more than a year out of date?

Re:Alternative

[ccandreva]

It's more complicated than that.

Older versions of clamd were going to crash on signatures that newer versions would accept, and they have been prevented for at least 6 months from using that type of signature. They have posted since then for people to upgrade.

When they did was publish this type of signature (has to do with length, greater than about 900bytes), where the signature itself is an error message, so when the program dumped the signature the error would be displayed.

That's all, not a kill switch as such, but using a known bug to deliver a message, rather than have it just bomb out with a hex dump when they tried to use a larger signature.

Re:Alternative

[CoolQ]

Uh, it HAS been filling your log files with warnings about upgrading for months, if not years. It's pretty f'ing explicit:

LibClamAV Warning: *** This version of the ClamAV engine is outdated. ***
LibClamAV Warning: *** DON'T PANIC! Read http://www.clamav.net/support/faq ***

--Quentin

Re:Alternative

[Jiro]

It may not have occurred to you that some of us only do IT for out organizations part time, and visiting the blogs of every single open-source component on our servers is not always practical.

The issue has nothing to do with your servers, really; it has to do with their servers. If you're using a free service on someone else's servers, you really can't be surprised when that service suddenly stops functioning. It's not your equipment.

And I would wager that while visiting the blogs of everything on your servers isn't practical, visiting the blogs of (or subscribing to a mailing list, or other monitoring of) everything that's on your servers but uses someone else's servers is practical

So you had 6 months to upgrade

[gparent]

And you didn't, and now are going to complain when shit doesn't work? Go fuck yourself.

Re:So you had 6 months to upgrade

[johnshirley]

Kinda my attitude, too. Had this affect a bunch of servers yesterday. Started researching, found the cause, and solved the problem in 30 minutes on 35 or so servers. Totally my own damned fault for not staying upgraded. Worst impact was that messages were delayed on a few mail server for half an hour and uploads to a handful of webservers threw errors because of the way I scan them. Users tried again. Problem solved.

Re:So you had 6 months to upgrade

[Culture20]

I'm effected by endless clueless customers whining that their email server broke.

While such an occurrence would prompt me into action, I doubt it would prompt me into existence. ;)

Re:So you had 6 months to upgrade

[masdog]

I had modded this overrated, but this really deserves a reply.

You're in the wrong place if you expect sympathy. There are a lot of other sysadmins here. There are a lot who wear all of the hats. You're not alone.

You had a poorly designed or poorly implemented mail system. That isn't clamAV's fault. It's not their fault that you didn't upgrade or check your system logs. This is no different than forgetting to pay the maintenance bill on a commercial mail gateway or hosted solution. Would you blame Symantec, McAfee, Microsoft, or CA if you didn't pay the bill and your mail stopped flowing?

The fact that you didn't follow a blog or mailing list about a critical piece of your infrastructure says a lot about you as a sysadmin. They're even on Facebook and Twitter. If you can't take the time to keep an eye on your mail gateway or antivirus product, what else aren't you keeping up on. Think about that for a few minutes, set up a Google reader account, and then start subscribing to blogs. If you have a smartphone, add Google reader to your RSS Reader. It makes good bathroom reading.

Got This Bounce This Morning

[WrongSizeGlass]

Diagnostic-Code: smtp;
451-4.5.0 Error in processing, id=02792-02, virus_scan FAILED: virus_scan: ALL VIRUS SCANNERS FAILED: ClamAV-clamd av-scanner FAILED: CODE(0x83d7540) Too many retries to talk to /var/spool/amavisd/clamd.sock (Can't connect to UNIX socket /var/spool/amavisd/clamd.sock: No such file or directory) at (eval 55) line 310.

ClamAV-clamscan av-scanner FAILED: /usr/bin/clamscan unexpected exit 50, output="LibClamAV Error: cli_hex2str(): Malformed hexstring: This ClamAV version has reached End of Life! Please upgrade to version 0.95 or later.

At least their error messages are descriptive and informative.

[clamav-announce]

[0racle]

It exists for a reason.

Re:[clamav-announce]

[entrigant]

announce lists are intentionally very low traffic. I'm subscribed to over 50, and I rarely receive more than 4 or 5 mails a week at most.

this is common

[digitalsushi]

This is what we get when we're all our own "netadmins". I'm one of them. I don't follow security lists. I don't upgrade my products. Why not? Because I'm not really a netadmin. I just have a little server that runs until it breaks. I think that's the difference between a netadmin and a fake netadmin -- a fake netadmin like me reacts. A real netadmin is proactive.

Which honestly, as pathetic as it sounds on the surface, works fairly well when your data and uptime don't matter. Because it's not pathetic because I have better things to do with my time than "run the family webserver".

No fallback ?

[morcego]

People with critical servers that don't have fallback configurations to handle this kind of thing deserve to have their servers shutdown.

I've been using 0.95 for some time now, so none of my servers were affected but, even if they were, my servers are smart enough not to interrupt the services, and to notify me.

It is really disgusting the way people build servers these days. They think all they need to do is to install a couple packages, change a couple config lines and boom, the server is ready. They are getting what they asked for when stuff like this happens.

*Correction*

[Slipped_Disk]

The method SourceFire chose to use was to encode a kill command in the ClamAV updates. If they had simply "shut down the [update] server" ClamAV would have continued to work, just without new signatures.

See their announcement at http://www.clamav.net/lang/en/2009/10/05/eol-clamav-094/

Re:*Correction*

[Anonymous Coward]

Wow. They could have just stopped publishing updates for older versions; they do have some method of versioning, right?. Older installations could have kept chugging along using the older definitions and newer installations could get the newer definitions. But to remotely *DISABLE* older installations? I don't care if the product and service is free or not; that is pretty fucked up.

Re:*Correction*

[compro01]

The definitions were up to date (but would become out of date when they started pushing large (>980 bytes) definition updates next month, which the old version cannot handle), but the version was not.

Tisk, tisk...

[fuzzyfuzzyfungus]

Should have switched to Norton. They would have had weeks of impossible-to-ignore yellow and black pop-ups demanding their credit card number as ample warning...

Those freetards just don't understand the valuable features provided by quality proprietary software.

EOL annountment from Oct 2009

[Anonymous Coward]

End of Life Announcement: ClamAV 0.94.x
Oct 5, 2009

All ClamAV releases older than 0.95 are affected by a bug in freshclam which prevents incremental updates from working with signatures longer than 980 bytes.
You can find more details on this issue on our bugzilla (see bug #1395)

This move is needed to push more people to upgrade to 0.95 .
We would like to keep on supporting all old versions of our engine, but unfortunately this is no longer possible without causing a disservice to people running a recent release of ClamAV.
The traffic generated by a full CVD download, as opposed to an incremental update, cannot be sustained by our mirrors.

We plan to start releasing signatures which exceed the 980 bytes limit on May 2010.

We recommend that you always run the latest version of ClamAV to get optimal protection, reliability and performance.

Thanks for your cooperation!

Re:FUCK JEWS

[jDeepbeep]

FUCK JEWS

When they are exceedingly attractive, female, not married, and expressing interest, I do.

Re:It's not like they didn't tell...

[mysidia]

SUPPORT WILL END does not imply killing instances in production. It implies you stop delivering support services (such as tech support or new updates).

How would you feel if the Ubuntu folks delivered a 'security update' to Ubuntu 8.x to disable your system entirely, until you can get a chance to go install a non-EOL'd major release of your OS?

How about all those Windows Vista users who haven't upgraded to Windows 7?

Firefox 2 users who haven't upgraded to 3.

Users who are still using IE6.

Would users trust the vendors anymore with auto-updates, if they all released updates to 'kill the old product' in order to force you to manually do a clean upgrade?

Yes, they did the right thing...

[Slipped_Disk]

As someone who was bitten by the issue (yeah, I'll man up and admit it - my company's mail server went wonky for about a half hour while I upgraded) I agree -- they pretty much did the right thing.

There was plenty of notice -- The fact that many of us weren't on the clamav-announce list is OUR fault, not theirs.
A kill command may not be the most "polite" way of retiring an old version of software, but for a free service I certainly don't expect them to invest huge amounts of time and money in figuring out how to support the old stuff forever.

Debian Debs Outdated

[TypoNAM]

I just tried to update:

# cat /etc/debian_version
5.0.4

aptitude output during update:

Setting up clamav-daemon (0.94.dfsg.2-1lenny2) ...
Starting ClamAV daemon: clamd LibClamAV Warning:
LibClamAV Warning: *** This version of the ClamAV engine is outdated. ***
LibClamAV Warning: *** DON'T PANIC! Read http://www.clamav.net/support/faq ***
LibClamAV Warning:
LibClamAV Error: cli_hex2str(): Malformed hexstring: This ClamAV version has reached End of Life! Please upgrade to version 0.95 or later. For more information see www.clamav.net/eol-clamav-094 and www.clamav.net/download (length: 169)
LibClamAV Error: Problem parsing database at line 742
LibClamAV Error: Can't load daily.ndb: Malformed database
LibClamAV Error: cli_tgzload: Can't load daily.ndb
LibClamAV Error: Can't load /var/lib/clamav/daily.cld: Malformed database
ERROR: Malformed database

It appears debian repositories also need to be updated. :(

NOTE: I removed the * (star) chars from the warnings due to junk filter.

Natalie and grits

[jDeepbeep]

Be careful, though. Natalie Portman might pour hot grits on you.

Where do I sign up sir?