Slashdot Mirror
Many Antivirus Tools Fail in LinuxWorld Test
talkinsecurity writes "In a public, side-by-side test conducted last night at LinuxWorld, ten antivirus products were confronted with 25 known viruses. The results were surprisingly disparate. Only three of the products caught all of the viruses; three only caught 61 percent, and one caught an abysmal 6 percent. The test, which wasn't particularly complicated, proves that there still are wide differences in the effectiveness of AV tools. A lot of people think all AV tools are the same — they're not!"
From TFA:
Kaspersky, Symantec, and Clam AV: 100% caught
FProt and Sophos: 94%
McAfee: 89%
GlobalHauri, Fortinet, and SonicWall: 61%
WatchGuard's Linux AV: 6%
And a graph of the results plus links to some of the test viruses: http://virus.untangle.com/
I know they can pick up alot of information when you connect to their website, IP, location, but hair pointiness?
are viruses on linux a overflow from WINE?
Not much here.
The story could have shown a list of the tested viruses verses the AV software being tested. A simple table would have conveyed a great deal more information than the druel the fellow wrote. Yes I RTFA and as I said - it is not very informative.
What about AVG? I really love it. I've installed on both my workstations and a server (Windows). It uses minimal resources, it's fast, and it's managed to catch more stuff then Trend Micro, Symantec and McAfee.
Also, Bitdefender and Nod32 are also good for the Windows enviroment. I'm curious to how all these ranked in the Linux world.
Life is not for the lazy.
How does i/25 not equal 4*i%? Were some of the 25 viruses half-caught, or one-quarter caught?
Something seems a little strange here. With 25 test cases, and a binary outcome (either the virus was detected or it was not), the %caught should proceed in even step of 4%. There's some number massaging going on somewhere.
Hmm... the Fight Club Website lists 35 test cases, not 25. It's not clear if there is any overlap between the various test cases. In fact, there's not any discussion of the testing methodology (let alone what precisely was tested) at all. Just "here's our numbers - believe them or infect your own machine and find out for yourself".
Now, while I admire the 'do it yourself' hacker ethos as much as the next guy - this is taking it a bit too far.
For fun I downloaded an application where I suspected the "keygen" was trojanized. I was correct; the real keygen had been bundled with some, as it would turn out, Off The Shelf trojan. However, I didn't know what trojan so I scanned with F-Secure's online-engine, which didn't detect anything (neither did my active AVG installation). So I sent in the exectuable as a sample, explained what little I had to say; where I found the file, that it was pecompact2'ed, that their online scan didn't detect it. The process of submitting a file req. you to attach the scanner log.
Got the reply that "The file you submitted was found to be malicious, and is already detected as Trojan-Downloader.Win32.Delf.asz using the latest virus definitions." and "Please update your virus definition databases to properly detect the file".
Remember, I had scanned it using their latest online scanner and provided the log where the trojan was NOT detected.
So, maybe an extra warning for online scanning engines.
PS.
Shortly after I had submitted the file to f-prot, AVG started detecting it.
Belief is the currency of delusion.
Nice to see opensource programs perform so well, so consistently. I only wish the author(s) maintained the ports and packages himself. The Win32 port seems a bit of an afterthought. Anyway, still a brilliant antivirus program.
(My other OS favourites include Audacity, CDex, The GIMP and OpenSolaris (you didn't expect that one coming, did you)).
"The agriculture ministry is not in charge of Gundam" - Japanese ministry official.
It is understandable why there are so many zombies out here spewing spam 24 hours a day. Nobody has a clean machine and there is no way to obtain one without reformatting the hard drive and reinstalling the minimum.
I'm fairly knowledgable about home computers (I bought my first one in 1976) and I have a weird feeling in my gut that there is something on this computer that shouldn't be there. But all of the tools I've tried (antivirus, antispyware, etc.) have found nothing wrong.
I coined a word a while back: filthify, v., to give a computer access to the Internet.
Fata viam invenient.
I assume the virus software was running on Linux but the viruses being detected were Windows viruses. You might want this type of virus software running on a Linux mail server or Samba server so Windows machines can't spread their viruses to other Windows machines through you. Of course we know they couldn't have come up with 25 Linux viruses, or even 1 for that matter.
Last I checked, OneCare did not run on Linux.
We use Sophos on our Linux mail relays and Trend on the desktops, servers and web proxy. We've only had one small virus outbreak in 15 months. I guess Trend isn't covered since there is no Linux client, but it is in the top bracket on every shootout I have seen in the last couple years.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
There were 25 viruses. How does something catch six percent? Eight or four, sure. But six?
...considering that most of the antivirus programs were tricked when a new "variant" of one of the worms back around '99 or so. So kids- just insert random whitespace into your worms!
The change? The line endings in the VBS script changed. It probably wasn't even intentional- some broken mail server probably modified CR's into CRLF's. It sailed right past Trend Micro's email scanner and infected several dozen systems.
I was the first person to notice why it slipped by, and brought it to the attention of a big-name "security expert" who ran a mailing list which shall go unnamed. He thanked us for the research, passed along my findings to the list, and then promptly went around doing interviews with the press using the first person voice. "I discovered that...", blah blah was what I read the next day.
Please help metamoderate.
Let me preface this by saying that I work in a Windows free environment. I understand that not everyone has this luxury.
Am I a bad citizen because I don't scan for Windows viruses on my Linux systems? It's almost like another Microsoft tax--you're expected to degrade your performance to prevent their victims, uh, customers (yeah, that's it) from infecting each other. Those folks need to be responsible for their own safety and not expect the rest of us to do it for them. They could start by holding Microsoft accountable and making other choices at purchasing time. To me, Windows isn't worth the hassle.
Interesting that SonicWALL only caught 61% compared to McAfee catching 89%. The virus protection on our SonicWALL at work is powered by McAfee.
The charts used those damned ClearType sub-pixelation fonts in the image, which is not going to work right with many monitors since they have to be tuned per user. When I see that rainbowy tinge, at first I check to make sure I haven't drank too much c c c coffee again.
Table-ized A.I.
Not to knock Clam but there is something odd about these results (Besides the absurdly low testbed). TFA says Clam won two years ago (which meant Untangle would use it), and again now. However, just last May the results from AV-Test.org (a real trusted legitimate source) against a comprehensive testbed put ClamAV near the bottom of the heap: http://www.pcmag.com/article2/0,1895,2135053,00.as p
I can't help but think that Untangle is trying to justify their own choice, rather than have a real test. With a testbed of only 25-35, it is possible to pick a group of malware that can put any AV on top. Even the user submitted malware is suspect, especially when that testset is also so low. ClamAV is great against virus outbreaks, with one of the fastest signature responses, but it has pretty atrocious trojan and zoo detection, since there is not enough man-power to collect and create signatures for less prevalent and non-replicating malware.
You mean corporate shills lie and cheat to get rich?
That would mean that it's performing just as well as it does in Windows. Good work Microsoft!
For the Excel-averse, I have uploaded the Excel Results of the test to the Zoho Viewer website. So you needn't install Excel or OO. http://viewer.zoho.com/docs/edblaI
I'm not sure I'd rely *only* on ClamAV for protecting incoming mail on my mail servers. But if you can hookup a way to check incoming mail against multiple AV providers, then definitely throw ClamAV into the mix. It's free and it works...
- Matt
The real question is, how many of these virus scanners detected and quarantined windows?
All joking aside, isn't it strange than with all of Microsoft's monopolistic tendencies, they haven't branched into the anti-virus market yet?
I recognize that this would be a paradox, but still....
All of them depend on guessing whether a file is good or bad.
All of them will have false negatives as well as false positives, most likely skewed to have fewer false positives to reduce the annoyance factor at the expense of missing real viruses - false negatives.
There are substantially better and computationally cheaper ways to protect your system than an anti-virus.
Because I see 1) unfreeze 2) installed warez 3) refreeze 4) zombie. It's a great idea if you have a really good working understanding of an operating system (although I've seen some pretty tricky virii/malware) but for your regular users this is complicated and confusing. In fact I would say it would probably be easier to train a user to use an unprivileged account (and we all know how well that's gone).
DeepFreeze is an excellent tool for administrators or powerusers. But certainly no silver bullet.
Quack, quack.
Something you get if you go online. Remember, you may not be infected by a virus, but you can still spread it. Signed, Computer User
GlobalHauri? Fortinet? Where are NOD32 and BitDefender?
I'd rather see commercially available AV tests, since that's what 99.9% of consumers use. I can (and have!) not use an AV scanner for 4 or 5 years and never see a virus, because I pay attention. How about Jimmy Bob Johnson who visits every porn and keygen site on the internet, but uses McAfee because his ISP bundles it?
This signature does not exist. It has never existed. It is all a figment of your imagination.
See, I thought that until I tried to download the sample file and open it. Then I noticed that the sample was 7.5MB in size, and that there were multiple instances of some viruses. The weird numbers come from having a partial success rate in capturing Virus X. That is... if I send 12 instances of MyDoom.M at a virus scanner, and it only catches 7 of them.
If you believe everything you read, you'd better not read. - Japanese proverb
Come on, an MCSE would expect those results.
Seriously, with a title like "Many Antivirus Tools Fail in LinuxWorld Test" you would expect something new. Well, I guess I was surprised. I didn't think Symantec had it in them. Kudos to them. ClamAV, no surprise there at all. Same goes for Kaspersky. You could've figured that out by using Google.
A thing you may receive in the near future, unless Apple finally gets around to implementing better handling of "safe" file types and Apple Mail attachments in 10.5...
One product, WatchGuard's Linux AV tool, caught fewer than 6 percent of the viruses sent to it. "We're not exactly sure what the problem with WatchGuard is," says Morris. "The test was set up the same way for all of the vendors."
:). My question would be which is it? Either way, my system would be compromised by either 24 or 25 viruses -- neither of which is a good scenario especially in regards to well-known viruses (according to the article no 0-day exploits were accepted).
This number quoted by the original poster missed the section in bold, it was technically < 6%, which could mean either 0 or 1 virus (funny how everything always works out to binary in some way or another
I think their counting frame has a cracked bead...
Excuse me, but please get off my Pennisetum Clandestinum, eh!
If you "catch a virus", you're infected.
"where's geoff today?",
"oh, he caught the flu"
"he caught it! nice one geoff, you managed to destroy that pesky flu & not get infected - so he's out celebrating right?"
"erm... fk off weirdo"
* Game Over * High Score: 264,846,927 -- Your Score: 14
Server is going so slow right now that I gave up on the download til the crowd thins, so don't know if this is what's going on, but:
Compressed executables can render the virus invisible. Different compression schemes can alter scanning results. So it behooves one to test both an uncompressed executable, and the same virus in a variety of compressed formats.
Incidentally, this problem was why I stopped using McAfee's DOS scanner, back about 1994: Every time its engine was updated, I'd test it against my zoo. One day the update was unable to find a common virus in compressed form, tho it could see it uncompressed. F-Prot saw them both. I switched, and never went back.
~REZ~ #43301. Who'd fake being me anyway?
Finally, someone knows why the value is not a multiple of 4!
:)
Why am I not surprised the summary had errors?
My guess is that it caught two, but was only able to remove or prevent one from infecting the machine. If it successfully caught and removed one, it would give it 4%, and if it caught (noticed) a second but was unable to do anything about it, it might have gotten a point or two for noticing it and being able to alert the user to something being wrong, but did not get full points since it could not remove or prevent an infection from the second.
That's just thoughts, though; I am too tired to read the article in-depth, but considering how people are responding, I have a feeling that this sort of thing isn't mentioned in it.
Calling a sword by a pretty name is no more than adding perfume to poison.
Given any set of 25 viruses, each virus represents 4 percent. So one antivirus caught a virus and a half?
Visit http://ringbreak.dnd.utwente.nl/~mrjb/growingbettersoftware to download your free copy of the book
Once a virus is in the machine it can do whatever it likes, including hiding itself from your antivirus. I've personally disinfected dozens of machines which have Norton+a virus.
The answer is usually to reboot in safe mode and scan from there.
PS: I use AVG. Norton is just too intrusive, bloated and causes too many problems with normal system operation.
No sig today...
What's a Mac?
Gentoo Linux - another day, another USE flag.
"Either way, my system would be compromised by either 24 or 25 viruses..."
24 or 25 out of 25?
Hmmm....
Does mean that *nix is finally ready for the desktop?..Just like Windows?
Uhmm..w00t!?!?
Disclaimer: coming to you from a Feisty Kubuntu PC that is running ClamAV.
Down With Slashdot BETA!!! I've been around the corner and seen the oliphant; you can only abuse me from your perspecti
I run a home network back home that has 1 OpenBSD server, 1 Linux server, 4 Linux laptops/desktops and 1 Mac. Precisely how are any of these "anti-virus" tools relevant to me ?
This 2001 Qnetic report for the UK gov.
http://www.govtalk.gov.uk/documents/QinetiQ_OSS_r
Makes this claim : "There are about 60,000 viruses known for Windows, 40 or so for the Macintosh, about 5 for
commercial Unix versions and perhaps 40 for Linux."
But viruses, by definition, will always have a hard time in Lunix. People generally don't share executables. Which leaves auto-opening files such as image preview, pdf, html and openoffice docs etc.etc.
It generally easier to exploit internet facing services such as DNS, HTTP, SMTP etc.
There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
Extracted the zip to a directory and let OfficeScan (8.0, Scan engine v8.500.1002, virus pattern 4.641.00) scan and quarantine all viruses. These files remained:
111_xxx.com :01:24)
112_untangle1.zip
114_untanged22.zip-(07-08_08_01 -- empty file, not extracted, ends in
115_untangle3.zip
116_untangle4.zip
I couldn't ignore the anal-retentive troll inside of me.
which could mean either 0 or 1 virus (funny how everything always works out to binary in some way or another :).
That is not binary, but rather only could be binary, but could be any m-ary. True, it could be binary, if you assume two viruses would be represented by 01, three by 11, four by 001, and so forth. As it is, it's ambiguous, as are all numbers. 234 viruses could be decimal, hexadecimal, or a higher base, just as X amount of something does not denote the actual base. Now, if there was a subscript "1," that could mean it was binary, but that's obviously not there, now is it, hmm.
On another tangent, I have seen a similar analog-digital converter in a PIC program quite a few years back. Basically, if I had an analog value that I knew were only going to be 0 or 1, I could convert it to a digital 0 or 1. For some reason the label of the value mattered more than the actual value in the application. What a fun program in school. I was able to use the free 15-step limited program to do what I wanted, while everyone else had to resort to some ungodly large amount of logic that required the paid program.
I use Watchguard all the time and nothing has ever gone wr&,;*..}..Get 3 months of Viagra free with our low mortgage rate offer now now now!
Table-ized A.I.
I checked their test set with NOD32. From 34 files NOD32 didn't find 4 (88% success). The files uncaught were 112_untangle1.zip, 114_untangle22.zip-(07-08-08_01, 115_untangle3.zip and 116_untangle4.zip - they're custom viruses. Personally I wouldn't worry about it because NOD32 releases definitions really fast. Since I don't surf infectious sites, my chances to cacth a new, yet unknown virus are minimal.
How come they didn't test F-secure, which was the first to detect the Sony rootkit, even before Mark Russinovich wrote his article? I once scanned my usenet inbox in linux using clamscan with the latest definitions. It found nothing. Then I dual-booted to windows and scanned the same folder with f-secure, and it found plenty of win32 trojans.
Is viruses can be a bitch to remove when the system is online, since the virus can do things to fight the scanner. I see a scanner running on a lice system as preemption, not recovery. You run it to stop the virus before it can cause harm. AVG seems good at that, it seems to notice viruses right away.
If you want to use a tool like that for recovery, they way to do it is on an offline system. Either take the disk to another computer and set it up as a non-system disk, or build yourself a PE boot disc and clean it from that.
It more or less the same for any sort of system analysis or recovery for malware, hacks, whatever. Running tools on the live system is of limited use since you might get back bogus answers. You can run them to see what it going on, but when you actually start cleaning up, you need to do it from a different system, or there may be something working to undo what you've done.
It found 0 viruses - are they sure they configured it correctly? Maybe it found no viruses because it found no files!
Just that some are better at smaller factors of immunity.... ;) It's okay, from what I hear, Vista is almost as good as Freedos now!
Semi-automatic amateur armchair Australian philosopher; conjecture ready at any moment...
I have been using avast for the past couple of years and as far as I can see it's done really well...however I guess that might not be a good sign! Does anybody know how it compares to for example norton?
No kitty, this is my pot pie!
Just like intaveinous drug using transvestite prostitutes, password protected zip files should be assumed to be virus positive. I've never encountered one instance of them serving a legitimate purpose, for privacy there's GPG and friends.
It's measured in typos. And the number of reboots it takes you to fill out a web form.
We have to use password protected zip files occasionally because most of the world is clueless about GPG.
Nice try, but AFAIK ClamAV doesn't clean viruses, it just is able to quarantine or delete the files. Cleanup is mostly a wasted effort in a time when you aren't trying to catch old school viruses attached to legitimate files but mere self-contained worms than you can simply send to the bit bucket.
Got Pike?
I want to preface that I run a BSD only network (OpenBSD on my router and FreeBSD on my desktop) so I have no need anymore for these applications.
One thing that most consumers don't realize is that the whole AV industry is basically a big scam. If software was designed more properly, malware would be less of an issue and AV software would be needed less (educating consumers is still an important aspect). In addition to this, the AV software itself would be better if they collaborated. The whole industry is setup to only benefit the companies and not really the consumers though. If they really tried to do the best for the customer, the organizations would collaborate to offer the best coverage (standardized definitions?) and therefore only the best software would make it on the market. Instead companies like Microsoft are in bed with the likes of Symantec/McAfee/Etc and have this whole industry setup to milk the consumer rather than direct that money to R&D and real advancement of technology.
Any thoughts?
Confucius say: "Man who associates with smarter men than himself is smarter than the men he associates with."
I agree about AVG why wasnt it tested. I also use FSecure with some clients and have been very happy with it. Would have been nice to see it in the running as well. Anyone have any experiance with FSecure?
Should have posted this too:
Time Module Object Name Threat Action User Information
8/10/2007 9:51:03 AM AMON file C:\Temp\7zE1C1C.tmp\111_xxx.com Win32/PSW.Lineage.NGI trojan quarantined - deleted
8/10/2007 9:51:03 AM AMON file C:\Temp\7zE1C1C.tmp\110_vvzh.scr Win32/Mydoom.R worm quarantined - deleted
8/10/2007 9:51:02 AM AMON file C:\Temp\7zE1C1C.tmp\109_virus_88.bin Win32/TrojanDownloader.Agent.BRK trojan quarantined - deleted
8/10/2007 9:51:02 AM AMON file C:\Temp\7zE1C1C.tmp\108_virus_87.bin Win32/TrojanDownloader.Agent.NQG trojan quarantined - deleted
8/10/2007 9:51:01 AM AMON file C:\Temp\7zE1C1C.tmp\106_Info.exe Win32/Bagle.X worm quarantined - deleted
8/10/2007 9:51:01 AM AMON file C:\Temp\7zE1C1C.tmp\105_image.jpg.exe Win32/TrojanDropper.Small.UU trojan quarantined - deleted
8/10/2007 9:51:01 AM AMON file C:\Temp\7zE1C1C.tmp\104_Attachment.scr Win32/Mydoom.R worm quarantined - deleted
8/10/2007 9:51:00 AM AMON file C:\Temp\7zE1C1C.tmp\020_test.zip Win32/Mytob.AE worm quarantined - deleted
8/10/2007 9:51:00 AM AMON file C:\Temp\7zE1C1C.tmp\019_scan_check.jpg.exe Win32/Spy.Goldun.S trojan quarantined - deleted
8/10/2007 9:50:59 AM AMON file C:\Temp\7zE1C1C.tmp\018_q347558.exe Win32/Swen.A worm quarantined - deleted
8/10/2007 9:50:59 AM AMON file C:\Temp\7zE1C1C.tmp\101_scan.jpg a variant of Win32/Spy.Goldun trojan quarantined - deleted
8/10/2007 9:50:58 AM AMON file C:\Temp\7zE1C1C.tmp\017_photo.pif Win32/Netsky.Q worm quarantined - deleted
8/10/2007 9:50:58 AM AMON file C:\Temp\7zE1C1C.tmp\014_message.pif Win32/Mytob.D worm quarantined - deleted
8/10/2007 9:50:57 AM AMON file C:\Temp\7zE1C1C.tmp\013_image.jpg.exe Win32/TrojanDropper.Small.UU trojan quarantined - deleted
8/10/2007 9:50:57 AM AMON file C:\Temp\7zE1C1C.tmp\015_mntrup.exe a variant of Win32/TrojanDownloader.Delmed trojan quarantined - deleted
8/10/2007 9:50:56 AM AMON file C:\Temp\7zE1C1C.tmp\012_fullstory.exe Win32/Fuclip.A trojan quarantined - deleted
8/10/2007 9:50:44 AM AMON file C:\Temp\7zE1C1C.tmp\011_abuselist.zip Win32/Netsky.Q worm quarantined - deleted
8/10/2007 9:50:33 AM AMON file C:\Temp\7zE1C1C.tmp\010_18_04_2005.exe Win32/TrojanDownloader.Small.ZL trojan quarantined - deleted
8/10/2007 9:50:26 AM AMON file C:\Temp\7zE1C1C.tmp\000_eicar.com Eicar test file quarantined - deleted
I filtered out user, machine name, and full description of action (offender and long description of action taken). You get the gist, and you can repeat it yourself of course.
It is impossible to catch 6% of 25 items. One is 4%, two catches is 8%.
:{
ug!
I was thinking the same thing about the less than bit, but other numbers that appear without such qualifiers in the article are 94%, 89%, and 61%. So, the '25 viruses' is the part that is inaccurate(per other posts).
I run Linux.
That is all.
The test they ran is completely meaningless from a statistical viewpoint and was almost certainly skewed in some way by the organizers, despite the submission of viruses to be tested from the audience (shills? Not impossible!)
Despite the claims of the company that conducted the test, ClamAV HAS been tested by several AV testing outfits, and it came up poorly in all of them. In some tests it was down around 36-60%. It did poorly on both "in the wild" and "zoo" viruses. While the commercial AV's also do extremely poorly when confronted with over half a million viruses, bots, spyware and trojans in two tests, ClamAV was not in the high range in either test.
ClamAV's only advantage appears to be in detecting email viruses (since it was mostly designed to be an email scanner frontend) and in being quick to issue new detection signatures due to its community-based submission process.
ClamAV might be suitable for home users, but it is not suitable for any company with a significant email and Web traffic. And those are the companies who would be using it in Untangle's and other appliances.
An exception might be made for the version integrated with Spyware Terminator. For a small company that doesn't have that many malware vectors, this combo is a lot cheaper (free) than paying $500-1000 a year for multiple licenses from the commercial AV companies and will probably protect against spyware better, which is the main threat these days. The only problem with Spyware Terminator is its intrusiveness, when the HIDS hueristic IDS is turned on (it's off on a default install.)
Richard Steven Hack - This sig is TOO GODDAMN SHORT TO DO ANYTHING USEFUL WITH! MORONS!
Just tested on our corporate Trend installation (Windows XP)
It does not detect:
111_xxx.com
112_untangle1.zip
115_untangle3.zip
116_untangle4.zip