Slashdot Mirror
Microsoft Fails Antivirus Certification Test (Again), Challenges the Results
redletterdave writes "For the second time in a row, Microsoft's Security Essentials failed to earn certification from AV-Test, the independent German testing lab best known for evaluating the effectiveness of antivirus software. Out of 25 different security programs tested by AV-Test, including software from McAfee, Norman, Kaspersky, and others, Microsoft's Security Essentials was just one out of three that failed to gain certification. These results are noteworthy because Microsoft Security Essentials is currently (as of December) the most popular security suite in North America and the world."
For anyone who didnt get why bundling MSSE with Win8 was a terrible idea, this is it. I guarentee it is now the very first thing malware authors test against prior to release, and the number one target for circumvension. Previously McAfee and Norton were heavily targetted for circumvention, and had correspondingly bad scores; now its MSSEs turn.
Really, its eerie how perfectly the timing corresponds with Win8's release.
Hooray monoculture! Hooray killing off a previously viable AV option!
MSSE sucks, okay. That aside, AV-TEST is a fucking joke. Their top three products on their site are the worst overall products I've ever seen. Yes, they detect viruses. They also slow your system to a crawl, have awful user interfaces, are terribly priced, have bad scanning options, slow scanning engines, have false positives like crazy, and and generally terrible. They apparently didn't take much if any of THAT into consideration unfortunately. Obviously the tests were tailored towards certain products so the whole site is a giant joke/advertisement.
If performance is your priority then don't use A/V.
How about: "If security is your priority then keep your computer powered off."
Obviously there are various trade-offs between these two extremes.
But how do they test for effectiveness against zero-day attacks? Where do they get the zero-days from? If I'm a virus author I'd test my zero day with one of those websites ( http://www.makeuseof.com/tag/7-reliable-sites-quick-free-anti-virus-scan/ ) that scan for viruses with practically all the AV software in the market.
So the zero day when finally released will NOT be detected by ANY of them!
Maybe what an AV vendor could do is secretly work with these AV websites to detect suspicious activity..
Heuristics. Basically AV vendors set their software to look for something, anything that could be judged as "virus like" and flag it.
As a result, tester's top AV software picks are also top picks in hogging system resources, and tend to produce ridiculous amounts of false positives. Because that's what massively overly tight settings on heuristics engine will do. But AV vendors sell FEAR first and foremost. The more "scary stuff" their AV finds, the more likely user will think "oh this AV just saved me from losing my bank account!" and buy more.
MSSE has worst success in zero day detection because their heuristics engine is one of the more sane ones on the market. It's light on resources and rarely (in comparison to the top picks of that tests) produces false positives. As a result, it also has a higher chance of missing zero day stuff that might have been detected by extremely aggressive heuristics scanner.
Your source links to a Wikipedia page that says the "plural of virus is viruses". Virii is not generally accepted. The word virus has no plural in latin. Here's some further discussion here.
Not all words ending in -us are plural with an -ii suffix. See genus (plural genera) for an example.
You do realise that AV-Test acknowledged that MSSE detected 100% of known malware threats. 100%. Where it failed was on 0-day viruses which aren't in the wild and which (per MS) only impacted 0.0033% of users (which may be several Win8 users, but considering how badly ignorant the general populace is of PC security, happily installing DOWNLOADFREEPORNMOVIES1080PHD.EXE, etc, this isn't many).
I understand you have a preconceived notion and have basically read the summary and decided that MSSE isn't any good at detecting viruses - while ignoring the actual facts of the issue - it IS good at detecting viruses. It's heuristics aren't as good as some (it only picks up 8 out of 10 brand new malware samples that aren't necessarily even in the wild) but it's detection routines are very good.
From AV-Test:
"AV-Test teams take malware that is minutes old, Marx explained, and run the data into the security testing suite. A testing process carried out by Microsoft much later would be bound to cover the malware tested, since samples would already have been reported.
Today, every two seconds we see three new malware samples, which are summing up to a few million samples per month. Instead of looking at millions of samples, our focus is on the unique families," Marx explained.
"Out of every family, we select recent samples in order to use them in our tests. So the impact of these samples is indeed low, however, the impact of the malware family is considerably high."
So they've acknowledged themselves that 1) the impact of the new samples they're testing is practically non existant, being minutes or even SECONDS old, and 2) by the time these samples are in the wild, Microsoft would have already added them to their detection routines.
Basically, MS and AVTest are looking at two different things. AVTest is basically testing to see "how good is a piece of software at detecting that certain code its never encountered before, is malware". MS, on the other hand, is constantly going "OK, what new malware is there for us to detect? Add it to the detection routines." And to be fair, MSSE was never meant to be a heavily analystic package. There's plenty of those available if you want them. MSSE is AV for the masses, and in terms of known-virus detection it's among the best available and has been for years.
"The true measure of a person is how they act when they know they won't get caught." - DSRilk