Slashdot Mirror
Microsoft Fails Antivirus Certification Test (Again), Challenges the Results
redletterdave writes "For the second time in a row, Microsoft's Security Essentials failed to earn certification from AV-Test, the independent German testing lab best known for evaluating the effectiveness of antivirus software. Out of 25 different security programs tested by AV-Test, including software from McAfee, Norman, Kaspersky, and others, Microsoft's Security Essentials was just one out of three that failed to gain certification. These results are noteworthy because Microsoft Security Essentials is currently (as of December) the most popular security suite in North America and the world."
For anyone who didnt get why bundling MSSE with Win8 was a terrible idea, this is it. I guarentee it is now the very first thing malware authors test against prior to release, and the number one target for circumvension. Previously McAfee and Norton were heavily targetted for circumvention, and had correspondingly bad scores; now its MSSEs turn.
Really, its eerie how perfectly the timing corresponds with Win8's release.
Hooray monoculture! Hooray killing off a previously viable AV option!
MSSE sucks, okay. That aside, AV-TEST is a fucking joke. Their top three products on their site are the worst overall products I've ever seen. Yes, they detect viruses. They also slow your system to a crawl, have awful user interfaces, are terribly priced, have bad scanning options, slow scanning engines, have false positives like crazy, and and generally terrible. They apparently didn't take much if any of THAT into consideration unfortunately. Obviously the tests were tailored towards certain products so the whole site is a giant joke/advertisement.
Aaaaaand AV-TEST responded already:
http://www.theregister.co.uk/2013/01/17/avtest_microsoft_test_dispute/
So long as you keep your software updated then there's not really much of a point other than the chance you'll spread an infected file onward without being infected yourself.
Think. No, that's not good enough, think some more: Viruses (we are explicitly talking viruses here, says "Antivirus" right in the test and headline) exploit unpatched vulnerabilities (mistakes) in software. Patched software is immune to the prior vulnerabilities, so AV won't "protect" you from things you're immune to. It also won't protect you from viruses with signatures that it doesn't know about. So, What's the point of wasting all those CPU cycles scanning? Oh, maybe you got infected and it could remove it later? WRONG. Viruses actually mutate, say a malware author snags a virus, they reverse engineer how the payload is delivered and they change the payload to theirs and send it on its way -- The malware can even install other malware once it gets running. So, the (automated) removal options/instructions are probably not complete if the code has ever had a chance to run before. Ah, so now you may be thinking that it's exactly the reason why you'd waste CPU time on an AV scan, to detect infection so at least you'll know -- Except that's just silly. Think. If you were a spy and I asked you if you were a spy then would you say yes? An AV running in an infected machine can not reliably determine the state of the infected machine. AV: "Any Viruses here" Virus: "Nope!"
Often times I'll get people telling me, no matter which AV product they're using, that their machine is working strange, slower, showing adverts and wrong websites, and their AV will be chugging along saying everything is fine. You get more reliable warning from the malware itself! "You may have been Infected with 2042 viruses!" the scareware will prompt every boot, while Norton, or McAfee, or AVG, or ANY AV product I run across the infected machine says the coast is clear. You can't "remove" malware -- Nuke it from orbit, and re-install, it's the only way to be sure.
Look, people, hardware supports virtualization now. If you're NOT running your Windows boxen in a VM, then you're not concerned enough about security to benefit from an anti-virus anyway. Boot from a known clean state, maybe even a LiveCD/USB then do your virus scanning from there if you want to be able to detect anything with any degree of certainty, and even then it's questionable. If your data partition is separate from your (virtual) OS partitions then you can just always run (or restore) from a known good snapshot, and install updates to the known good snapshots, then make another snapshot before you do anything else.
I'm no Microsoft apologist, I don't have to worry about such things as much anymore because I use an OS that gets the patches out much faster than MS does, but I can certainly see where the people who understand the issues in Microsoft might realize that Antivirus isn't really the right option anyway, it's just a waste of time and there are other better solutions... Windows Steady State (or whatever it's called now), for example.
"Insanity: doing the same thing over and over again and expecting different results."
"The significant problems we face can not be solved at the same level of thinking we were at when we created them."
- Albert Einstein
Does anyone else think it is kind of funny that the Microsoft response is (to paraphrase); We did not detect any of the software they say we could not detect. That being said they may have a real point that their software is designed to detect real world threats and not proof of concepts that never leave the lab. Without more in depth analyses than I am willing to do, I can do little more than jump to conclusions based upon my own personal bias.
"I myself am made entirely of flaws, stitched together with good intentions."
You do realise that AV-Test acknowledged that MSSE detected 100% of known malware threats. 100%. Where it failed was on 0-day viruses which aren't in the wild and which (per MS) only impacted 0.0033% of users (which may be several Win8 users, but considering how badly ignorant the general populace is of PC security, happily installing DOWNLOADFREEPORNMOVIES1080PHD.EXE, etc, this isn't many).
I understand you have a preconceived notion and have basically read the summary and decided that MSSE isn't any good at detecting viruses - while ignoring the actual facts of the issue - it IS good at detecting viruses. It's heuristics aren't as good as some (it only picks up 8 out of 10 brand new malware samples that aren't necessarily even in the wild) but it's detection routines are very good.
From AV-Test:
"AV-Test teams take malware that is minutes old, Marx explained, and run the data into the security testing suite. A testing process carried out by Microsoft much later would be bound to cover the malware tested, since samples would already have been reported.
Today, every two seconds we see three new malware samples, which are summing up to a few million samples per month. Instead of looking at millions of samples, our focus is on the unique families," Marx explained.
"Out of every family, we select recent samples in order to use them in our tests. So the impact of these samples is indeed low, however, the impact of the malware family is considerably high."
So they've acknowledged themselves that 1) the impact of the new samples they're testing is practically non existant, being minutes or even SECONDS old, and 2) by the time these samples are in the wild, Microsoft would have already added them to their detection routines.
Basically, MS and AVTest are looking at two different things. AVTest is basically testing to see "how good is a piece of software at detecting that certain code its never encountered before, is malware". MS, on the other hand, is constantly going "OK, what new malware is there for us to detect? Add it to the detection routines." And to be fair, MSSE was never meant to be a heavily analystic package. There's plenty of those available if you want them. MSSE is AV for the masses, and in terms of known-virus detection it's among the best available and has been for years.
"The true measure of a person is how they act when they know they won't get caught." - DSRilk