Slashdot Mirror
Microsoft Fails Antivirus Certification Test (Again), Challenges the Results
redletterdave writes "For the second time in a row, Microsoft's Security Essentials failed to earn certification from AV-Test, the independent German testing lab best known for evaluating the effectiveness of antivirus software. Out of 25 different security programs tested by AV-Test, including software from McAfee, Norman, Kaspersky, and others, Microsoft's Security Essentials was just one out of three that failed to gain certification. These results are noteworthy because Microsoft Security Essentials is currently (as of December) the most popular security suite in North America and the world."
For anyone who didnt get why bundling MSSE with Win8 was a terrible idea, this is it. I guarentee it is now the very first thing malware authors test against prior to release, and the number one target for circumvension. Previously McAfee and Norton were heavily targetted for circumvention, and had correspondingly bad scores; now its MSSEs turn.
Really, its eerie how perfectly the timing corresponds with Win8's release.
Hooray monoculture! Hooray killing off a previously viable AV option!
When people have invested time and money into learning and deploying a technology, there is no argument, no matter how rational, that will persuade them to use something different.
It's a very sad state of affairs.
Norman
"Our two-party system is like a bowl of shit looking at itself in a mirror." - Lewis Black
Popularity shouldn't be based on the number of installs, but the number of people who use it, and how often they use it. Microsoft has more or less forced people to install Microsoft Security Essentials, so I don't think it's a fair comparison at all. I don't use it, but it's there and Windows Update gets psychotic with errors and alerts if it's uninstalled. More so than if it's not "genuine" even!
#fuckbeta #iamslashdot #dicemustdie
MSSE sucks, okay. That aside, AV-TEST is a fucking joke. Their top three products on their site are the worst overall products I've ever seen. Yes, they detect viruses. They also slow your system to a crawl, have awful user interfaces, are terribly priced, have bad scanning options, slow scanning engines, have false positives like crazy, and and generally terrible. They apparently didn't take much if any of THAT into consideration unfortunately. Obviously the tests were tailored towards certain products so the whole site is a giant joke/advertisement.
This is always the problem with testing AV software in a lab -- it's barely indicative of anything in the real world, and you can't truly test in the real world due to having no idea what you've missed (unless you go back and search as MS apparently did in this case).
So the question is whether Microsoft's reponse is correct or FUD. Did they perform better in the real world than on this test? Do they perform better in the real world compared to competitors who did well on the test? Those are super hard questions to answer.
I use MSE in large part because it's really lightweight. Norton is a pig and AVG never failed to fuck itself up on my system. And so far I've had no malware issues, so I'm inclined to believe them here even those my experience is anecdotal.
I doubt this company tests all those AV suites out of the kindess of their own heart. A "test" commissioned by the for-profit AV industry is going to show their products in a favorable light. (Or you'll never see it published)
AV at this point is damn near snake oil. Well, at least anything beyond the coverage that MSE provides.It keeps old threats from spreading, which is good. It's damn foolish to be hit by a 2 year old virus. In the enterprise/buisness having an AV suite is just PR move. A CYA to show that you put a token of effort in to protecting your systems. (Hey! We had an AV suite. It's not our fault our network is riddled with worms)
But the real threat is still the new stuff. The bad guys still do quite well for themselves even if they have to write a new virus every few weeks. Who gives a wet fart about how well your signature based AV suite (which the all are) does against zero day threats? Nobody. Because it's impossible for a signature based AV suite to offer any kind of effective defense against unknown threats.
Aaaaaand AV-TEST responded already:
http://www.theregister.co.uk/2013/01/17/avtest_microsoft_test_dispute/
A piece of software might be #1 in one market (the US), #1 overall (the world), but not #1 in other markets (like Europe, Japan, or South Africa).
Actually, considering they are mentioning company names, and not products, I'm sure they meant Norman. "Norton" is the name of the product by Symantec, and Norman is listed on the tests.
morcego
That is not what it is saying at all. It is a compound sentence that is stating two things:
1) It is the most popular security suite in North America.
2) It is the most popular security suite in the world.
These things are not mutual, so it makes sense to state both. It could be the most popular in the N America, but some other AV product in China could be even more popular and hold the rank of "most popular in the World". Now I'm sure some people would say why then doesn't it fairly list off dozens of other countries, etc. I'm not going to get into all that.
Sigh.
Saxon AV has always been better.
I bullshit you not, there's a Norman: Security Suite Pro 9.0. I seriously doubt that's what they meant to type though, given the context.
Actually both Norman (it's real) and Norton passed. http://www.av-test.org/en/tests/home-user/windows-7/novdec-2012/
Be seeing you...
Yes.
And yet, its quite possible for something to be the most popular in North America but not the most popular in the world, or vice versa. So, inasmuch as both "North America" and "the world" are interesting scopes of analysis, it is meaningful to identify that MSSE is the most popular in each of those scopes.
What if 100,000 people used in the North America, and that is more than any other AV product in North America, but in China 5,000,000 use Chinese National AV Protection service(I made the name up) and no one uses MSSE outside of N America. So then MSSE wouldn't hold the title of "in the world" now would it?
So they are stating:
1) It is the most popular security suite in North America.
2) It is the most popular security suite in the world.
These things are not mutual, so it makes sense to state both. They are independent, and one does not imply the other.
There's only one thing worse than a grammar Nazi, and that's a grammar Nazi that doesn't know grammar.
We seceded, because we were tired of having to put up with everyone elses crap.
So long as you keep your software updated then there's not really much of a point other than the chance you'll spread an infected file onward without being infected yourself.
Think. No, that's not good enough, think some more: Viruses (we are explicitly talking viruses here, says "Antivirus" right in the test and headline) exploit unpatched vulnerabilities (mistakes) in software. Patched software is immune to the prior vulnerabilities, so AV won't "protect" you from things you're immune to. It also won't protect you from viruses with signatures that it doesn't know about. So, What's the point of wasting all those CPU cycles scanning? Oh, maybe you got infected and it could remove it later? WRONG. Viruses actually mutate, say a malware author snags a virus, they reverse engineer how the payload is delivered and they change the payload to theirs and send it on its way -- The malware can even install other malware once it gets running. So, the (automated) removal options/instructions are probably not complete if the code has ever had a chance to run before. Ah, so now you may be thinking that it's exactly the reason why you'd waste CPU time on an AV scan, to detect infection so at least you'll know -- Except that's just silly. Think. If you were a spy and I asked you if you were a spy then would you say yes? An AV running in an infected machine can not reliably determine the state of the infected machine. AV: "Any Viruses here" Virus: "Nope!"
Often times I'll get people telling me, no matter which AV product they're using, that their machine is working strange, slower, showing adverts and wrong websites, and their AV will be chugging along saying everything is fine. You get more reliable warning from the malware itself! "You may have been Infected with 2042 viruses!" the scareware will prompt every boot, while Norton, or McAfee, or AVG, or ANY AV product I run across the infected machine says the coast is clear. You can't "remove" malware -- Nuke it from orbit, and re-install, it's the only way to be sure.
Look, people, hardware supports virtualization now. If you're NOT running your Windows boxen in a VM, then you're not concerned enough about security to benefit from an anti-virus anyway. Boot from a known clean state, maybe even a LiveCD/USB then do your virus scanning from there if you want to be able to detect anything with any degree of certainty, and even then it's questionable. If your data partition is separate from your (virtual) OS partitions then you can just always run (or restore) from a known good snapshot, and install updates to the known good snapshots, then make another snapshot before you do anything else.
I'm no Microsoft apologist, I don't have to worry about such things as much anymore because I use an OS that gets the patches out much faster than MS does, but I can certainly see where the people who understand the issues in Microsoft might realize that Antivirus isn't really the right option anyway, it's just a waste of time and there are other better solutions... Windows Steady State (or whatever it's called now), for example.
"Insanity: doing the same thing over and over again and expecting different results."
"The significant problems we face can not be solved at the same level of thinking we were at when we created them."
- Albert Einstein
Does anyone else think it is kind of funny that the Microsoft response is (to paraphrase); We did not detect any of the software they say we could not detect. That being said they may have a real point that their software is designed to detect real world threats and not proof of concepts that never leave the lab. Without more in depth analyses than I am willing to do, I can do little more than jump to conclusions based upon my own personal bias.
"I myself am made entirely of flaws, stitched together with good intentions."
If performance is your priority then don't use A/V.
How about: "If security is your priority then keep your computer powered off."
Obviously there are various trade-offs between these two extremes.
From the article:
“The other 94 percent of the samples don't represent what our customers encounter. When we explicitly looked for these files, we could not find them on our customers' machines.”
Or in other words: "Thank you for installing the software necessary to allow us to browse through the contents of your computer when we feel like it and report any interesting findings back to us..."
All in good faith, of course.
I am convinced there must be at least ONE shady AV company that creates viruses to make money. Hard to prove, but very well possible.
KERNEL PANIC -SIGFAULT AT ADDRESS #51A54D07
I take it a step further. I carry around a "1" and a "0" in my pocket.
If I need to compute something I pull them out and get to work.
> Microsoft DOS 6 with AV built in ... was defeated by every virus writer
That's because MSAV included the classic, textbook example of "security through obscurity." Utilities like FORMAT and FDISK would do the same things as some malware, which would cause false alarms. The users would be terrified by this, so there was a solution: a "secret" (wink, wink!) system call in the OS that their utilities used to temporarily disable the alarms. (!!!)
It was top secret ... so naturally, everyone knew about it. A call to disable VSAFE became one thing that EVERY DOS virus writer put at the top of his code. Naturally. Of course.
Ah, you're bringing back memories now. :)
Cogito, igitur comedam pizza.
Your source links to a Wikipedia page that says the "plural of virus is viruses". Virii is not generally accepted. The word virus has no plural in latin. Here's some further discussion here.
Not all words ending in -us are plural with an -ii suffix. See genus (plural genera) for an example.
You do realise that AV-Test acknowledged that MSSE detected 100% of known malware threats. 100%. Where it failed was on 0-day viruses which aren't in the wild and which (per MS) only impacted 0.0033% of users (which may be several Win8 users, but considering how badly ignorant the general populace is of PC security, happily installing DOWNLOADFREEPORNMOVIES1080PHD.EXE, etc, this isn't many).
I understand you have a preconceived notion and have basically read the summary and decided that MSSE isn't any good at detecting viruses - while ignoring the actual facts of the issue - it IS good at detecting viruses. It's heuristics aren't as good as some (it only picks up 8 out of 10 brand new malware samples that aren't necessarily even in the wild) but it's detection routines are very good.
From AV-Test:
"AV-Test teams take malware that is minutes old, Marx explained, and run the data into the security testing suite. A testing process carried out by Microsoft much later would be bound to cover the malware tested, since samples would already have been reported.
Today, every two seconds we see three new malware samples, which are summing up to a few million samples per month. Instead of looking at millions of samples, our focus is on the unique families," Marx explained.
"Out of every family, we select recent samples in order to use them in our tests. So the impact of these samples is indeed low, however, the impact of the malware family is considerably high."
So they've acknowledged themselves that 1) the impact of the new samples they're testing is practically non existant, being minutes or even SECONDS old, and 2) by the time these samples are in the wild, Microsoft would have already added them to their detection routines.
Basically, MS and AVTest are looking at two different things. AVTest is basically testing to see "how good is a piece of software at detecting that certain code its never encountered before, is malware". MS, on the other hand, is constantly going "OK, what new malware is there for us to detect? Add it to the detection routines." And to be fair, MSSE was never meant to be a heavily analystic package. There's plenty of those available if you want them. MSSE is AV for the masses, and in terms of known-virus detection it's among the best available and has been for years.
"The true measure of a person is how they act when they know they won't get caught." - DSRilk
No... it is not. Using an ending of 'i' for the plural form from words where the singular form ends in 'us' comes from Latin, and is as such only applicable to Latin plurals. Virus is originally a Latin word, but in Latin could not itself possess a plural, because it did not denote a single thing. It is best likened to an English noun which does not have a quantity associated with it, such as "happiness" or "everything", and so does not make any sense to try to pluralize. If you are a native English speaker, trying to pluralize such words is going to probably sound sort of odd. That's because it's wrong. In modern English, we have have altered the conceptual meaning of the word virus so that it can refer to a unique thing, but because that is an English invention and not Latin, the plural follows English convention for pluralization and not Latin. Hence, viruses.
File under 'M' for 'Manic ranting'
Whether virus has a morphologically marked plural in latin is debatable. The discussion you link to claims that "virus" is a 4th declension noun, but all dictionaries I've checked (including Oxford!) says it's a 2nd declension noun. Anyway, "virus" is a neuter, not masculine noun, which means that the latin plural (if it really is 2nd declension) is not "viri" ("virii" does not make sense to me; is it an anglicism?), but "vira", which btw is well established as an alternative to "virus", at least in Denmark.
But that is not an -ii suffix. It's still an -i suffix (Pri-us, Pri-i)!