Microsoft Fails Antivirus Certification Test (Again), Challenges the Results
redletterdave writes "For the second time in a row, Microsoft's Security Essentials failed to earn certification from AV-Test, the independent German testing lab best known for evaluating the effectiveness of antivirus software. Out of 25 different security programs tested by AV-Test, including software from McAfee, Norman, Kaspersky, and others, Microsoft's Security Essentials was just one out of three that failed to gain certification. These results are noteworthy because Microsoft Security Essentials is currently (as of December) the most popular security suite in North America and the world."
For anyone who didnt get why bundling MSSE with Win8 was a terrible idea, this is it. I guarentee it is now the very first thing malware authors test against prior to release, and the number one target for circumvension. Previously McAfee and Norton were heavily targetted for circumvention, and had correspondingly bad scores; now its MSSEs turn.
Really, its eerie how perfectly the timing corresponds with Win8's release.
Hooray monoculture! Hooray killing off a previously viable AV option!
GROW UP >> "in North America and the world"
It's likely one of the most popular due to:
Free
Least amount of bloatware
WTF editors.
When people have invested time and money into learning and deploying a technology, there is no argument, no matter how rational, that will persuade them to use something different.
It's a very sad state of affairs.
Popularity shouldn't be based on the number of installs, but the number of people who use it, and how often they use it. Microsoft has more or less forced people to install Microsoft Security Essentials, so I don't think it's a fair comparison at all. I don't use it, but it's there and Windows Update gets psychotic with errors and alerts if it's uninstalled. More so than if it's not "genuine" even!
#fuckbeta #iamslashdot #dicemustdie
MSSE sucks, okay. That aside, AV-TEST is a fucking joke. Their top three products on their site are the worst overall products I've ever seen. Yes, they detect viruses. They also slow your system to a crawl, have awful user interfaces, are terribly priced, have bad scanning options, slow scanning engines, have false positives like crazy, and and generally terrible. They apparently didn't take much if any of THAT into consideration unfortunately. Obviously the tests were tailored towards certain products so the whole site is a giant joke/advertisement.
Since when is North America not part of the rest of the world?
Microsoft Security Essentials is currently (as of December) the most popular security suite in North America and the world.
Sorry to nitpick, but seriously? Did it REALLY need to be specified that it's the most popular in North America AND the world?
They DO realize that North America is part of the planet, right?
I don't even recommend to use any other AV because they mostly scare users, keeps reminding them that antivirus is updated, might be out of date and do another annoying stuff.
This is always the problem with testing AV software in a lab -- it's barely indicative of anything in the real world, and you can't truly test in the real world due to having no idea what you've missed (unless you go back and search as MS apparently did in this case).
So the question is whether Microsoft's reponse is correct or FUD. Did they perform better in the real world than on this test? Do they perform better in the real world compared to competitors who did well on the test? Those are super hard questions to answer.
I use MSE in large part because it's really lightweight. Norton is a pig and AVG never failed to fuck itself up on my system. And so far I've had no malware issues, so I'm inclined to believe them here even those my experience is anecdotal.
I doubt this company tests all those AV suites out of the kindess of their own heart. A "test" commissioned by the for-profit AV industry is going to show their products in a favorable light. (Or you'll never see it published)
AV at this point is damn near snake oil. Well, at least anything beyond the coverage that MSE provides.It keeps old threats from spreading, which is good. It's damn foolish to be hit by a 2 year old virus. In the enterprise/buisness having an AV suite is just PR move. A CYA to show that you put a token of effort in to protecting your systems. (Hey! We had an AV suite. It's not our fault our network is riddled with worms)
But the real threat is still the new stuff. The bad guys still do quite well for themselves even if they have to write a new virus every few weeks. Who gives a wet fart about how well your signature based AV suite (which the all are) does against zero day threats? Nobody. Because it's impossible for a signature based AV suite to offer any kind of effective defense against unknown threats.
Aaaaaand AV-TEST responded already:
http://www.theregister.co.uk/2013/01/17/avtest_microsoft_test_dispute/
Ha Ha Ha!
Does anyone else remember Microsoft DOS 6 with AV built in? It was defeated by every virus writer imaginable before it was released. Hell, even VCL (virus creation lab) had it circumvented before released.
Okay, but seriously. If anyone trusts a company with a known history of abuses to audit and secure themselves, PT Barnum had you pegged.
-The wise argue that there are few absolutes, the fool argues that there are no probabilities.
Yes.
And yet, its quite possible for something to be the most popular in North America but not the most popular in the world, or vice versa. So, inasmuch as both "North America" and "the world" are interesting scopes of analysis, it is meaningful to identify that MSSE is the most popular in each of those scopes.
So long as you keep your software updated then there's not really much of a point other than the chance you'll spread an infected file onward without being infected yourself.
Think. No, that's not good enough, think some more: Viruses (we are explicitly talking viruses here, says "Antivirus" right in the test and headline) exploit unpatched vulnerabilities (mistakes) in software. Patched software is immune to the prior vulnerabilities, so AV won't "protect" you from things you're immune to. It also won't protect you from viruses with signatures that it doesn't know about. So, What's the point of wasting all those CPU cycles scanning? Oh, maybe you got infected and it could remove it later? WRONG. Viruses actually mutate, say a malware author snags a virus, they reverse engineer how the payload is delivered and they change the payload to theirs and send it on its way -- The malware can even install other malware once it gets running. So, the (automated) removal options/instructions are probably not complete if the code has ever had a chance to run before. Ah, so now you may be thinking that it's exactly the reason why you'd waste CPU time on an AV scan, to detect infection so at least you'll know -- Except that's just silly. Think. If you were a spy and I asked you if you were a spy then would you say yes? An AV running in an infected machine can not reliably determine the state of the infected machine. AV: "Any Viruses here" Virus: "Nope!"
Often times I'll get people telling me, no matter which AV product they're using, that their machine is working strange, slower, showing adverts and wrong websites, and their AV will be chugging along saying everything is fine. You get more reliable warning from the malware itself! "You may have been Infected with 2042 viruses!" the scareware will prompt every boot, while Norton, or McAfee, or AVG, or ANY AV product I run across the infected machine says the coast is clear. You can't "remove" malware -- Nuke it from orbit, and re-install, it's the only way to be sure.
Look, people, hardware supports virtualization now. If you're NOT running your Windows boxen in a VM, then you're not concerned enough about security to benefit from an anti-virus anyway. Boot from a known clean state, maybe even a LiveCD/USB then do your virus scanning from there if you want to be able to detect anything with any degree of certainty, and even then it's questionable. If your data partition is separate from your (virtual) OS partitions then you can just always run (or restore) from a known good snapshot, and install updates to the known good snapshots, then make another snapshot before you do anything else.
I'm no Microsoft apologist, I don't have to worry about such things as much anymore because I use an OS that gets the patches out much faster than MS does, but I can certainly see where the people who understand the issues in Microsoft might realize that Antivirus isn't really the right option anyway, it's just a waste of time and there are other better solutions... Windows Steady State (or whatever it's called now), for example.
"Insanity: doing the same thing over and over again and expecting different results."
"The significant problems we face can not be solved at the same level of thinking we were at when we created them."
- Albert Einstein
Does anyone else think it is kind of funny that the Microsoft response is (to paraphrase); We did not detect any of the software they say we could not detect. That being said they may have a real point that their software is designed to detect real world threats and not proof of concepts that never leave the lab. Without more in depth analyses than I am willing to do, I can do little more than jump to conclusions based upon my own personal bias.
"I myself am made entirely of flaws, stitched together with good intentions."
Personally I think this is pretty much irrelevant. The antivirus model in general is extremely dated and innefective. I see infected machines left and right with every antivirus out there. I usually install Security Essentials simply because it is lightweight and has no leg-humping pop-ups every time you so much as scratch your nose. Otherwise the most effective protection is to remove every security hole-ridden piece of crapware and browser add-on that you don't use (yes that includes Java), install an ad-blocker, and don't be a freaking retard about what you click on and/or download. So long as people expect their antivirus to be a magic malware-blocking forcefield(and as long as the vendors continue advertising them as such), this problem will not get any better.
AV-Test’s review looks at three key areas of security software, including protection, reparability, and usability of the whole computer based on the software’s impact. Across those three areas, Microsoft Security Essentials scored a 1.5 out of 6 on protection against viruses and worms, a 3.0 out of 6 on a reparability scale, and a 5.5 out of 6 on the usability scale, where “lower values indicate better results.” This is incorrect, higher values indicate better results, otherwise this article would be about how great MSE is at detecting viruses and worms, but how no one uses it because the usability is awful.
From the AV-TEST test results, it appears the issue with MSE doing poorly in this test is a poor score in protection against 0-day malware attacks (~70% vs an average of ~90% protected) and in detecting relatively newer malware "Detection of a representative set of malware discovered in the last 2-3 months" (~90% vs an average of ~97% detected). Although things like "representative sets" could potentially be used in a biased manner...
If performance is your priority then don't use A/V.
How about: "If security is your priority then keep your computer powered off."
Obviously there are various trade-offs between these two extremes.
Let me know once you've successfully installed Norton OS 5.0, AVG OS 3.2 or even AVAST! OS 13.5...
From the article:
“The other 94 percent of the samples don't represent what our customers encounter. When we explicitly looked for these files, we could not find them on our customers' machines.”
Or in other words: "Thank you for installing the software necessary to allow us to browse through the contents of your computer when we feel like it and report any interesting findings back to us..."
All in good faith, of course.
I went to the AV-Test Web site at http://www.av-test.org/en/home/. First of all, there is indeed a Norman Security Suite at http://safeground.norman.com/us/home_and_small_office. AV-Test listed Norton under Symantec. Yes, AV-Test evaluated both Norton and Norman.
For home users of Windows XP, Microsoft's Security Essentials has a AV-Test certified seal with a test date in August 2012. For corporate users of Windows XP, Microsoft's Forefront Endpoint Protection has a AV-Test certified seal with a test date also in August 2012. Neither product has the certified seal for Windows 8. But then how many corporate users have actually adopted Windows 8?
Besides AV-Test, there is also ICSA Labs at https://www.icsalabs.com/. ICSA Labs also reports on Norman.
ICSA Labs certifies Microsoft Security Essentials for home users of Windows XP and Microsoft Forefront Endpoint Protection for Windows 7 without any dates indicated. Apparently, ICSA has not certified any anti-virus applications for Windows 8.
I use AVG 2013 Free, which is certified by AV-Test but has not been evaluated by ICSA Labs since 2005 (many versions ago). I also prefer to go to the original sources of information on software -- AV-Test and ICSA Labs in this case -- not to news reports often written by reporters who might not understand the subject.
Other than it's actual effectiveness, I guess, I really like MSE for its clean, no-nonsense UI -- as opposed to every other AV software maker has elected to use some batshit redarted-ass UI that changes on a daily basis because AV software is otherwise boring and unglamourous.
I am convinced there must be at least ONE shady AV company that creates viruses to make money. Hard to prove, but very well possible.
KERNEL PANIC -SIGFAULT AT ADDRESS #51A54D07
I used to read the AV comparisons once in a while. MS Security Essentials used to score fairly high on these tests! Back when it was one of the top rated products I installed it on the two machines that in my house that still run windows -- my wife's laptop and my son't netbook. I assumed (obviously wrongly) that the quality had been maintained.
May the best liar win. Sure they both have their own peculiar brand of corruption; but they're both liars.
I take it a step further. I carry around a "1" and a "0" in my pocket.
If I need to compute something I pull them out and get to work.
To be honest, 99.9967% isn't very bad at all. It's pretty close to the golden "five nines."
Security software is like birth control, no one form is 100% effective; therefore always use two. Unless you mean abstinence. And that's no fun
"virii" is not a fucking word, moron.
Are they not included in the test or am I just missing them?
"I believe in Karma. That means I can do bad things to people all day long and I assume they deserve it." : Dogbert
It is better than a Norton or McAfee that came with the system, had the subscription lapse, and hasn't had updated defs for at least a year or more. And it also doesn't bork your system worse than if you had malware on it.
Are you serious, you either pass or fail this Kind of test. Microsoft is known for creating the most hideously insecure software on the market, im not surprised they fail attempting to create antivirus software. If Microsoft was good at it, third party companys wouldnt make billions supporting windows.
To believe that Microsoft Security Essentials is any good at what it is ostensibly meant to do is to believe that Microsoft is good at detecting and clearing viruses from users' systems, but to believe this is to hold a contradiction to every observation made of various versions of the Windows operating system.
To me it seems even more contradictory to have no trust in Microsoft's coding ability and yet continue to run Windows. If the people who know the most about Windows can't secure it, what makes you think some third-party can?
If you don't trust MS, you shouldn't run Windows.
Do what thou wilt shall be the whole of the Law
There's no software available that will protect against the weak link in the chain ... the nut at the keyboard. I've been using MSSE for years and have yet to get a single infection. I'm sure I'll slip some day but so far it's been a good backstop for me.
You do realise that AV-Test acknowledged that MSSE detected 100% of known malware threats. 100%. Where it failed was on 0-day viruses which aren't in the wild and which (per MS) only impacted 0.0033% of users (which may be several Win8 users, but considering how badly ignorant the general populace is of PC security, happily installing DOWNLOADFREEPORNMOVIES1080PHD.EXE, etc, this isn't many).
I understand you have a preconceived notion and have basically read the summary and decided that MSSE isn't any good at detecting viruses - while ignoring the actual facts of the issue - it IS good at detecting viruses. It's heuristics aren't as good as some (it only picks up 8 out of 10 brand new malware samples that aren't necessarily even in the wild) but it's detection routines are very good.
From AV-Test:
"AV-Test teams take malware that is minutes old, Marx explained, and run the data into the security testing suite. A testing process carried out by Microsoft much later would be bound to cover the malware tested, since samples would already have been reported.
Today, every two seconds we see three new malware samples, which are summing up to a few million samples per month. Instead of looking at millions of samples, our focus is on the unique families," Marx explained.
"Out of every family, we select recent samples in order to use them in our tests. So the impact of these samples is indeed low, however, the impact of the malware family is considerably high."
So they've acknowledged themselves that 1) the impact of the new samples they're testing is practically non existant, being minutes or even SECONDS old, and 2) by the time these samples are in the wild, Microsoft would have already added them to their detection routines.
Basically, MS and AVTest are looking at two different things. AVTest is basically testing to see "how good is a piece of software at detecting that certain code its never encountered before, is malware". MS, on the other hand, is constantly going "OK, what new malware is there for us to detect? Add it to the detection routines." And to be fair, MSSE was never meant to be a heavily analystic package. There's plenty of those available if you want them. MSSE is AV for the masses, and in terms of known-virus detection it's among the best available and has been for years.
"The true measure of a person is how they act when they know they won't get caught." - DSRilk
If you don't have an AV, Windows 7 nags you about getting one and it leads you to Security Essential.
is the user space not 99.9997% segregated from the OS by a thin strand of heavily scrutinized string of operations by the OS itself?
The OS should be WELL hidden from not only the user but ALSO the likes of Java, Adobe any piece of software that thinks it is ok to install system level easter eggs that should only be ran in user space. The whole ideology is a profit gold mine and is NOT designed to be made extinct, just remedied; like big pharma. The OS should be creating the user in a VM sandbox each time it loads up and the users interaction with the OS should never be allowed. Both the OS and the User can run side by side in separate VM's and the OS should be one way communicating with the User, period.
If the company can't even write a decent, secure operating system to begin with, ...
In reality no one can write a decent, secure operating system.
Computers are meant to be used, therefore they can be hacked. It is sort of like saying why can't someone make a decent, secure lock.
I don't use any antivirus software.
AccountKiller
The most popular platform for viruses. Why not switch??
1. An up to date patched system is secure
This is wrong.
A patched system is protected against attacks that the vendor has taken trouble to create a patch for.
That means that the vendor is aware of the problem, has taken action against the problem, and has
successfully patched it.
A lot of the vulnerabilities exploited by malware are unknown and may stay unknown for a long time.
During that time a fully patched and up to date system is vulnerable.
I'm not an Apple fanboy by any means. But hasn't signed code proven to be the best method to prevent viruses? Dont know how it might work in the PC ecosystem but I'm convinced it should be attempted, perhaps a trusted third party set up to regulate, funded by PC manufacturers, banks, insurance companies, and legitimate software companies. Give developers the option to turn off but make it in BIOS or something so the average Joe cant just be tricked into doing it.
Is the fact that these competing antimalware companies do not openly publish and/or share detection methods or datasets. This ultimately does little more than give the users a false sense of security no matter which product is being used. What should be done (and what I've been attempting to do for quite some time) is to have a centralized/universal database of definitions, and from there, the real competition would be who, or what company can write the most effective *scanner*, thus benefiting the user, and weeding out ineffective coding practices, half-baked theories and groundless conjecture. To illustrate what I'm referring to, here are the datasets I maintain on a fairly regular basis. Keep in mind that 0-F is not an actual URL, but some of the datasets are defined as single characters, and sorted accordingly.
http://www.tot-ltd.org/blacklist/0-F/
http://www.tot-ltd.org/whitelist/0-F/
http://www.tot-ltd.org/API
http://www.tot-ltd.org/heuristics.dat
http://www.tot-ltd.org/installation.db
http://www.tot-ltd.org/packer.db
http://www.tot-ltd.org/files-wl/
http://www.tot-ltd.org/files-bl/
In the end, sure, there are several million files, but each specific group is only a few hundred bytes in size, which reduces a LOT of overhead and brings individual scantimes to near zero with a halfway decent connection speed. By doing this, a single scan is limited only by your hardware and internet latency.
Using MSSE to secure your Windows computer is like buying a car from a car company, realizing they forgot to include locks on the doors, and when you take it back to complain, they suggest you just use their after-market anti-theft system. Thanks, but I'm looking for a car that comes with locks, and/or an operating system with minimal vulnerabilities in the first place.
If Microsoft's engineers knew how to make Windows virus-proof, it would ship that way.
I've had MSSE on my dad's laptop for a few years now. He recently mentioned his laptop was very slow, so I guessed it was overdue for some maintenance. A quick look at task manager immediately flagged up stdrt.exe using lots of resources so I got MSSE running while I checked online to find out what this thing was. It's not new by any stretch and looks to be fairly commonplace. MSSE failed to find anything, even when I scanned the executable directly. I had to install Malwarebytes Anti-Malware to remove it (which it did quickly and easily).
I've removed MSSE and set him up with AVG Free and Malwarebytes.
One thing about MSSE I do like is if it finds something it is something for me to perk up about. Unlike some that squawk that you have 1228 files that may be suspicious. You open it up and find out it scanned you cookie folder and called them all suspicious.
Also if I am installing something 'from the wild'. I run it thru one of those mega sites that has all the AV scanners and see what they say about it.
The easiest way to nuke 99% of all infections is a simple NAS/Firewall, adblock, no-script, and disable java/flash if you do not need it. The funny thing is Adblock gets most of it. These days most of the ones I see pop up come thru some 3rd party ad network.
I hope that MS will fix the accuracy of MSSE, for an antivirus is essential to have on Windows (at least for non-technical users) and what the competition offers tends to be heavy, infested with nag screens and unwanted features. Somehow this reminds me of the days of DOS 6, when Microsoft had added a nice built-in antivirus to the OS (MSAV), but then stopped upgrading it, and removed it altogether from later versions of their OSes.
I clicked on DOWNLOADFREEPORNMOVIES1080PHD.EXE, but nothing happened. Could you post that link again?
Public companies have their books audited by external entities to ensure impartiality and objectiveness. Microsoft writing its own anti-whatever fails that test. It is also so typical of Redmond to whine whenever anyone criticizes them. So what if the AV test people run a harness that covers everything? All MS competitors' offering faced the same test, except they passed. I run Norton 360, have done for years, at $60/yr for 3 PCs it's a bargain. None of the issues of false positives, verbose updates etc. that others on this thread report. You get what you pay for, which explains why the MS protection software is free.
So, if all these 0-Day infections are UNDETECTED BY MICROSOFT, then HOW could Microsoft's telemetry show them that the vast majority of its users are unaffected? If Microsoft knew about these things' existence, it stands to reason that it's product would block them.
Independent testing groups hold AV vendor's feet to the fire like a good free press does to politicians'. When caught, both groups tend to respond the same way: deny the problem and accuse the whistle-blower of being out of touch or inappropriate.
The thing of MSSE is that it stays current on it's own. I come across machines running the other products all the time that are months out of date, because someone bot the product one time or just stuck with the trial that shipped with their computer, and couldn't be bothered to re-subscribe later on. With MSSE, there is no risk of that, and for this reason alone I'd rank it above most of the other products.
That said, I give good scores to AVG for the same reason, and to a lesser extent also to AVast (still requires re-registration every 14 months, but at least it's free, which removes one barrier to keeping it current).
Zero-day doesn't stay zero-day. sooner or later a new dat file will detect it... right? Plus with crc checking of system files you should see something is wrong. I get what you are saying but I get what MS is saying. Basically MS is saying a little more than what is true but probably pretty close to the facts...
It's probably not that odd in the internets, you know...
Actually, internet is a gramatically pluralizable word, since you can identify a quantity for it, even if that quantity is one.
File under 'M' for 'Manic ranting'
take malware that is minutes old
I can't help but be cynical and think that minutes old could also mean submitted by their advertisers and supporting companies, minutes ago. How does AV-Test make money, but from the very publishers of the security software that they are supposedly independent from?