Slashdot Mirror

← Back to Stories (view on slashdot.org)

Microsoft Explains Why Windows Defender Isn't Ranked Higher in New Antivirus Tests (zdnet.com)

Posted by msmash on from the closer-look dept.

In its most recent reports, AV-Test had very few flattering things to say about Windows Defender. Microsoft's security suite was rated as the seventh best antivirus product in the independent test. In total, 15 AV products were tested. Microsoft, however, has now disputed AV-Test's methodology and conclusion. For some context, the top AV products rated by AV-Test on Windows 10 were Trend Micro, Vipre, AhnLab, Avira, Bitdefender, Kaspersky, and McAfee.

Windows Defender was able to detect 100 percent of new and old malware, but it lost few points for performance (which, AV-Test measures on the basis of how a security suite slows applications and websites on the test computer); and usability (which counts false-positives or instances where AV wrongly identifies a file as malicious.) From a report: Windows Defender's performance rating was dragged down because it slowed the installation of frequently used applications more than the industry average, and wrongly detected 16 pieces of legitimate software compared with the industry average of four. But Microsoft wants enterprise customers to know that Windows Defender is only half the picture, given the option for customers to also deploy Windows Defender Advanced Threat Protection's (ATP) "stack components" including Smartscreen, Application Guard, and Application Control.

In the January and February test Windows Defender also scored 100 percent on protection. However it did miss two samples. Since then it's retrained its machine-learning classifiers to detect them. But Microsoft notes in a new paper that Defender ATP did catch them, which isn't reflected in AV-Test's or other testing firms' result. Microsoft hopes to change this so that testers include so-called stack components available in ATP. "As threats become more sophisticated, Microsoft and other security platform vendors continue evolving their product capabilities to detect threats across different attack stages," Microsoft's Windows Defender Research team writes. "We hope to see independent testers evolve their methodologies as well. Our customers need greater transparency and optics into what an end-to-end solution can accomplish in terms of total preventive protection, including the quality of individual components like antivirus."

45 of 85 comments (clear)

  1. Attack surface by sinij · · Score: 4, Insightful

    MS Defender has one very clear advantage over competition - it doesn't create an additional attack surface and installs yet another vendor's application with deep kernel hooks, network connectivity, and an equivalent of root privileges.

    1. Re:Attack surface by Anonymous Coward · · Score: 3, Insightful

      but it lost few points for performance (which, AV-Test measures on the basis of how a security suite slows applications and websites on the test computer);

      I would like to know which non-Microsoft AV is this polite. Long, long ago, McAffee was a minimal AV option, but then it joined Norton and all the other "security suites" as a bloated and unwieldy mass of advertising other McAffee products and panicing over 1st party software patches.

    2. Re:Attack surface by Sarten-X · · Score: 1

      Along similar lines, AV hooks are one of the common causes of system instability, usually blamed on something else, like browsers or Windows itself.

      --
      You do not have a moral or legal right to do absolutely anything you want.
    3. Re:Attack surface by phantomfive · · Score: 5, Informative

      it doesn't create an additional attack surface

      Unfortunately, yes it does.

      --
      "First they came for the slanderers and i said nothing."
    4. Re:Attack surface by danbert8 · · Score: 4, Interesting

      I use Windows Defender because it's the only AV that isn't worse than the viruses it is supposed to be protecting against...

      --
      Yes it's an anecdote! Were you expecting original research in a Slashdot comment?
    5. Re:Attack surface by sinij · · Score: 2

      Key to understanding my post is "additional". Defender isn't categorically better than other AVs, but you are not giving additional access to a third-party into your system. That is, MS already has that level of access. Plus, since they wrote OS, Defender will play nice with it.

    6. Re:Attack surface by butzwonker · · Score: 3, Informative

      An additional attack surface is one that exists if you install and run the software but doesn't exist when you don't install or run the software. Microsoft Defender adds an additional attack surface like any other antivirus software.

    7. Re:Attack surface by omnichad · · Score: 2

      I had a customer last week where every time they switch user or log off, the entire graphics subsystem shuts down and the monitor goes to sleep instead of showing the login screen. Turns out it was caused by Avira antivirus.

    8. Re:Attack surface by Riceballsan · · Score: 3, Insightful

      so by surface, you mean company? Windows defender is an attack surface, in the sense that it is a piece of software with admin access that rests in addition to the OS as a whole and can in some situations be tricked into doing bad things. If you install bitdefender or something else they generally disable windows defender, which closes down those possible attack vectors, and replace them with whatever the other protection's vectors are. No matter what protection you are using, you've got the same number of attack surfaces, it's just that all attack surfaces are owned by the same company, instead of by 2 companies.

    9. Re:Attack surface by epine · · Score: 1

      An additional attack surface is one that exists if you install and run the software but doesn't exist when you don't install or run the software. Microsoft Defender adds an additional attack surface like any other antivirus software.

      He/she meant an additional attack surface beyond the necessary entailment of the category itself. In language, "additional" can be deployed anywhere along the semantic chain, so long as the situation can get worse, or worser, or worstest.

      However, depending on how Microsoft manages their AV team, it's not obvious that receiving code from this team is inherently better than receiving code from a third party. You'd think the internal lines of communication and access to deep information would help, but golly, when did that ever show up in Microsoft product quality in their first three decades, when the entire corporate ethos was "ship it now, fix it later" (if luck would have it that a fix remained feasible)?

      I think the actual risk accruing to adding a third party to the mix is that it brings yet another aggressive business model to the table, and that AV software isn't in the business of protecting your computer, so much as in the business of making you believe that its your constant, faithful companion. Thus they have an inbuilt incentive to abuse the API in a way that Microsoft hasn't precisely catered to, and doesn't dogfood themselves.

      But this is all pretty speculative. Surface is just a proxy, after all.

    10. Re:Attack surface by drinkypoo · · Score: 1

      Along similar lines, AV hooks are one of the common causes of system instability, usually blamed on something else, like browsers or Windows itself.

      The design of windows itself is a more common cause. And now Chrome has antivirus capabilities, so it's both a browser AND an antivirus and can fuck up your computer in both ways

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    11. Re:Attack surface by bobby · · Score: 1

      I've never been much of a fan of any AV product, but sometime in the last 2 years I discovered McAfee "Real Protect". It seems to work as described. Basically instead of scanning everything you do, it just watches critical system files and disk areas and flags suspicious activity. I've had it alert me a few times, and it was spot-on. Quite happy with it. (Windows Defender still running too, with no detections that I remember).

    12. Re:Attack surface by arglebargle_xiv · · Score: 1
      That's what I was wondering too:

      it lost few points for performance (which, AV-Test measures on the basis of how a security suite slows applications and websites on the test computer)

      given that the scale for the metric "Fucks up your computer's performance" is rated from 1 to McAfee.

    13. Re:Attack surface by drinkypoo · · Score: 2

      Chrome doesn't hook into the kernel like normal AVs so can't do more than usual userspace programs.

      Yes, all it can do is destroy your data, the only thing of value on your computer. How silly of me.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
  2. Optics by phantomfive · · Score: 1, Funny

    Our customers need greater transparency and optics

    Oh, they are laying fiber now?

    --
    "First they came for the slanderers and i said nothing."
    1. Re:Optics by TeknoHog · · Score: 1

      Our customers need greater transparency and optics

      Oh, they are laying fiber now?

      Is that a euphemism for a healthy poop?

      --
      Escher was the first MC and Giger invented the HR department.
  3. What you really need? by The+Fat+Bastard · · Score: 2

    I have Malwarebytes Anti-Malware Scanner and Windows Defender installed on my Windows systems at home. I haven't had any issues since the Windows XP era.

    1. Re:What you really need? by ctilsie242 · · Score: 2

      I have uBlock Origin, SandboxIE, and virtualization. This has kept bad things at bay since the early 2000s. An ad blocker does more for security than most AV programs (which usually are good enough to catch older stuff, so better than nothing.) Of course, virtualization and sandboxing ensures that stuff that gets out is well contained.

    2. Re:What you really need? by Bearhouse · · Score: 1

      +1 for Malwarebytes if you *have* to use Windows

  4. New math? by Anonymous Coward · · Score: 1

    In the January and February test Windows Defender also scored 100 percent on protection. However it did miss two samples

    So which is it? 100% or missed two samples? Because I can tell you my kid doesn't get 100% if she misses 2 questions on an exam.

    1. Re:New math? by Pascoea · · Score: 1

      I can tell you my kid doesn't get 100% if she misses 2 questions on an exam

      What if they are grading on a curve? Granted, It's like saying "A condom works 100% of the time, except when it doesn't."

    2. Re:New math? by Rhipf · · Score: 1

      I don't know how many items were used in the test but for an antivirus test it could be thousands. If Defender missed two of two thousand it is close enough to 100% to just use that figure. They could have said it was 99.9% in that example but as the number of viruses tested increases the easier it is to just say the test was 100%.

    3. Re:New math? by Riceballsan · · Score: 1

      It may be easier to say... but it kills a lot of the meaning. phrases like "over 99%", or 99.99% or something like that carry a HUGELY different connotation than 100%. Perfect is a very specific thought in peoples mind, IE they couldn't find something it couldn't handle. There's a reason even Lysol doesn't advertise as killing ALL germs, even though from what I understand the .01% that it misses are litterally just germs it doesn't touch.

    4. Re:New math? by Anonymous Coward · · Score: 1

      I worked for a company that did this sort of thing for a couple of years.
      The had over 32K signatures in their database.

  5. Hard to believe by Comboman · · Score: 1

    it lost few points for performance (which, AV-Test measures on the basis of how a security suite slows applications and websites on the test computer); and usability (which counts false-positives or instances where AV wrongly identifies a file as malicious.) . . . . . . But Microsoft wants enterprise customers to know that Windows Defender is only half the picture, given the option for customers to also deploy Windows Defender Advanced Threat Protection's (ATP) "stack components"

    I have a hard time believing that adding additional components with additional functionality will speed up performance. Experience tells me the opposite is far more likely.

    --
    Support Right To Repair Legislation.
  6. Relative rankings mostly worthless. by Anonymous Coward · · Score: 4, Insightful

    Anyone should understand that Relative rankings are mostly worthless. If all the products in the top 10 are excellent, but one product has slightly less points than the top 9, does it really matter than it ranked 10th?

    The main advantage of Windows Defender is it's free. For most people that trumps all the other rankings. It's free, it protected against everything the competition did, it's nearly as usable, and slightly slower. That's good enough to not buy something else.

    The AV vendors should be quaking in their boots. Why would you buy another product when what MS puts out is generally fine? My guess is they'll improve the usability a bit, and they'll rank in the top 3. Then start saying goodbye to several of the other AV vendors.

    1. Re:Relative rankings mostly worthless. by ranton · · Score: 3, Interesting

      The AV vendors should be quaking in their boots. Why would you buy another product when what MS puts out is generally fine?

      One reason is because many users have learned they should pick an anti-virus software suite every time they go to Dell and order a new computer. Retailers have an incentive to only offer paid versions because they will get their cut. So many users will keep on choosing either McAfee or Norton just because those are options they are given.

      I'm not sure how many users this describes, but my guess is a lot of them. Then again any significant loss is sales should have them quaking in their boots.

      --
      -- All that is necessary for the triumph of evil is that good men do nothing. -- Edmund Burke
  7. Where am I? by houghi · · Score: 1

    I also filter out 100% of the spam mails. I am even close to 160% depending on how many spam I get. I just point everything to /dev/null in .procmailrc .

    --
    Don't fight for your country, if your country does not fight for you.
  8. I think I am done with Slashdot by Anonymous Coward · · Score: 1

    Showing GDPR acceptance plea in barely readable greyed out text in a box that covers whole screen is a new low for this site which posts articles about security and privacy.

    How ironic!

    KTHXBYE!

  9. Low score by PPH · · Score: 1

    Couldn't keep Windows off my machine.

    --
    Have gnu, will travel.
  10. AVERAGE security by franblets · · Score: 2

    I am not defending MS here - but who wants to be compared to industry _averages_ when it comes to security. The people adjusting the ranking because it does not compare well to an average are what I like to call stuupid (it is not a typo). You should want perfect security - to hell with averages.

  11. Because it doesn't slow the system by thegarbz · · Score: 1

    Virus scanners are judged on how well they completely cripple a target system. Windows Defender doesn't do that so it just isn't any good.

    Oh and First post. Or at least it would have been if I wasn't running McAfee.

    1. Re:Because it doesn't slow the system by Anonymous Coward · · Score: 2, Informative

      The test is wrong somehow or misleading somehow. The fact that they try to lay AVG and McAffee in the same performance hit flies in the face of all anecdote I've collected over the last few years working on BYOD and personal Windows computers.

      McAfee and AVG are FAR slower than Defender. It is true that I have not done objective testing on the software, but I've consitently observed a "Before and After" effect with both AVG and McAfee (un-install only with McAfee .. ) while installing or un-installing them from people's computers. Defender has *never* had that kind of effect. They all obviously slow the computer when actually doing a "full-scan," but during normal operation with the realtime scanning active, they're not even close.

    2. Re:Because it doesn't slow the system by thegarbz · · Score: 1

      I find there's a difference between checking a few files in transit as they are loaded, and whatever it is that McAfee is doing that is using up an entire core of CPU for the best part of an entire day.
      I'll take 30seconds longer to install software any day over whatever shit my work computer thinks it is doing for my protection.

    3. Re:Because it doesn't slow the system by gravewax · · Score: 1

      no, it lost points only during installation process of apps, that is hardly a common task, I would hazard a guess it isn't even 1% of what a computer is generally doing and when they are they don't give a shit if that 2 minute install takes an extra 5 10 or even 60 seconds.

  12. defending defender... by Anonymous Coward · · Score: 5, Interesting

    Ok, direct experience here, and I am absolutely no fanboy of ms software. But, as part of a offensive security cert a few months back, I got heavily into writing and compiling windows exploit code, and one of the course exercises walk through testing a piece of malware by the virus total site.
    So as part of my studies and self learning I wrote a non self propagating malicious exploit, but it did elevate privileges from the user to admin and get access to things and start calc as a admin user to prove it was exploiting. I took a common windows POC exploit and modified it heavily in ways I will not discuss to a wider audience (because teaching people av evasion techniques is best left to offsec and their ilk, to the right people) and compiled it.
    Out of sheer curiosity I submitted the original POC code, one encoded by a old common packer & my heavily modified "malware" to virus total, and the original and encoded packed version was picked up by about 45/47 av's straight off. The *ONLY* av that managed to detect my custom payload was.... Windows Defender. It must have opened the executable and saw where it hooked when it shouldn't, and the competition seem to rely on pattern matching instead.
    So yeah, sign me up for free windows defender. When the subject comes up with lay people who ask me what to use, its what I would recommend them. From first hand testing.

    Anon, because even with all the above, I'm basically admitting to authoring a custom exploit, and while I'm employed in this field, I could do without the extra attention.

  13. I'm not the biggest fan of Microsoft... by QuietLagoon · · Score: 2
    (as regular readers here may note)... but... so what. Windows Defender was ranked 7th seems to be the big takeaway in the summary. What if the top 10 are all good to use, does being 7th really matter? I've been using Windows Defender for a couple of years (when Avast started their annoying desktop pop-up adverts that I could not disable, I switched to Windows Defender).

    .
    Additionally, Windows Defender does not seem to install all manner of additional software that digs deep into the Windows kernel in order to do its job. For my needs, Windows Defender is a simple, effective a/v solution that works well. Why should I care if it ranks 7 or 3 of even 1?

  14. Not an issue for 2018 by Billly+Gates · · Score: 1

    Things have improved tremendously since the Windows XP era in terms of Windows and app security. Also people tend to use adblockers and flash isn't on by default on newer systems.

    Adobe now has sandboxing and Windows gets new security updates each month. IT departments now update software regularly and people use ancient IE almost never outside of a Citrix or vdi environment.

    The use of AV software to protect idiots who click on everything is unheard of as people know better now than in 2000.

    --
    http://saveie6.com/
  15. MS knows how to harden Windows ... by Big+Bipper · · Score: 1

    Defender proves this. So why doesn't Microsoft just sell a hardened Windows ? Why sell an insecure product and then addon security ?

    --
    You live and learn, or you don't learn much.
    1. Re:MS knows how to harden Windows ... by gravewax · · Score: 1

      Defender is built in, it isn't an addon. To use the other vendor products you are removing Microsoft's hardening and adding another vendors.

  16. nice info by CứngĐờ · · Score: 1

    nice info tkss bro https://ordertrungquoc.com.vn/

  17. You don't need an AV by Bearhouse · · Score: 1

    If you keep your HOSTS file updated regularly!
    Must be true, I read it here!!!!!

    *ducks*

  18. AV publishers pay PC makers a commission by tepples · · Score: 2

    So what's the incentive for Dell to keep including this option?

    You answer your own question:

    If the license is already free for Dell, just start asking for money from the AV vendor to install their product

    So the incentive is the same as that for any of the other "bloatware" or "trialware" included on most Windows PCs or Android phones: the AV publisher pays Dell gets a commission on new installs. You'll notice that Windows 10 Signature Edition PCs and Google Pixel phones, which specifically exclude third-party bloatware, carry a higher MSRP because the manufacturer isn't getting that sweet, sweet commission revenue. The same is true of PCs including a free operating system. I looked on Dell's website a couple months ago, and an XPS 13 with Ubuntu cost $50 more than an XPS 13 with identical specs and Windows 10. Again, no commission.

  19. MS Defender Less Trustworthy than Kaspersky??? by TomGreenhaw · · Score: 1

    If anything has complete keys to your PC kingdom, it would be anti-virus software.

    With everything going on in the news today, how can anybody truly trust any of these solutions? If you think that there isn't a cold war going on in the internet, you are uninformed. I see it for myself in logs and honeypot activity all the time. I have to chuckle every time I hear somebody swear by an antivirus program because it doesn't cause trouble and they think they aren't getting infected. I do however expect more of people who test and rank this stuff.

    If an American has made the decision to trust Microsoft Windows, why would they expand that circle of trust to a company whose headquarters is in Moscow? By that same logic, if you trust a Russian company more than an American company, it would be logical to use Kaspersky.

    --
    Greed is the root of all evil.
  20. uhh.. by SuperDre · · Score: 1

    what kind of testing methods did the 'independent' AV-test use, as my own experience with Trend Micro and Kaspersky is they are CRAP and have a very big impact on the performance of your computer. Trend Micro is really rubbish, if an application deletes multiple files after each other (using simple API calls) it immediatly removes it without warning, even though there is not a single virus/malware signature in it.