NSS Labs Browser Report Says IE Is the Best, Google Disagrees

adeelarshad82 writes "Independent testing company NSS Labs recently published a report on the ability of popular browsers to block socially engineered malware attack URLs. The test, funded by Microsoft, reported a 99 percent detection rate by Internet Explorer 9 beta, 90 percent by Internet Explorer 8, and 3 percent by Google Chrome. However, Google doesn't entirely approve of this report's focus and conclusions. According to Google not only didn't the report use Chrome 6 for the tests, the current version is Chrome 8; it also focused just on socially engineered malware, while excluding vulnerabilities in plug-ins or browsers themselves. Google defended its browser by claiming that it was built with security in mind and emphasized protection of users from drive-by downloads and plug-in vulnerabilities."

Comments

It's Clear to Me Why They Waited

[eldavojohn]

From the response article:

It's not clear why Microsoft and NSS Labs waited until December to release the results.

Maybe it's like the last time this happened?

Furthermore, Moy said, the study started as a private test for Microsoft's engineering team, which was seeking to make internal improvements. "They decided to release it based on the positive results. Many of the test reports we write do not get released by vendors, but they do get used to improve products. So what does 'sponsored' mean in this case?"

So you (internally) strike a deal to test your browser (but also your competitors') with an "independent company" that you pay to perform this service. You get to define the "success parameters" of the test. Then you get the results back and you fix everything. After that time spent fixing has passed, you release the report and add that you have fixed all the problems with your product. Unsurprisingly, you look really really good when this news hits. Since your competitor is not also paying NSS Labs, NSS has no reason to update the report to meet the latest and greatest version of browsers. Meanwhile you can decide if your competitor's browser performed inadequately enough or not for the report -- maybe you even select the success parameters afterward? Heck, you already waited to see if you could release the report.

Independent? HA!

Re:Socially engineered attacks ARE a huge problem

[mcgrew]

The test, funded by Microsoft

That says it all.

Re:Socially engineered attacks ARE a huge problem

[rtfa-troll]

So its results are unquestionably incorrect and/or irrelevant?

They may be technically true in some sense or other. However, in past such situations, Microsoft has been seen commissioning several similar reports; possibly even iterating the instructions for running the reports; then throwing away (under NDA) all the ones which don't match with their marketing wishes. You can basically assume that whatever it says is the opposite of the truth in some way or another because if it was true they would be able to just say directly it instead of commissioning someone else to say it to they can avoid claims of false advertising (for example, their old "Get the Facts" campaign was one of the few things of this type the ASA has clearly stated was misleading). And yes; most companies do this to some extent, but few other companies could come near to sustaining the level of deception Microsoft does because eventually some employee would become disenchanted and start leaking results. For example, have a look at the Comes documents, which only came out because of a lawsuit, to get some idea of the kind of things they can keep secret. Nowadays Microsoft's data destruction policies are much stricter and they ensure that all deals are finalised by lawyers and so are legally privilaged. This kind of secrecy and professional deception means that almost any marketing claim from them should be disregarded completely until there is some level of independent confirmation.