Slashdot Mirror
NSS Labs Browser Report Says IE Is the Best, Google Disagrees
adeelarshad82 writes "Independent testing company NSS Labs recently published a report on the ability of popular browsers to block socially engineered malware attack URLs. The test, funded by Microsoft, reported a 99 percent detection rate by Internet Explorer 9 beta, 90 percent by Internet Explorer 8, and 3 percent by Google Chrome. However, Google doesn't entirely approve of this report's focus and conclusions. According to Google not only didn't the report use Chrome 6 for the tests, the current version is Chrome 8; it also focused just on socially engineered malware, while excluding vulnerabilities in plug-ins or browsers themselves. Google defended its browser by claiming that it was built with security in mind and emphasized protection of users from drive-by downloads and plug-in vulnerabilities."
It's not clear why Microsoft and NSS Labs waited until December to release the results.
Maybe it's like the last time this happened?
Furthermore, Moy said, the study started as a private test for Microsoft's engineering team, which was seeking to make internal improvements. "They decided to release it based on the positive results. Many of the test reports we write do not get released by vendors, but they do get used to improve products. So what does 'sponsored' mean in this case?"
So you (internally) strike a deal to test your browser (but also your competitors') with an "independent company" that you pay to perform this service. You get to define the "success parameters" of the test. Then you get the results back and you fix everything. After that time spent fixing has passed, you release the report and add that you have fixed all the problems with your product. Unsurprisingly, you look really really good when this news hits. Since your competitor is not also paying NSS Labs, NSS has no reason to update the report to meet the latest and greatest version of browsers. Meanwhile you can decide if your competitor's browser performed inadequately enough or not for the report -- maybe you even select the success parameters afterward? Heck, you already waited to see if you could release the report.
Independent? HA!
My work here is dung.
This is all true and no one should really have a problem with it unless(until?) Microsoft starts marketing it as more than it is(essentially suggesting that IE9 blocks 99% of malware with the small print saying it only applies to social engineering)
Google is complaining that a report on socially engineered attacks is only focused on socially engineered attacks? And they're whining that a study done back when Chrome 6 was the most recent release doesn't mention Chrome 8, which is currently the most recent release? Seriously?
According to Google not only didn't the report use Chrome 6 for the tests where as the current version is Chrome 8...
Should it be:
According to Google not only did the report use Chrome 6 for the tests, whereas the current version is Chrome 8...
I'm well aware of what social engineering is, but what are "socially engineered malware attack URLs"? Those things that pop up in my inbox say "check out this picture of us!" with a link that looks like someone smashed their head on the keyboard?
The test, funded by Microsoft
That says it all.
Free Martian Whores!
As independent as a politician that accepts campaign contributions from AT&T or SEIU.
An "independent" test that was "funded by Microsoft". WTF? How is that independent?
This: "The test, funded by Microsoft"
The real warning flag is that it doesn't say that on NSS Lab's site nor does it say it anywhere in the report. So if I was being paid to do this, I would have that in big bold letters as a disclaimer on the front page of the report if I wanted to maintain credibility. So either the Google response article is wrong (which the same IE8 report from last year was funded) or you're just being flat out disingenuous when you say "independent." We just happen to receive funding from one of the participants and they decide when and if the report is released.
One more thing, if you dig into this report, the parts where they reference Microsoft read like an advertisement:
It became obvious from this test and comparisons to the earlier test that Microsoft continues to improve their IE malware protection in Internet Explorer 8 (through its SmartScreen® Filter technology) and in Internet Explorer 9 (with the addition of SmartScreen application reputation technology). With a unique URL blocking score of 94% and over-time protection rating of 99%, Internet Explorer 9 was by far the best at protecting against socially-engineered malware. The 89% zero-hour block rate suggests a far superior malware identification, collection, and classification method.
"What kind of registered application reputation technology did you say they used? Simply revolutionary progress!" Compare that section to that same section on Chrome:
With a protection rating of just 3%, Chrome 6 dropped more than 14% from our last test. And, Chrome’s unique URL score of 4% was also a major decline. Chrome’s overall poor protection makes it difficult to compare it to other Safe Browsing API-related products.
"Boo, Chrome sucks!" Hahaha oh my this is too funny. Google shouldn't have to explain themselves. Just take what you can to improve from this report, become aware of your opponent's tactics and move forward.
My work here is dung.
So its results are unquestionably incorrect and/or irrelevant?
Looks like the test was a perfect example of social engineering.
They certainly cannot be considered "independent" or "unbiased" at a minimum. So they aren't of much value until real 3rd party tests are performed.
You mean the story on the front page isn't enough? http://tech.slashdot.org/story/10/12/15/1822216/Todays-WikiLeaks-News
Or the one from last night - http://tech.slashdot.org/story/10/12/15/0038211/Air-Force-Blocks-NY-Times-WaPo-Other-Media
Or the one from yesterday - http://idle.slashdot.org/story/10/12/14/1612247/Julian-Assanges-Online-Dating-Profile-Leaked
Or the other one from yesterday - http://tech.slashdot.org/story/10/12/14/168248/Michael-Moore-Posts-Julian-Assanges-Bail
Or the two to four a day we've had for days?
If anything /. has too much about Wikileaks right now, Reddit is slammed with it as well
Your ass is safe, for now.
According to Google not only didn't the report use Chrome 6 for the tests where as the current version is Chrome 8
dude, really? couldn't you have said it without using a double negative?
Seriously. What were they even testing? I was under the impression that social engineering was a security flaw in the user, not in the application. Reading the report, it sounds like they were just testing the browsers' databases of known malware/phishing sites. Which, really, has little to do with the security of the browser itself.
...researchers discovered that hot supermodels would be most fulfilled in a relationship with Slashdot user GodfatherofSoul*.
* This study funded by GodfatherofSoul
I swear to God...I swear to God! That is NOT how you treat your human!
So Google is not quite as bad as MS, but complaining that a reviewer used an old version is a tried and true attempt at diverting attention for genuine deficits in the product.
"She's a scientist and a lesbian. She's not going to let it slide." Orphan Black
Do you value the "UL Listing" on electrical gear that you buy? I certainly take that as an assurance that stuff won't just randomly catch fire. All UL Listed testing is paid for by the vendor - and vendor-paid testing is normal in the real world.
This test may be a crock, but you can't just assume that from the fact that MS paid for it. The simple fact is: anyone competent to test browser security probaly has a strong opinion about MS, and pretty much anyne will have a reason to be biased. The professionalism of the tester is what matters, not the existance of a reason to be biased.
Socialism: a lie told by totalitarians and believed by fools.
"microsoft funded". Google could by rights fund a test of the current Chrome version against IE7/IE8 version from one or two years ago unpatched.
They would have had to intentionally install a old version of chrome with a standalone installer, and prevent it from updating by circumventing google updater which silently updates chrome. Talk about stacking a test.
After logging in slashdot still does not take you back to the page you were on. It's been that way for 20 years.
Data points: IE 9 gets 99%, Chrome gets 3%, Funded by Microsoft. What a beautiful line.
No I haven't looked at how the study worked, on Slashdot being first is better than being right.
So its results are unquestionably incorrect and/or irrelevant?
They may be technically true in some sense or other. However, in past such situations, Microsoft has been seen commissioning several similar reports; possibly even iterating the instructions for running the reports; then throwing away (under NDA) all the ones which don't match with their marketing wishes. You can basically assume that whatever it says is the opposite of the truth in some way or another because if it was true they would be able to just say directly it instead of commissioning someone else to say it to they can avoid claims of false advertising (for example, their old "Get the Facts" campaign was one of the few things of this type the ASA has clearly stated was misleading). And yes; most companies do this to some extent, but few other companies could come near to sustaining the level of deception Microsoft does because eventually some employee would become disenchanted and start leaking results. For example, have a look at the Comes documents, which only came out because of a lawsuit, to get some idea of the kind of things they can keep secret. Nowadays Microsoft's data destruction policies are much stricter and they ensure that all deals are finalised by lawyers and so are legally privilaged. This kind of secrecy and professional deception means that almost any marketing claim from them should be disregarded completely until there is some level of independent confirmation.
=~ s,(.*),<sarcasm>$1</sarcasm>,g if any_point_you_wish();
UL is to test your products for saftey, this is a *comparative* test against several competing products for quality.
Apples, meet Oranges, meet troll.
a handful of selfish greedy people are no match for millions of selfish, greedy people -u4ya
The report is almost useless because it has compared the latest stable and dev releases of IE with versions of Firefox and Chrome that are years old.
To use a car analogy, it is comparing the safety features of a '10 Chev Corvette and a 1970 Chev BelAir. I would be embarrassed if the company I worked for released such a report.
Just what exactly *is* 'socially engineered malware' ?? which is apparently 'actually a huge problem currently.' ?
I'm curious to know?
In what was is it different to any run of the mill link that attempts to exploit browser vulnerabilities?
Most of which I believe are fixed by the browser vendors pretty quickly the minute they're known about.
Otherwise this whole study seems like a made up problem which is a bit of a non-issue and which appears to be
miraculously solved by only one vendor. Unsurprisingly the sponsor of said report.
don't be a spelling loser
The test, funded by Microsoft
That says it all.
And the response from google criticizing it was by someone right on google's payroll representing google's interests. I guess we can ignore their criticism then too?
Or perhaps we should let the work stand for itself, evaluate the methodology, strip away the marketing spin, and come away with some nugget of truth, regardless of who funded it. Of course that's "work".
Tests like this are done for marketing purposes. The professionalism of the tester will make sure the test is rigged to give Microsoft the result they want. Get the facts.
UL testing isn't a product comparison, it's a test for standards conformance. The requirements for independence and impartiality are substantially different.
The test has an odd kind of validity; The foolish who choose Internet Explorer (instead of Firefox, Chrome, Safari or Opera) would be also the foolish victims of "Socially Engineered Malware". That is, the web browser for dupes protects its users from the same vulnerability which causes them to use it.
Ceci n'est pas une signature.
I think you missed the other important part: "Also, the version of Chrome that NSS says all this about is two major versions behind the current stable release, while the version of IE they say is better is the current beta release."
A more relevant comparison would be IE 8 to Chrome 8 (current generally release version of both version), or IE 9 to Chrome 9 (current publicly available pre-release version of each browser.)
Perhaps someone should do a similar comparison, but using Chrome 9 and IE 6, instead...
The report is of greater value to Microsoft, the paying customer, the less obvious it is the Microsoft is the paying customer.
I don't know about you but I rarely receive tarballs, rpms or debs from friends to compile or install on IM or facebook. That's the good thing about the repository system, where there is a (hopefully) trusted source where you install the majority of your applications.
I can't really see socially engineered malware taking off under Linux, really.
:. Ultimate Control Dedicated/VM Servers
I work for UL. you don't know shit - UL's tests and the kind of stuff going on here are entirely different.
you can actually reproduce UL's tests, and they aren't out there to "compare to another company".
It'd be more like this:
NSS labs browser report says IE blocks 99% of social networking vectors.
Nothing about "in comparison to chrome", or "excellence", or how well it does. Yet all of those are in the study.
In fact, it's incredibly unethical to comment on the performance of a product as a testing studio as good, bad or otherwise. That by itself in the studies guarantees you that these studies are biased due to the funding.
[I'll suppose that you were being facetious, but my sarcasm detector is in the shop---]
Nope, that merely gives you reason to question the outcomes and examine the experimental procedure in depth. It's a meta-level reputation system. If an entity has shown a lack of bias in the past, you can generally choose to accept their work. Otherwise you examine the experiment design and see if anyone was playing fast and loose with the statistics and analysis. Microsoft probably qualifies for most people.
The summery raised a few points. It may be that some older version of Chrome is crap in some aspect compared to the beta of Microsoft's latest and greatest. That's a simple fact that can be supported by some measurement. You can question whether what is being measured is actually useful, but let's assume that it is. Still doesn't change the fact that it's and old version of Chrome vs. a beta version of IE. The study is still perfectly valid, just utterly pointless. It would be like a study on carbon emissions that only looks at cars made during the 60's. Utterly pointless for the current situation.
Skepticism is fine, but be a good skeptic who evaluates the experimental methods and study conclusions. Don't call a club a spade just because it suits your black-and-red view of the world. Otherwise you're really no better than what you purport to despise.
I know this isn't in the spirit of the other posts on this topic today, but I applaud MS for concentrating on security and the best interests of their end users. It's good to see they are taking these matters seriously as part of the product development process.
Don't get me wrong, I'm always happy when security is improved -- even in the most hated of products by the most hated of companies. The problem I have is when marketing gets a hold of this and spins it to attack competitors, thereby improving the public perception of their own product. This could have all been avoided had Microsoft just kept the report internal like most of NSS Labs' customers. And doing so while comparing the latest IE9 to Chrome 6 and releasing that to the public as a 'current' report now ... well, that's what I have a problem with. If a Chrome user read that report as today's news they're going to think that it's been done with today's Chrome.
My work here is dung.
If I want to know what is unlikely to burn my house down, I look for the UL listing, and rely on vendor-performed standard tests.
If I want to know whether product A or B is better, I check out Consumer Reports, which accepts no funding from any vendor, not even advertising.
I was willing to believe that IE wouldn't burn my house down anyway, so this report gives me precisely no useful information.
"When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes
If UL tests 2 products, and finds one passes and another fails, there's certainly a comparison that can be made between them, and a company selling the passing product might feel inclined to draw attention to this (of course, UL itself never comments publicly on failed tests). In this case, the tester tested two products and rated one "99%" and one "3%" against some standard. The methodology might have been totally bogus (no idea), but the act of paying for the test isn't automatically so.
Socialism: a lie told by totalitarians and believed by fools.
I must have missed the part where Net Applications is a shill for Mozilla, Google, and/or Apple.
The credibility issue here is with a Microsoft. A company that has been shown, time and again, that they're not above tweaking the facts (lying) about their products and their competitors' products. That, and the fact that they paid for this supposed bit of research.
"The test, funded by Microsoft..."
That told me everything I needed to know.
MCSE? No, sir...I don't do Windows. Yes, I am an idealist. What's your point?
When one uses only a single test, perhaps a specially crafted one, the conclusions may be misleading.
As an extreme example if one takes an area of a country where people are very well fed, and perhaps taking in far more nutrients than needed, it is entirely possible that one could come up with a study showing substantial nutritional value in sewer waste. Without taking into account the other characteristics (bacteria, viruses, levels of toxic medications, smell etc.), sewer waste might actually be portrayed as a good inexpensive source of nutrition.
I don't mean to pass any judgement on whether IE is wonderful or terrible. The point is that one narrow measure should be kept in perspective and not used as the sole basis for an overall opinion.
What the Faceless Google rep said was that this test cannot be peer reviewed because they did not release all the data (specifically the URLs visited). Now releasing a report that does not allow for independent review does not make for good science.
The tests may be valid. But until there is enough information to confirm this, I can only be skeptical of the faceless Microsoft rep.
A sig is placed here
To display how futile
English Haiku is
Woah.
I haven’t seen style this terrible in a long, long while. Even the GNAA trolls are more legible.
Ignore this signature. By order.
It raises a red flag, but that is all. They could very well be unbiased and independent.
Yes, like all tests, confirmation from others is a good thing.
Look at the data. compare to the conclusions. Do the match the conclusions? Is the methodology the correct one for the tests they are doing?
That's the only way to tell if a study is good.
The Kruger Dunning explains most post on
I would love to see a study funded by X that does not then show X as being the best product. Given that it seems $ > Truth, I doubt such a thing will ever happen.
This is totally different.
In this case, the tester tested two products and rated one "99%" and one "3%" against some standard.
The key difference is that UL tests against a pre-existing standard. Not a standard that they made after looking at the product. UL can't customize their test to make one product look better or worse.
The methodology might have been totally bogus (no idea), but the act of paying for the test isn't automatically so.
The act of paying for a test to be designed for you, or a test you designed ahead of time to make your product look good, is bogus. Paying to have a test executed for you is not bogus. One is independent, the other is not.
Browsing is something that occurs OFF the business desktop and NOBODY TRUSTS MICROSOFT not to rat them out to the corporate IT department.
What does this even mean??
"16MB (fuck off, MiB fascists)" - The Mighty Buzzard
The report is almost useless because it has compared the latest stable and dev releases of IE with versions of Firefox and Chrome that are years old.
What. No, wait, what?
Read on to the end, because later I'm going to tell you what's really wrong with the test and why it's bullshit, but I have to first burn down the obvious straw man you've introduced.
The report was released in October 2010. http://www.nsslabs.com/assets/noreg-reports/NSS%20Labs_Q32010_Browser-SEM.pdf
It used Google Chrome 6, which was the current stable Chrome at the time (6 came out in September 2010). Google Chrome has gone from 6 to 8 in two months. It used Firefox 3.6, which is the current stable Firefox RIGHT NOW, two months after the report was released. 3.6 was released in January 2010, but Mozilla has only done "dot" releases since October. It also included Internet Explorer 8, which was released in March 2009.
In other words, if you want to say "older is worse", then IE8 should have been absolutely fucking pasted by this test. Ummm, right? It's the oldest browser in the test by almost a year.
Now we get to the point that won't upset you, because THIS is what is wrong with the test.
According to their test, what they were really testing was vendor responsiveness to known threats (on-time maintenance of the blacklist), not some response internal to the browser. They took a bunch of really recent entries of bad sites from someone and plugged them into the browsers, getting a new batch of URLs every few hours. The time was measured in hours, so what this is really saying is that Microsoft seems to be the best vendor at maintaining the server-based "bad URLs" list, though it took them 4 hours on average to block sites as opposed to Firefox's 6 hours.
If they got these sites from their paid sponsor, then the list could easily have been biased. But there's more actual provable bias to the test than just that.
The real bias is in the percentages. They do not actually represent "Microsoft browsers blocked 90% of sites while Firefox only blocked 20%". they are a grade-type score, where 100% means all sites were blocked immediately, while a 0% means no sites were blocked, ever. Early detection (measured in hours) seems to play a much larger role than actual number of sites detected. The scores appear to have been done on some form of normalization curve, with the sweet spot being somewhere around "One Half Hour Longer than Internet Explorer".
Otherwise, how does an increase in response time from 4 hours (IE, both versions to within a few minutes plus or minus) to 6 hours (Firefox) make your score go from 90% to 20%?
The net conclusion is, if you're going to use a web browser and you depend on vendor-maintained "baddie" lists as your primary line of defense (rather than script protections like NoScript, which don't depend on a vendor to maintain stuff for you), you're better off with Internet Explorer than any other mainstream browser in the market.
It doesn't make you "70% safer" or protect you from "70% more threats", it means that it has, on average, 2 hours of lead time on the next-best browser in terms of the list of sites it protects you from. It's like saying that McAfee is better than Norton because McAfee generally releases specific virus signatures, on average, 2 hours before Norton does.
So, the test is correct, it's just expressing the results in a very misleading way, showing a very low number for "everyone but Microsoft" because the test results were designed to score what IE did best in the highest way possible. They even spelled that out in their results:
The value of this table is in providing context for the overall block rate, so that if a browser blocked 100% of the malware, but it took 264 hours (11 days) to do so, it is actually providing less protection than a browser with a 70% overall bloc
"This post contains words, known to the State of California to cause thought. Wash brain thoroughly after reading."
Depends on how the funding takes place, and for what purposes. Did they fund This test? DO they just make a annual payment to a generic fund to be part of the 'club'? Are the a testing lab where everyone knows the test is paid for by the vendor*?
*UL safety testing is paid for by the vendor, at it works very well/ Different kind of testing, but hopefully you see my point.
The Kruger Dunning explains most post on
Too bad the said Microsoft paid for the test. They even put it where it goes in ALL tests.
Anyone who reads these tests know exactly where to look for funding. IT was NOT hidden.
4.4 ABOUT THIS TEST
This private test was contracted by Microsoft’s SmartScreen product team as an internal benchmark,
leveraging our Live Testing framework. It has subsequently been approved for public release.
The Kruger Dunning explains most post on
Lame troll is lame. How many boxes do YOU repair in a week? I average about 6, sometimes more when I've not got so many builds in progress. And I can tell you that without a shadow of a doubt that socially engineered attacks account for a HUGE amount of infections and is in fact growing rapidly. try looking up "Security Tool 2010" or "Rogue AV 2010" and looking at the numbers these things are racking up. As home users slowly move away from XP to Windows 7, which has file and registry virtualization, ASLR and DEP, and which you can even easily add Structured Exception Handling Overwrite Protection , the low hanging fruit is increasingly becoming PEBKAC. I can tell you I see socially engineered bugs spreading a hell of a lot faster on newer OSes than I do anything else, whereas with XP it is still drive bys thanks to running as admin. As XP dies out this problem will only be getting worse.
Now I don't recommend IE OR Chrome to my customers, as I don't like the data mining in Chrome and have had bad luck in the past with IE, if MSFT can get 99% of the social engineered bugs blocked, along with someone cooking up something like ABP for IE 9? Then I'll be happy to recommend my customers use IE over other browsers. I'm already starting to get pissed at Mozilla for refusing to support low rights mode in Windows 7 even though this tech has been out since 07 simply because Linux doesn't have it. Chrome mines waaay too much data for my taste, so that leaves Comodo Dragon and IE. Does anyone know of a good ABP for Chromium based browsers? Or an ABP for IE 9? Because in the end ABP will be the deciding factor for me and my customers. If IE 9 can block 99% of the social engineered attacks while I can block ad based attacks with an ABP clone then it is a no brainer to switch. I just wish the Mozilla team wouldn't act like asses and refuse to support a technology that would help protect so many simply because it isn't supported on a platform that doesn't need the damned thing anyway.
ACs don't waste your time replying, your posts are never seen by me.
Posting as AC so I don't get fired.
This one originated within the team itself and was not a marketing exercise.
I work on a different product group but the last 18 months or so the whole company has been encouraged to put their products through outside testing during alphas so that the goals are useful to the market rather than simply focused on what the marketing and research drones came up with. This test is one of those, the purpose was to see how good the anti-malware protection was, Chrome & IE scored about the same (FF is lagging) across the board but IE did significantly better in this single category (with google doing better in all manner of download exploits to balance things out).
While I don't think they should have released the results the method is actually sound, product teams are fucking awful at looking at competitor products and recognising where they do better. Going outside for advice refocuses us and gives a much clearer picture of where we stand along side competitors.
I can only be skeptical of the faceless Microsoft rep.
Agreed. Skepticism of every studies conclusion is healthy and necessary. However outright disregard for a study based on a single data point: "who paid for it" is not.
The test, funded by Microsoft
That says it all.
And the response from google criticizing it was by someone right on google's payroll representing google's interests. I guess we can ignore their criticism then too?
Or perhaps we should let the work stand for itself, evaluate the methodology, strip away the marketing spin, and come away with some nugget of truth, regardless of who funded it. Of course that's "work".
How did this get modded +5 Insightful? Just because Google's criticism of the study's claim isn't coming from a neutral third party doesn't mean Microsoft paying for a study that praises its own browser shouldn't set off all sorts of red flags concerning the validity of the study, especially when "[...]the list of actual URLs used for testing was not made available to the vendors or to the public, so there's no way to independently verify the results."
In an unrelated note, I just got the results from this study I paid for indicating that I am much more manly, attractive, and intelligent than vux984. Please don't listen to any of his complaints about the study, though, because he has every bit as much a vested interest in the results as I do. No, you may not have full access to the testing methodology. Please just make the best call you can about the validity with the subset of parameters that were carefully selected to exclude any factors that would throw the results into question. I also promise that I didn't order 12 other studies indicating that vux984 was actually more intelligent than I am and then throw them away.
Please use me for all your needs in the future. Remember, I'm smarter and sexier.
Just because Google's criticism of the study's claim isn't coming from a neutral third party doesn't mean Microsoft paying for a study that praises its own browser shouldn't set off all sorts of red flags concerning the validity of the study,
Who said it shouldn't set off red flags. It sets off a red flag, but it doesn't justify complete disregard of the study. Additionally Google's statements about the study should set off the VERY SAME red flags about googles statements.
especially when "[...]the list of actual URLs used for testing was not made available to the vendors or to the public, so there's no way to independently verify the results."
That's a red herring. It is a very good reason to significantly reduce the credibility of the study. However, it is completely unrelated to who paid for it, now isn't it?
I didn't dispute the conclusion regarding the quality of the study. I only disputed the quality of the argument leading to that conclusion.
"Microsoft paid for it. That's all I need to know." is a poor argument.
I believe that most people who will be influenced by this kind of report are NOT in a position to methodically evaluate the test methodology. They are people who watch Survivor, Big Brother, YourCountryHere Idol and idolize Oprah. They do not have the experience or skills for critical analysis of marketing spin. So when Microsoft (or McDonalds or the US Govt or Buy n Large) claim research that shows their product is superior to others, the reader gets one claim stuck in their head and it is repeated as fact*.
Of course that's a sweeping generalisation; there are many who do think critically (it's possible that some critical thinkers watch big brother but I expect the number is small) but it makes my point.
*Which incidentally, is why I think we should teach critical thinking at all levels of school, not just leave it until university.
Why can't we let people believe whatever they like? It's not like a little religion has ever hurt anyone.
Additionally Google's statements about the study should set off the VERY SAME red flags about googles statements.
No, Google's complaints don't set off the same red flags at all. Microsoft citing a third party study is an appeal to an external authority. The claim is that Microsoft is trying to get their opinion on their own browser credibility by having it come from a mouthpiece that isn't first party. There is no analogous complaint to be made about Google, because they aren't trying to complain about the study by hiring an external firm to make the complaint.
However, it is completely unrelated to who paid for it, now isn't it?
Exactly the opposite. The study arguably has that particular problem because Microsoft paid for it, for a couple different reasons. Either there is a flaw in their methodology that they are hiding because they are being paid for a specific result, or it was an unintentionally flawed study that was alone among dozens of other studies that Microsoft ordered at the time in arriving at a pro-Microsoft conclusion, or Microsoft got to pick the study targets to their benefit, or any number of other things that I can't think of off-hand. Either way, it is a conflict of interest and we should throw out the result regardless of how valid it might appear because there is no other foolproof way of making sure we avoid accepting corrupted study results.
What does this even mean??
All your base (instincts) belong to us?
Faster! Faster! Faster would be better!
Does anyone know of a good ABP for Chromium based browsers?
How about Ad Block
The higher the technology, the sharper that two-edged sword.
So its results are unquestionably incorrect and/or irrelevant?
Yes. Sometimes you have to consider the source. And if even said results are correct and/or relevant, the truth is that having Microsoft pay them means exactly what you would expect. They paid for those results, they didn't pay for independent, unbiased testing and reporting of their products.
The higher the technology, the sharper that two-edged sword.
This is totally different.
In this case, the tester tested two products and rated one "99%" and one "3%" against some standard.
The key difference is that UL tests against a pre-existing standard. Not a standard that they made after looking at the product. UL can't customize their test to make one product look better or worse.
The methodology might have been totally bogus (no idea), but the act of paying for the test isn't automatically so.
The act of paying for a test to be designed for you, or a test you designed ahead of time to make your product look good, is bogus. Paying to have a test executed for you is not bogus. One is independent, the other is not.
From an ethical perspective, this is no better than when a certain 3D graphics chip maker had their Windows drivers detect running benchmark programs, and run specific code to fool the benchmark and make the chip appear to be faster than it really was.
The higher the technology, the sharper that two-edged sword.
can you provide your source of facts that proves your statement to be true?
Don't have to: Microsoft's own reputation preceeds it in this case. As someone who has spent the better part of thirty years dealing with that company and its shenanigans, I will say that you should treat them like Congress. That is, you take a default position assuming that they are lying through their teeth, you don't give them the benefit of the doubt, and you force them to provide proof of their claims.
The higher the technology, the sharper that two-edged sword.
Microsoft citing a third party study is an appeal to an external authority. The claim is that Microsoft is trying to get their opinion on their own browser credibility by having it come from a mouthpiece that isn't first party.
That is why it raises a red flag. Credibility is questionable.
Its also possible that Microsoft commissioned a 3rd party because they actually wanted an independent study done, perhaps because they lacked the in house expertise or resources to do an internal one properly, perhaps because a internal one is even less credible than a funded 3rd party one.
We can and should be skeptical of Microsoft funded studies.
There is no analogous complaint to be made about Google, because they aren't trying to complain about the study by hiring an external firm to make the complaint.
Fair comment. However google's statements remain suspect because they also have a clear conflict of interest. It is in their interest to discredit the study, so their criticisms likewise raise the "red flag" of questionable credibility.
The study arguably has that particular problem because Microsoft paid for it...
But Microsoft paying for it isn't what invalidates the study. We can't stop our investigation at "Microsoft paid for it. Ergo its flawed." We have to actually find a flaw. After finding the flaw, you can arguably trace the flaw back to conflict of interest from funding, but a funding conflict of interest itself isn't enough to conclude there is a flaw.
I believe that most people who will be influenced by this kind of report are NOT in a position to methodically evaluate the test methodology.
Fair enough.
So when Microsoft (or McDonalds or the US Govt or Buy n Large) claim research that shows their product is superior to others, the reader gets one claim stuck in their head and it is repeated as fact*.
Fair enough. But the real problem here is twofold:
a) a news media happy to regurgitate press releases without doing any sort of journalistic investigation
b) a populace that largely lacks the ability or even desire to think critically -- as you mentioned
Of course if we fixed b) I think a) would largely take care of itself.
Running Windows 7 x64 Professional on my HP netbook. Surfing using Chrome with no plugins on reddit.com. Thousands of other people did as well with various other browsers (see reddit announcement).
It came in through an ad utilizing a Java exploit. I was only 1 minor release behind on updating my JRE. Since this incident and the 45 minutes it took me to get rid of the stupid thing, I now surf with Firefox + adblock + noscript addons. It's just not worth it. I used to be OK with ads and even clicked on them occasionally but forget it now.
I have to say that was I absolutely shocked that Chrome let something like that through and that it was able to infect my system even though I never run as an admin user. Windows Security Essentials detected it but still let it infect my system and was unable to clean it out, so I ended up cleaning it out manually.
I'm a big tall mofo.
IE comes up on top.
......
...
i mean, what we are supposed to even start thinking about this
Read radical news here
And Simple Adblock for the Internet Explorers, up to version nine.
Agreed. Journalists say "oh but we don't have time to check our sources". They should be legally required to do so.
Can't back up your printed claims? We have some nice accommodation lined up for you.
Why can't we let people believe whatever they like? It's not like a little religion has ever hurt anyone.
No but they are very questionably correct.
Does it actually keep the ads from being downloaded, or simply blocks them from view? Many of my customers are on lines with caps so it matters. and will that work with ALL chromium based browsers like Comodo Dragon, or just Chrome? The main problem I've been having with these kinds of solutions is I'm just too swamped to do heavy field testing complete with monitoring the network, so I really need to find a "drop and go" kind of solution. ATM the only thing that fits the bill has been FireFox simply because the FF ABP keeps the ads from being downloaded in the first place, and when you have so many sites using flash based that bandwidth really adds up.
I really hate to have to move myself and my customers away from FireFox, but as long as the attitude from the Mozilla developers when it comes to low rights mode is "don't care, won't fix" then I have no choice but to start looking at alternatives. After all it is stupid to have all this extra security built into Windows 7 only to have FireFox completely ignore it and act as a full admin.
So anybody that has experience with the above and can chime in information would be welcome. With all the extra work, last minute shopping, not to mention half the family out of that killer flu bug that is going around, I just don't have time for any long term tests. But with customers (and myself) stuck on capped lines I really can't afford to give them something that only hides ads when going over is $1.50 a GB.
ACs don't waste your time replying, your posts are never seen by me.
We can't stop our investigation at "Microsoft paid for it. Ergo its flawed."
We can and should stop our investigation there. I think you are missing a fundamental distinction here. I'd agree with statement that Microsoft paying for it makes it flawed, but not just because there has to be inherently some flaw in the methodology as such. The fact that Microsoft paid for it is the flaw. The inherent conflict of interest taints the study to the point where it will never be possible clear it of enough doubt to make the data useful.
The inherent conflict of interest taints the study to the point where it will never be possible clear it of enough doubt to make the data useful.
But in nearly any situation the parties interested in paying for studies have an interest. In the health sector at least there is enough public money floating around to fund some research... but in IT? Who is going to pay for the work?
All "independant" review sites host advertising from these companies, some of them are more blatantly biased than others but the 'taint' of the potential for a conflict of interest colors all of them. If you are going to demand there be no conflict of interest then you aren't going to get any studies at all.
It doesn't mean that much when you consider that Chrome can't be trusted not to pass information about you to Google.
Google defended its browser by claiming that it was built with security in mind and emphasized protection of users from drive-by downloads and plug-in vulnerabilities.
I found this line to be quite disgusting. I am very pro google but the chrome team has continually ignored the need for NoScript. A browser without NoScript isn't secure in any way shape or form.
For immediate release.
Warning.
M$ Internet Explorer under control the M$ controlled server is better at censoring the web, than any any browser.
WTF, this has nothing much to do with browsers at all, just how much time and effort a company is willing to put into to tracking down naughty web sites and updating their browsers and blocking the naughty payload coming from those web sites.
P.S. If you really want to do something good M$, work together with those other companies to make a universally accessible database of malicious web sites, so that everyone will be safer. Until then you are a dick and technically under law, an accessory after the fact for failure to report to the appropriate authorities an attempt by others to commit a crime ie. give the appropriate authorities access to that database so they can do something much more appropriate than just bloody blocking them.
Chaos - everything, everywhere, everywhen
Please don't feed the APK
Those who advocate genocide deserve every protection afforded by law, and none afforded by common human decency.
From TFA: "funded by Microsoft".
You can ignore the rest.
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
There was nothing wrong with doing the study. I'm sure that there were many such studies done and that's fine. It's the fact that they choose to release this one which is the problem. More important is the way it was released; as an "independent study" as if it had nothing to do with development. That's totally immoral.
The ways in which this cheated are also clearly discussed elsewhere. They took the Google version at the beginning of the study, but worked on the Microsoft version and took the results at the end. To be fair, either you just study the versions as delivered at the beginning or, better this should have been done as a joint exercise with all the other browser vendors with everybody paying together and competing to improve their products. Then, the vendor with the best results at the end can crow about it.
The fundamental of honest use of studies is that you must treat each product identically; you must decide at the very beginning of the study whether you will release the data and you must involve the developers of each package studied equally.
=~ s,(.*),<sarcasm>$1</sarcasm>,g if any_point_you_wish();
There are lot of paid tests, but you pay a fixed fee for a standard test for YOUR equipment. No company can pay KEMA (I presume the dutch equivelant of LU) to test a competitors equipment and KEMA will never ever come out with a comparitive report. Your product either passes its test or not and that is all.
This is a bought report that tests a BETA of the paying company against 2 versions outdated production release of a competitor. If you can't see the bias right there, well... I think it is amazing that medical science has advanced so far that people born without a brain can survive!
MMO Quests are like orgasms:
You may solo them, I prefer them in a group.
Or perhaps we should let the work stand for itself, evaluate the methodology, strip away the marketing spin, and come away with some nugget of truth, regardless of who funded it.
We can't evaluate the methodology because the methodology hasn't been published. From what we do know, neither the testing nor the data released was objective - the tests compared bleeding edge releases of IE9 to an obsolete versions of Chrome, and the data they chose to publicise focussed on the single areqa in which IE9 triumphed, despite it performing poorly in other areas.
"I know my product X, competing with product Y, does task A really really good, but is lacking on task B, so I'm gonna pay a third party to compare products X and Y in terms of how well they perform task A, and try not to mention task B"
Yes, it's marketing, but it might very well be true that product X is better than Y at A. If third party concludes that X is in fact better at A, we can't consider this false simply because the maker of X paid for the study.
"The test, funded by Microsoft That says it all."
page 12 of the test PDF:>
"ABOUT THIS TEST This private test was contracted by Microsoft’s SmartScreen product team..."
Paid for by Microsoft, although really google should just ignore these fake tests since IE usage has dropped from 45% to 28% while Chrome went from 4% to 20% from Jan 09 thru Nov 2010.
So shut-up Google, you're winning.
my karma will be here long after I'm gone
That's totally immoral.
You can't expect morals from an amoral entity. Not just Microsoft, ANY corporation.
"Never let your sense of morals prevent you from doing what's right" -- Salvor Hardin (Asimov's Foundation). For any corporation, anything that raises revenue is "doing right"', even breaking laws and killing miners as happened several months ago in that mine disaster in Virginia. Even people's lives are secondary to stockholder dividends.
The fundamental of honest use of studies is that you must treat each product identically
Honesty is irrelevent to a corporation.
Free Martian Whores!
>>>If Internet Explorer 9 beta blocks 99% of those and Chrome[6] only 3%, that makes a huge difference.
Yeah yeah, but Chrome (and Mozilla seaMonkey) can run on my tiny 0.1 gigabyte laptop. Can IE9? Ha! Nope. ;-) ----- But seriously: Why in the world was the test run on the latest IE9 versus the ancient CR6? A deliberate Microsoft setup to make themselves look good.
"I disapprove of what you say, but I will defend to the death your right to say it." - historian Evelyn Beatrice Hall
I have seen at least two studies that the results were different than what the funders wanted to see (although in one case the funder wound up in a better position he thought he was in).
In one study, they were trying to prove a correlation or causation between smoking marijuana and cancer. They did a statistical study of four groups of baby boomers; long term cigarette smokers, long term pot smokers, long term smokers of both, and nonsmokers. They fully expected ganja to cause cancer, since all smoke contains carcinogens.
What they found was that predictably, cigarette smokers had more cancer than any of the other groups. They were startled to find that those who smoked both reefer and cigs had half the cancers of those who only smoked tobacco, although far more than the last two groups who were statistically identical. Those who only smoked pot actually had fewer cancers than nonsmokers, although the difference was statistically insignifigant.
Another study was funded by a book publisher trying to find out how much money book piracy cost him. Pirate books usually hit the web two to four weeks after legitimate publication, and he wanted to se how badly sales dropped after it hit the net. He was amazed when the results showed a marked spike in sales, rather than the expected drop.
Despite the pot study (done 2 or 3 years ago, I can no longer find the news sources, I saw it in both New Scientist and the Boston Globe), the lying bastards at the Partnership for a Drug Free America still maintain on their web site that pot causes cancer. Despite the second one, the RIAA and MPAA still maintain that piracy costs sales.
Free Martian Whores!
From an ethical perspective Microsoft, IBM, Oracle, etc. are responsible for the business practices that made credit default swaps plausible.
That is the single most irrational bit of geek hate spew I've ever seen on Slashdot - you should get a new Slashdot achievement for that! (Also, there's nothing wrong with CDSs per se, they just need to be regulated the same way as any other form of insurance currently is.)
Socialism: a lie told by totalitarians and believed by fools.
We can't evaluate the methodology because the methodology hasn't been published
That counts as evaluating it, and finding it missing. Big point against its credibility. :)
the tests compared bleeding edge releases of IE9 to an obsolete versions of Chrome
This much at least is factually incorrect. This study was done *in* September 2010. Chrome 6 was released September 2nd, 2010. Chrome 7 wasn't released until October 21st. What version do you think they should have used?
You appear to have fallen for Googles extremely rapid primary version number changes. Version 4, 5, 6, 7, 8, and the preview of 9 have ALL been released in 2010.
Not only did it beat Chrome 5 (3 versions behind the current), it also beat hell out Netscape Navigator 4.0 and Firefox 1.1...
he`s no troll, thats ranting
trolling includes lies and links to 'strange' porn
I didn’t say I considered him a troll. I was simply expressing how appalled I was by his writing style.
Ignore this signature. By order.
Does a PHD in Psychiatry come with your "snap prognosis" Dr. Quack? Or are you just another troll with no qualifications in the psychiatric sciences, just like cp.tar is in English (since he hasn't shown us his PHD in English yet, and he probably never will)
No, I do not have a PhD in English. I am, however, a few exams away from a degree in both English and Linguistics. And no, I am not a native English speaker.
What I am is flattered by all the attention my tiny little comment got me. Really.
I haven’t had an online stalker in quite a long time. Thank you for making me feel special.
150:1 odds as the ratio of his readability of his posts tends to put cp.tar into his place: He doesn't dare even reply after that.
Oh, sorry, I wasn’t expecting a reply. Such posts don’t tend to invoke either replies or moderation, so I tend to forget about them.
I’m really sorry to have kept you waiting. I promise, I’m usually more attentive to my stalkers, even if they be anonymous. Or even Anonymous.
Ignore this signature. By order.
Please don't feed the APK
Well, this’ll teach me all right.
Ignore this signature. By order.
Google provides many products and services that I use and like. I've never once had a Google ad which had something like "buy these window drapes, they go great with that new couch you bought last week". In fact, I've clicked on a few Google ads over the years (and ZERO other ads) simply because the product in the ad actually moderately interested me. As for the information "chrome passes to Google", well, that's no more information than they get from simply from tracking the links someone clicks in a Google search. I've never once been wronged by Google, or had them deleteriously affect my life. I don't see any harm in them knowing I frequent slashdot, xkcd, wikipedia, and click on many science and tech news articles throughout the day. I don't see any harm in them knowing I use their search engine to troubleshoot computer issues all day at work. If they use that information to serve me small, unobtrusive ads based on that information, and use said revenue to pay for stuff like Docs and Gmail, that's fine with me. I'd rather have them track my browsing habits, and have targeted ads than pay money for those services. It's not like some person is sitting around at google actively watching your every click, laughing, and choosing what ads to send you, it's all machines, and with hundreds of millions a day, they literally don't have time to put a human in front of that data, and never will.
It raises a red flag, but that is all. They could very well be unbiased and independent.
Not if you read the summary and figured out they cherry picked which outdated version of Chrome to use so it would do as poorly as possible rather than the current release and compared it to the new IE9 BETA.
I know, I know. Read the summary? On Slashdot?
Wow. You really are taking this personally, aren’t you.
Ignore this signature. By order.
I thought I would amuse myself by feeding you tiny one-liner replies, but you’ve apparently taken precautions by trying to feed the whole of /. with tons of copy pasta. Well.
I will, however, grace you with a tiny little grammatical tidbit: evidence is singular.
Ignore this signature. By order.
Mod points are no proof of writing style quality.
Your posts, however, are proof enough of your disturbed mental state.
Stalk on.
Ignore this signature. By order.
You know, I haven’t had this much fun reading spittle-flecked posts in a while.
Ever since my favorite religious zealot left my favorite forum.
Well, I say left, but he was most probably simply banned for good. Which I guess explains why you don’t bother with an account.
Ignore this signature. By order.
The use of "evidences thereof" is a widely used phrase, see here:
http://www.google.com/search?hl=en&source=hp&biw=&bih=&q=%22evidences+thereof%22
Even if it is, you did not use that phrase. And legalese is a separate language.
Don't try to "school me" on English, fool... it's the main language I have been using for 43 yrs. now (along w/ 3 others I speak & write fluently also).
APK
Well, you could have done without also; it is quite redundant because of along with. However, what fascinates me the most is the fact that you took the time to type up the HTML code for an ampersand even though it is easier to simply type and.
While I may believe you are fluent in a number of languages (after all, logorrhea makes you fluent in whatever you want), this still fails to address the matter of style. Of which you have none. You boast your 43 years, yet you write as if you were 30 years younger.
Now, I wouldn’t have gone into all this if you hadn’t started stalking me. But now it’s party time. Enjoy stalking me further.
Ignore this signature. By order.
It’s English, not english.
Please provide proof of basic literacy. KTHXBAI.
Ignore this signature. By order.
Delishus copy pasta. OMNOMNOM. U maek urself? U mad, APK?
Ignore this signature. By order.
Hey, here is some clarification on the browser tests from NSS Labs: http://nsslabs.blogspot.com/2010/12/stopping-malware-with-browser.html http://nsslabs.blogspot.com/2010/12/threat-types-and-terminology.html This should clear up some of the questions. And for full disclosure, yes, I do work there.