Firefox Most Vulnerable Browser, Safari Close

An anonymous reader writes "Cenzic released its report revealing the most prominent types of Web application vulnerabilities for the first half of 2009. The report identified over 3,100 total vulnerabilities, which is a 10 percent increase in Web application vulnerabilities compared to the second half of 2008. Among Web browsers, Mozilla Firefox had the largest percentage of Web vulnerabilities, followed by Apple Safari, whose browser showed a vast increase in exploits, due to vulnerabilities reported in the Safari iPhone browser." It seems a bit surprising to me that this study shows that only 15% of vulnerabilities are in IE.

Comments

I wonder

[somersault]

How many of these vulnerabilities were due to Firefox itself, and how many due to plugins?

Re:I wonder

[Shatrat]

Haven't RTFA yet but I bet they are using patch notes as their source of vulnerabilities.
If that's the case then obviously well-documented and frequently-patched browsers will be over-represented.

Re:I wonder

[dkleinsc]

So in other words, this isn't a count of how many vulnerabilities there are, it's a count of how many vulnerabilities are found and fixed.

Something tells me their methodology is a bit flawed. Of course, that's by design, given Cenzic's financial ties to Microsoft.

Re:I wonder

[PNutts]

I haven't read your post yet but you're wrong.

Re:I wonder

[rudy_wayne]

I get your point, but in the end, what is the difference? Many people are die hard users of the plugins (I use firefox and I'll never understand the hype) that they insist they could never go without them, and in many cases it's the primary force in their decision to use firefox.

You're confusing plugins with extensions.

Re:I wonder

[MozeeToby]

Even if their information is accurate, which I don't see how it could possibly be, it is meaningless. Number of flaws is a horrible way to measure system security since it doesn't take into account severity, ease of attack, unreported flaws, or un-acknowledged flaws. When you get down to it, there really isn't any good way to measure security, but I would bet hours spent in code reviews would correlate much better than number of reported flaws.

Re:I wonder

[Teflonatron]

I didn't see anything in the actual report that explained how their results were arrived at. For that reason alone, this report is worthless. It's just a marketing document for use in selling their own security products.

However, it did make reference to the numbers being representative of "reported vulnerabilities", which we all know is going to make Firefox look worse that IE. This is verified by realizing Opera (also closed source) scored less than IE.

Re:I wonder

[Sandbags]

Worse, patch SEVERITY was not accounted for in these results, nor was the fact that many patches were for unexploited vulnerabilitys, and others were to close ITW threats...

FF and Safari rank bad in this article, but when looking at the raw data, patch severity, and explited patch footprint, IE is the worst, even though not patched very often.

I'd also note that a single patch may include fixes for numerous bugs, and this is additionally not covered in the scope of this article. A single patch in IE recently fixed more than 10 vulnerabilties...

Re:I wonder

[Shatrat]

lol, touche.
Still, do you really have to read it?
It seems like one of these bootlicking/astro-turfing 'studies' from some consulting agency or 'solution' vendor comes along about every 6 months in the Slashdot headlines.
Upon reading TFA, this one seems no more credible than any other.

Re:I wonder

[ircmaxell]

What about IE vulnerabilities that are inherent from its close tie to the OS? I'll bet that they didn't count vulnerabilities like today's http://tech.slashdot.org/story/09/11/11/0053244/Microsoft-Plugs-Drive-By-and-14-Other-Holes since it wasn't a flaw in IE itself. It was just attackable through IE....

Re:I wonder

[calidoscope]

The Register's article on the Cenzic report also speculated the the report was based on published vulnerabilities. They made some rude noises about Cenzic's focus on the number of the vulnerabilities as opposed to the severity of vulnerabilities.

Re:I wonder

[Actually, I do RTFA]

So in other words, this isn't a count of how many vulnerabilities there are, it's a count of how many vulnerabilities are found and fixed. Something tells me their methodology is a bit flawed. Of course, that's by design, given Cenzic's financial ties to Microsoft.

Actually, in other words, the GP was making shit up. But since it conformed to your worldview, you agreed with it and based an entire post on it even though he said he didn't RTFA. Somehow it then got modded to +5.

In reality, the vulnerabilities were culled from a variety of 1st and 3rd party sources.

Re:I wonder

[natehoy]

Have read the article, and the attached PDF, and they only state the conclusions. No mention is made of how they counted vulnerabilities, only that Firefox had 44% of them, and that they represented "Web Vulnerabilities by Major Type". Adding to the confusion was that they also talked about applications and servers and alternated back and forth between the three with little warning.

Also interesting was that "ActiveX" was listed as a technology separate from Web Browsers, the one time it was mentioned. In other words, their vulnerability percentage, which is already vague, may not include ActiveX vulnerabilities within IE. Or they may. All we know is that they claim IE has 15%.

Nowhere is there mention of what constitutes a reportable vulnerability, what versions of each browser were counted, how they were classified or even what the classifications were, what sorts of reports were included by browser (did plugins or addons get included in Firefox? ActiveX for IE? For multiplatform browsers like Opera, Firefox, and Safari, were vulnerabilities mitigated by only being exploitable on some platforms and not others, or reported multiple times - once for each vulnerable platform?)

The PDF was severely [citation needed], but remarkably honest in that it expressed surprise that Firefox was the most vulnerable web browser when compared IE, Safari, and Opera, and comprised almost half the identified vulnerabilities among the four browsers.

If this is like most reports of the same type, they are using vendor-reported bugs. Firefox would, by definition, have the largest bug list by any stretch in such a report. They are the only web browser development team that allows (and encourages) access to the same bug-tracking database that their developers use. Safari, IE, and Opera only report vulnerabilities when (a) they have been fixed, or (b) when so many reports have come out that they finally have to 'fess up.

Re:I wonder

[Galestar]

The PDF in the article is mostly marketing, and does not do much in the way of explaining how they arrived at those numbers other than; "Cenzic analyzed all reported vulnerability information from sources including NIST, MITRE, SANS, US-CERT, OSVDB as well as other third party databases for Web application security issues reported during the first half of 2009." We can therefore conclude that those numbers are based upon reported vulnerabilities, regardless of whether or not they were fixed. From my experience Firefox has a good habit of quickly patching security vulnerabilities. For example, there is the SSL spoof vulnerability discovered late July that Firefox fixed in 5 days and IE/Safari/Chrome still haven't fixed in over 3 months AFAIK) So there is nothing to indicate that Firefox is necessarily a less secure browser.

Re:I wonder

[Nikker]

I actually RTFA and the vulnerabilities it accounts for are

• SQL Injection 25%
• XSS 17%
• Web Server 2%
• Buffer Errors 12%
• Web Browser 8%
• Authentication / Authorization 14%

Plus a few under 10%. The funny thing is that the article seems to blame the browser for SQL Injection, Web Server, Information Leak / Disclosure? WTF?

Information Leaks could be the result of any attack, SQL Injection has nothing at all to do with any browser and "Web Server"? There is no real information other than a nice shaded 3D pie chart so what this guy is trying to prove is beyond me. It also includes Path Traversal which is server side as well, code injection well injection into what? The browser, the server ... what?

Popular vendors including Sun, IBM, and Apache continue to be among the top 10 most vulnerable Web applications named.

Even if some agrees that these companies are actual web applications and not software companies, you would have to agree that there really are only about 10 commonly used web servers in total so Sun, IBM and Apache will be on this list regardless of the exploit.

Looking at the real report all of the exploits blamed on the browsers are based on SQL Injections and propagating malicious code from the originator of the web site so how could one browser handle this more effectively then another? This doesn't really make a lot of sense so anyone gifted with more ability then myself please reply below.

Re:I wonder

[Bloody Peasant]

So in other words, this isn't a count of how many vulnerabilities there are, it's a count of how many vulnerabilities are found and fixed.

I read the report. It is a marketing document, with one person (Mandeep Khera, Chief Marketing Officer) identified in it as both project lead and executive editor.

Also, despite the fact that the report itself downplays browser vulnerabilities (8% vs. 90% web apps, 2% web servers), they still put in a single token page which just seems out of place. Nowhere does it say what their methodology is for determining what comprises a "vulnerability". Another poster already pointed out the google search results on the CERT site (~367 for IE, ~61 for Firefox; that's over 6 times more vulnerability reports on the CERT site for IE versus those for Firefox; oops, was I shouting?).

I suspect the authors' methodology is simply to count something like the number of patches. Given Microsoft's monthly bundling of their security patches, and the Mozilla Firefox project's immediate release of more frequent version updates in response to vulnerability reports and discoveries, such methodology leads to a systematic undersampling of those for IE. A better approach would be to count verified CVE candidates.

Pure speculation: were they paid by anyone to put that browser breakdown in (it really doesn't seem to belong in my opinion), or was it ignorance or lack of thinking? Without an honest clarification from the company we'll probably never know.

Re:I wonder

[http]

Pardon my ignorance, but how exactly is Cenzic tied financially to Microsoft again? Google's got nothing (and bing has less).

Re:I wonder

[w0mprat]

I don't even bother posting I use a form.

-- Slashdot posting form --
...
[ ] RTFA
[ ] In soviet russia ____ YOU!
[ ] Obligatory XKCD
[ ] _____ you insensitive clod.
[ ] Get off my lawn
[x] I don't even bother posting I use a form.
...

Re:I wonder

[WinterSolstice]

I was going to point this out as well - there was nothing really backing up the browser diagram at all. They didn't even really go into how they determined these vulnerabilities existed, even though they did go into how web apps break down (reasonably enough).

Just another BS FUD report

Re:I wonder

[commodore64_love]

The thing noticed is that the "most vulnerable" browsers were open-source (Firefox, Safari) and the least vulnerable were closed-source (Explorer, Opera) with a huge gap in between these two types.

Could it be that closed-source aps simply don't publish their vulnerabilities, so that makes them look better?

Re:I wonder

[GumphMaster]

In what way is a Microsoft Certified Partner not financially tied to the maintenance of the Microsoft ecosystem in the face of encroaching offerings, particularly in the browser space?

A more cynical person might assert that a company peddling security assessment tools for web servers would actively promote less secure server systems that kept them in business. Spreading FUD about a browser is only peripheral to that but it does feed the "non-Microsoft is bad" or "open-source is bad" ethic of senior management and bean counters... keeping major systems on Microsoft platforms and Cenzic in business. As I say though, you'd have to cynical ;)

Re:I wonder

[Bigjeff5]

Safari is not open-source, WebKit is. Prove me wrong by finding a copy of Safari 4's source code. Yeah, didn't think so. The vulnerabilities aren't necessarily related to the browser engine (though they certainly can be).

From what I understand the report was based on the number of vulnerabilities patched, not announced. for IE these are released every tuesday of every month, for FireFox I believe they are released whenever they are finished.

Vulnerabilities patched is a decent indicator, because for closed source you would not know about any unpatched vulnerabilities that were discovered internally (and there are a lot) before patching. Any serious vulnerability that MS knows about MUST be patched for IE, for if it is discovered they knew for any extended period about a serious vulnerability and did nothing, they risk losing the confidence of their business partners.

So despite the fact that some people, particularly open-source advocates, don't trust MS to patch vulnerabilities, it is certainly in their best interest to do so. The evidence is the speed and number of vulnerabilities they patch.

I don't think severity would help the metric in favor of Firefox or Safari because serious vulnerabilities get patched as quickly as possible on all sides (except maybe when Safari devs don't consider a severe vulnerability severe, heh), and a large portion of patches that MS releases for IE are less than critical.

With the most recent versions of IE Microsoft has really cleaned up its act in regards to security, and they have the ability to be the best at it if they choose to be.

Patched vulnerabilities may not be the best metric, but I think you'd be hard pressed to find a better one.

Re:I wonder

[eulernet]

If we continue Cenzic's approach, we can prove that IE6 is the most secure browser, since there are no more patches for it.

Re:I wonder

[tbannist]

Most of your analysis just seems completely wrong. Microsoft has left vulnerabilities unpatched for years after they were to disclosed before, I see no reason they wouldn't do it again. In theory their business partners might lose confidence, but let's be frank most of Microsoft's business partners are entirely reliant on Microsoft, it'd takes years for them to make any significant changes. Effectively Microsoft can do whatever it wants, and it has.

Vulnerabilities listed in patch notes are not a good metric for determine which browser is "most vulnerable" because patch notes can be easily gamed by a closed source company. Simply roll up a bunch of nominally related bugs into one patch and suddenly your browser is more secure than the competition. It relies on the all of the groups involved acting in good faith which is naive at best.

Yesterday Microsoft released a patch for IE that prevents a drive-by rooting of your computer on all versions of Windows (Except 7 and 2008 R2) and all versions of IE. Sure. And yet it's somehow supposedly to be more secure than Firefox?

We've heard the same tired refrain from Microsoft sponsored people every time they target a new company. They pay people to make up statistics and lie about the competition. I, for one, am tired of it.

Re:I wonder

[Anonymous Coward]

Well it seems Opera are not too impressed with the report either, despite the fact they come first:
http://my.opera.com/haavard/blog/2009/11/10/cenzic-security

Which is interesting. Not often you see a company criticise a report that shows them in such a good light

Re:I wonder

[IgnoramusMaximus]

With all that said, I can confidently say that I have absolutely no idea how valid this survey is. It seems pretty legitimate to me but I haven't exactly scruitinized it either.

The "study" was conducted by methodology unknown, includes no references to raw data and goes completely against publicly available data (which many posters on this thread provided references to, such as the lists of CERT advisories and the like). This, combined with the fact that the company seems financially motivated to produce pro-Microsoft propaganda, leads sane observers to dismiss the thing out of hand.

So I'm assuming you don't trust surveys sponsored by rivals of MS either, right?

Why, yes! The value of a survey is in its methodology and 3rd-party verifiability, not in who produced the thing. Science and all that, no?

Huh?

[Anonymous Coward]

So just down the page on slashdot, this very day, there are warnings about a "Windows kernel vulnerability" that is exploited through IE. I'll take three cross-site scripting bugs any day over a kernel level compromise, thank you.

I know the world doesn't have a good objective measure of "impact" to assign to these things so that one could assess the total "probable inconvenience" of the presented security vulnerabilities, and that makes unbiased data gathering difficult, but this feels pretty absurd.

Certified

[rwv]

It seems a bit surprising to me that this study shows that only 15% of vulnerabilities are in IE.

There is an explanation for that.

Cenzic Recognized as a Microsoft Certified Partner, Experiences Substantial Momentum in Q2

Re:Certified

[MiniMike]

That makes sense, because if anyone had told me that Firefox had more vulnerabilities than all the other browsers I would think that they were certifiable...

Re:Certified

[cmeans]

And then there's this:

http://www.cenzic.com/pr_20061011/

Re:Certified

[adamchou]

You didn't mention how to become an MCP though. Its not just a matter of filling out a form and sending it to Microsoft. These companies go through a rigorous set of evaluations based specifically around Microsoft products in order to become MCP. So although Microsoft might not control them, their pocket books do and they sure as hell invested a lot of money to become MCP's.

who is cenzic?

[bl8n8r]

Just another consultant hired to slant reality if you ask me.

http://search.cert.org/search?q=advisory+internet+explorer
http://search.cert.org/search?q=advisory+firefox

How the results were compiled

[Anonymous Coward]

According to the report, as best I can determine, this is how they found their results:

"Cenzic analyzed all reported vulnerability information from sources including NIST, MITRE, SANS, US-CERT, OSVDB, as well as other third party databases"

It seems reasonable that any/all open source software would have a higher number of reports in these databases than proprietary software, simply because more people are able to publicly scan and report on vulnerabilities... by definition, open source software conducts it's business in public, while proprietary software does so behind it's private curtain.

"Reported" bugs?

[Bluemumba]

Isn't counting bugs released as part of press releases and change logs kind of like saying "All confirmed criminals are in jail?"

Yet another deliberately lying bullshit story!

[Hurricane78]

Comparing openly known vulnerabilities, and calling it "all in all vulnerability".
As if they wouldn't know perfectly well, that Microsoft sends a cease and desist letter to anyone who is even talking about a vulnerability that is not official to MS.

I guess the old saying is true, that:

If you can't program, you teach.
If you can't teach, you administrate.
If you can't administrate, you report.
If you can't report, you criticize.

Microsoft beats all in security tests!

[David Gerard]

Microsoft is reeling from the vicious and unwarranted slanders of security companies and the US government’s Computer Emergency Response Team that its Internet Explorer web browser has alleged “security holes” or is in any way less than the finest software known to mankind and excellent value for your money. "Cenzic proves it's Firefox! FIREFOX DID IT! Fuckers."

The festering paedophiles of CERT have gone so outrageously far as to make the ludicrous claim that just viewing a malicious webpage in IE could leave your computer open to being hacked and turned into a Russian Mafia spam server. “We don’t know what could have triggered such vindictiveness,” sobbed Microsoft marketing marketer’s marketer Steve Ballmer. “Do they hate free enterprise that much?”

There are things you can do to make your computing experience even more secure. Microsoft’s official suggestion — make sure your anti-virus software is up to date and using an entire CPU doing nothing much, click through five screens to run IE in “protected mode,” click through four screens to set zone security to “high,” click “JUST BLOODY DO IT WILL YOU” when the User Access Control asks if you really want to do this, enable automatic updates with the minor side-effect of installing Microsoft DRM on your system or Windows Genuine Advantage randomly turning your computer into a paperweight, and sacrifice a goat to Microsoft at midnight on a moonless night — is simple and straightforward. “It’s the quality you’re paying for.”

On no account should you consider that there might be other web browsers out there, as researchers have demonstrated that all of them automatically download the cover of Virgin Killer. “I saw a report,” said marketing marketer John Curran of Microsoft Completely Enderlependent Analysts, Inc., “that another browser had more vulnerabilities than ours! People would be very foolish indeed to move from the latest IE to Netscape 4.01.”

“These CERT wankers are Mactards and trolls,” said Guardian marketing marketer Jack Schofield. “They just want to take IE users out, brutally sodomise them, gas them in concentration camps and” [This comment has been removed by a Guardian moderator. Replies may also be deleted.]