Oracle Has More Flaws Than SQL Server

jcatcw writes, "Next Generation Security Software Ltd. of Surrey, England, compared bugs in Oracle and SQL Server that were reported and fixed between December 2000 and November 2006. The tally: Oracle had 233; MS SQL had 59. The products compared were Oracle 8, 9, and 10g; SQL Server 7, 2000 and 2005. From the article: '[The head of the survey said,] "The results show that the reputation that Microsoft SQL Server had back in 2002 for relatively poor security is no longer deserved."' Oracle's response: 'Measuring security is a very complex process, and customers must take a number of factors into consideration — including use-case scenarios, default configurations, as well as vulnerability remediation and disclosure policies and practices.'"

Comments

translation

[User 956]

Oracle's response: 'Measuring security is a very complex process, and customers must take a number of factors into consideration -- including use-case scenarios, default configurations, as well as vulnerability remediation and disclosure policies and practices.'

Oracle's response in english: Clearly you have no idea what you're doing, because your results showed us in a poor light. Perhaps you'd like to try again. We have a bag of money for you.

Re:translation

[HairyCanary]

I tend to agree. But Oracle does have a point. Trying to distill a security argument down to number of bugs is oversimplifying. The severity of the bugs, how easy they are to exploit, etc are all important to consider. Even more important in my opinion is how quick the vendor is at fixing them. If Oracle's average time to fix was 24 hours compared to six months for Microsoft, the 4:1 bug ratio is not such a big deal.

Re:translation

[SatanicPuppy]

It's typical MS fud. They LOVE to harp on how many bugs their competition has, but there is a hell of a lot more to it than quantity. Slammer anyone?

Oracle is a huge robust database with lots of extremely security conscious clients. A high number of reported bugs and fixes shows that they're executing due diligence, and working to keep their system as secure as possible. MSSQL's low number of bugs suggests that Microsoft isn't digging hard into their code, but only waiting for big public flaws.

They used the same argument in claiming that IE was less buggy than Firefox (see this crappy article) and it's just as untrue in this case.

Re:translation

[Anonymous Coward]

I'm not an oracle person, but from my understanding oracle allows you to have finer grained security on data, stored procedures and so on than sql server. Perhaps the complexity of oracle compared to sql server is part of the reason there are more bugs.

Lets face it, a bug report can be anything from a misspelled error message to a gaping sa/root/admin (whatever oracle calls it) compromise.

Severity is important. For instance, most popular linux distros (minus gentoo) have quite a few security holes do to third party package inclusion. Often the holes are not severe, but they do make linux look artificially insecure compared to some other operating systems. If redhat pushed 90 updates a month at you and Microsoft only 35... well who looks less secure? How many were feature enhancements? How many did each vendor NOT include a fix for?

Disclaimer: My above reference to linux distros only includes bloated packages like redhat, suse, etc. Most people using these distros tend to do a "full install". I'm a mysql or sql server user whenever possible.

Often one could argue that smaller companies get less attention so a large number of vulnerabilities would indicate a very insecure product. Oracle is obviously smaller than microsoft as a whole. In this case, oracle gets a lot of attention as its used for large scale deployments as well as their *lovely* business practices.

Re:translation

[ZachPruckowski]

You're right. This survey is pretty messed up. I mean, we're comparing *bugs fixed*. Not bugs still open, or any measure of severity, or what got exploited, or any measure of turn-around time.

This is like saying that Fire Department A put out less fires than Fire Department B. That's nice, but what I really want to know is how long it took for the trucks to arrive, the size of the fires, and also if there are any houses that burned down before the Fire Department got there.

Re:translation

[arivanov]

Oracle is also the database with the longest time to fix security bugs. I will simply quote the message from BUGTRAQ which is most relevant to this thread. It about says it all:
Thor (Hammer of God) wrote:
David Litchfield is one of the most predominant security researchers in the field, particularly in the area of database security. He and NGS have discovered more combined security vulnerabilities in leading DBMS products than anyone else in the world.
Given this fact, I think that not only is it appropriate for David to give whatever opinions he chooses in his research, but that it is his opinions that actually give the research real, tangible, applicable value. With his indisputable status as an authority on database security and his unwavering integrity, I have no problem whatsoever in considering Dave's opinions to be "fact."
Actually the whole discussion on BUGTRAQ is definitely worth reading. By the way the vulnerability behind Slammer was discovered by guess who - David Litchfield.

Re:translation

[A_Non_Moose]

Lets face it, a bug report can be anything from a misspelled error message to a gaping sa/root/admin (whatever oracle calls it) compromise.

If that is the case, oracle's mgmt tools heavy reliance not only on java, but *specific* version of java
w/o updates I'm aware of, would explain a lot.

off the top of my head:
Input fields that don't register the first key press, menu item that don't redraw for some reason, refreshes and connection errors that require exit/relaunch.

Other frustrations like that, that aren't oracle's "fault" per se, but don't help the spec/check sheet for bugs.

Didn't RTFA (yet), but are those counted as bugs? I'd like to know.

Summary title is vague

[ArcherB]

MSSQL is a SQL Server. MySQL is a SQL Server. Oracle is a SQL Server. Please be more specific and explain which SQL Server you are talking about.

Granted, the summary does explain that the article does indeed refer to MSSQL Server, but please stop calling it just SQL Server. MSSQL Server != SQL Server

(OK, I feel better. What is the moderation for RANT?)

Re:Summary title is vague

[hobo sapiens]

Microsoft just so happens to be so uncreative that they gave their DB server application a name that is merely a description. Calling it SQL Server is appropriate, since that is, after all, what it calls itself and as far as I know, is the de facto name for the software. Yes, it's a bit like calling a Web Browser WebBrowser. Blame MS for picking a nondescript name.

Re:Summary title is vague

[drinkypoo]

Actually, the name of the product is "Microsoft SQL Server". Still a stupid name but it's not just "SQL Server". Lazy techies are responsible for not using the full name, not that I blame them. What I want to know is how Microsoft managed to convince a court that the name of another product of theirs was actually "Windows" and not "Microsoft Windows" (look at the box sometime!) which forced all those other people to change their product names.

Oracle is more complex

[sitturat]

Anyone that has tried to read (or even tried to lift up) one of the oracle manuals knows that this is seriously feature-rich and complicated stuff. It would be more interesting to see how many bugs per line of code the two contenders have.

Oracle is right

[Josh Lindenmuth]

While the # of vulnerabilities is unacceptable, Oracle is right ... just comparing the # of bugs is not really valid. Now if Oracle has had more Severe security violations that Microsoft, it would be a different (and far more interesting) story. Oracle is still a more robust database, so one would expect there to be more bugs than another app with fewer modules and lines of code.

Stop counting flaws!

[91degrees]

The number of flaws doesn't matter. a slice of cheese has one flaw as a database. It isn't a database. This doesn't make it a better product.

Check the data and the criteria before deciding

[Graabein]

and customers must take a number of factors into consideration

Not least the criteria for selecting and enumerating flaws, and any differences between those criteria for the two products. Not saying that there is a problem, just that any prospective customer needs to take this into consideration and check his facts.

This whole study reminds me of a couple of years ago, when someone decided to make a comparative list of security flaws between Windows and Linux. For the former, they only included official Microsoft security fixes. For the latter, they included just about every bug in every open source project known to man. Big surprise, Windows was found to have less flaws.

When it comes to security, trust no one. Especially not research firms, security "specialists" and people mouthing off about security on Slashdot.

Hey, waitaminute....

Reported AND fixed

[nels_tomlinson]

From the summary: ... compared bugs in Oracle and SQL Server that were reported and fixed between December 2000 and November 2006.

Reported and fixed means that the company which doesn't fix bugs looks more secure. Not that I'm implying that MS is worse than Oracle on this, mind you. I just wanted to point out that this metric has loads of potential flaws.

What, specifically, are those "bugs"?

[khasim]

Between December 2000 and November 2006, external researchers discovered 233 vulnerabilities in Oracle's products compared with 59 in Microsoft's SQL Server technology, according to NGSS. The study looked at vulnerabilities that were reported and fixed in SQL Server 7, 2000 and 2005 and Oracle's database Versions 8, 9 and 10g.

Let's see that again.

The study looked at vulnerabilities that were reported and fixed...

So, if it wasn't fixed, was it counted?

The results show that Microsoft's software development life-cycle processes appear to be working, he said.

Huh? Security is not about "software development life-cycle".

That's why you have almost daily updates of anti-virus software for Microsoft products.

In an e-mailed comment, an Oracle spokeswoman said the number of reported vulnerabilities in a product alone is not a measure of the overall security of that software.

Big time. One remote root vulnerability is worth 10,000 local app crash vulnerabilities.

"Measuring security is a very complex process, and customers must take a number of factors into consideration -- including use-case scenarios, default configurations as well as vulnerability remediation and disclosure policies and practices."

Yep. Because Ubuntu has, by default, no open ports. So it is, by default, 100% resistant to worms.

Remember, you can never count on a user applying a patch. Your system has to be as secure as possible in the default, unpatched, configuration.

Basing a product's security just on the number of vulnerabilities discovered and fixed may not be the best approach, said Pete Lindstrom, an analyst at Midvale, Utah-based Burton Group.

Not only is it not "the best approach", it is a fucking idiotic approach only used by morons who have no understanding of what "security" is.

It's not the number of bugs. It's what access can be gained by that bug and how easily it is to invoke that bug in the various "standard" configurations.

Re:What, specifically, are those "bugs"?

[Rich0]

While I agree with 95% of what you said, I'd take issue with this:

Ubuntu has, by default, no open ports. So it is, by default, 100% resistant to worms.

Not all worms require open ports to spread - a worm might target a low-level kernel flaw in the network stack (remember the ping-of-death?).

David Litchfied

[Cally]

It should be pointed out that this is not just A.N. Random UK Software Co trying to flog product. This is David Litchfield, one of that small number of security researchers whose names and work any self-respecting infosec analyst should be familar. He's done a lot of really superb security work, including trashing several versions of SQL Server; so he knows whereof he speaks.

NGS have of course done work on SQL Server for Microsoft; I refer you to the brief and rather one-sided flamewar on Bugtraq/FD that erupted when this was pointed out... actually see for yourself... (and here's the Bugtraq thread). I predict this will deal with 75% of the "but this is nonsense, because..." posts ;)

He's got a lot of credibility. This is the point I'm trying to make :)

59 bugs reported and fixed...

[Ant P.]

x bugs reported and ignored, y bugs not reported at all and not fixed.

My experience

[truthsearch]

I worked extensively with Oracle and SQL Server for 10 years at 2 companies. I ran into bugs with both systems. There was a vast difference between how each company responded to our bug reports.

We never contacted Microsoft with anything but the most severe bugs, and only those not documented on their web site. Even having the highest contract possible with Microsoft, they charged us for each phone call. Never once did the first 3 people we talked to have a clue. After going through 3 or 4 people we got to speak to a developer. For every bug except one, we were told to wait for the next official patch or Service Pack to fix our issue. One time we were fortunate enough to have a DLL updated by a developer and sent to us directly. Response by developers was very quick, but the other staff responded slow.

At the same time, Oracle was paying out $10,000 for each bug found. I thought I found the golden ticket. Turns out someone else had reported this extremely obscure bug I found earlier, but it wasn't yet published online anywhere. Every time we contacted Oracle we got to speak to a developer very quickly. On at least one occassion they sent a developer to our office to help investigate a bug. Every bug we reported got a patch very quickly.

The support from Oracle was far far superior to Microsoft. The bugs I ran into with Oracle were also far more obscure than those I found in Microsoft's SQL Server. I couldn't believe some of the things Microsoft left broken for months. Even if Oracle has a larger number of reported bugs I'd pick them over Microsoft any day.

Re:My experience

[anto]

Have you tried to call MS & log a 'support' call - more than once we have had to hand over the credit card no before the call will be forwarded on. Of course with the promise that if there was an issue they wouldn't charge it.

Oracle on the other hand request your support contract no (which they will actually look up for you) once you get past that really minor issue you never hear anything about money again. If you are unlucky enough to have a real bug that gets escalated you have the fun experience of hearing from someone from oracle every few hours - the calls seem to come from all over the world (based on accents etc)

More than once I have had a custom patch created for what to oracle must have seemed like a really minor bug.

In Oracle's (Pseudo) Defence...

[Randolpho]

... they are rather quick to quash and fix a discovered security bug. Yes, there's a reason why I used both words. Check out the aftermath of this example at The Daily WTF.

Re:If you offer a ton of additional features...

[RevMike]

If you offer a ton of additional features...then it stands to reason that you will have a ton of additional bugs.

This argument in no way excuses Oracle for their timely patch cycle (or lack thereof), but may explain the higher number of patches.

It is also important that Oracle supports virtually any server platform in current use, while SQL Server only supports a small number of similar platforms. Back in 2001 I was still getting support for Oracle 7.0 on VAX/VMS! One get Oracle on Linux, AIX, Solaris, HP-UX, zOS, OS400, Windows, a variety of Alpha platforms, Itanium platforms, etc. And this isn't shallow level support. Oracle can utilize their own file systems, so they are going at the bare hardware on all these systems. Care to guess what that does to the QA cycle?

Oracle is the shiznit when it comes to high performance general database work. It will scale far beyond almost everything else, with DB2 a close #2. Niche players like TeraData have their place too, but only Oracle can scale across the entire enterprise.

This just in

[mattwarden]

My left arm has more dead skin cells than my right index finger.