Slashdot Mirror
Study Finds Windows More Secure Than Linux
cfelde writes "A Windows Web server is more secure than a similarly set-up Linux server, according to a study presented yesterday by two Florida researchers." In addition to the Seattle Times article, there is also coverage on VNUnet. From the article: "The researchers, appearing at the RSA Conference of computer-security professionals, discussed the findings in an event, 'Security Showdown: Windows vs. Linux.' One of them, a Linux fan, runs an open-source server at home; the other is a Microsoft enthusiast. They wanted to cut through the near-religious arguments about which system is better from a security standpoint."
... another pissing match.
Before any liberals are tempted to mod up one of my comments, a word of warning: I'm actually making fun of you.
Study finds Slashdot as repetitive as Philip Glass
Well, apparently this is the second time Microsoft has come out on top of a research project by Mr. Richard Ford.
http://www.virusbtn.com/magazine/articles/letters/ 2004/01_01.xml
Apparently there was some question to the validity of an earlier project because it was sponsored by Microsoft.
However, I would like to note that both researchers seem very well educated, especially in computer security. And, additionally, they both note that a lot more could be done to lock down the Linux server.
I don't get it. I guess I need to read the article.
A webserver needs port 80 and maybe 443 open. Any webserver can be secured.
Where's the news?
The man who trades freedom for security does not deserve nor will he ever receive either. - Benjamin Franklin
Interesting. Some relevant snippets:
.
A Linux enthusiast at the RSA Conference in San Francisco has reluctantly concluded that Microsoft produces more secure code than its open source rivals.
In an academic study due to be released next month Dr Richard Ford, from the Florida Institute of Technology, and Dr Herbert Thompson, from application security firm Security Innovation, analysed vulnerabilities and patching and were forced to conclude that Windows Server 2003 is more secure than Red Hat Linux.
Now, I'll concede that Dr. Ford and Dr. Thompson do sound reputable, but one is an admitted Windows enthusiast and while the other one is a Linux fan who changed his minds, this hardly sounds like a study
It's an interesting question, and I'm sure there is no clear cut answer, but a more systematic study (with more parties, rather than just two scientists) is going to be needed to answer this sort of question before the 'results' are trumpetted. I'm sure Microsoft will pick this one up and run with it, however.. more of those annoying ads that seem peppered throughout Slashdot.
"There's no success like failure, and failure's no success at all."
- Bob Dylan
"Their criteria included the number of reported vulnerabilities and their severity, as well as the number of patches issued and days of risk -- the period from when a vulnerability is first reported to when a patch is issued."
So Windows is more secure than Red Hat because Microsoft chooses to report less vulnerabilities and release less patches? Hmmm...
(Move along, nothing new to see here.)
Until the report is released this is a non-story, just fuel for the FUD machine. Unfortunately we will have to wait for a month to actually discuss what this means so I don't even no why I am bothering to post to this!
Never underestimate the dark side of the Source
Now let the flaming begin, so you can all argue about the number of patches/updates required for each system, how long it takes for Linux/Windows to respond to problems, and all that good stuff. We all know that's the only reason this kind of story shows up on Slashdot is to start a good flame/troll war!
"...Ford said the idea was to represent what an average system administrator may do, as opposed to a "wizard" who could take extra steps to provide plenty of security on a Linux setup, for instance." By his own admission the Linux administrator is a "Wizard" compared to the average MS Systems Admin. Well, that just about says it all doesn't it?
"Sheep just follow the easiest path and run from scary noises and intimidating creatures." - Me
No matter how fast a patch is issued, you still have to install it for it to work.
Doesn't Microsoft encourage delaying announcing vulnerabilities until a patch is available?
How the hell can anyone claim to be a "Microsoft enthusiast"?! It's hardly a hobby.
Smokey, this is not 'Nam, this is bowling. There are rules.
This was a hardly a study. I don't see any data presented here, and certainly no methodology used to gather the data. Sorry, but the scientific method always wins.
Sorry, but this "study" is not a study.
Why was this even posted?
Read it for yourself. It reads:
"Believe it or not, a Windows Web server is more secure than a [i]similarly set-up[/i] Linux server, according to a study presented yesterday by two Florida researchers."
So when you load a linux server with software that has known security holes....they are both equally as secure.
It's not groundbreaking news.
OpenBSD runs chroot() Apache. Does IIS have similar capability?
The chroot() patch was never taken up, but it would probably not be that difficult to install on Linux.
I would be disinclined to run any other way at this point.
A Linux enthusiast at the RSA Conference in San Francisco has reluctantly concluded that Microsoft produces more secure code than its open source rivals.
Umm, so MS showed him their source code? I find that a little hard to believe.
If he can't see the source, how can he make any determination at all?
And, to the grandparent -- if you read your own link, the previous study was not sponsored by Microsoft.
What I'm listening to now on Pandora...
Neither article defined "days of risk" to my satisfaction. Is it "days since the vulnerability was published" or "days since the vendor was informed of the vulnerability"? I suspect that Microsoft is more likely to hear things privately early. ASN.1 library anyone? It was discovered in July 2003, and announced and patched in February 2004. Was that six months of risk or one day?
Secondly, there's no discussion of how the criticality of a vulnerability was weighed. If every "day of risk" for Windows was "critical," and every "day of risk" for RedHat was "moderate," then I'd differ with their conclusions. Further, there was no mention of whether they considered actual exploits in the wild.
What I say does not represent the views of my employers, my friends, my cats, or myself.
cfelde writes "Satanism is less evil than a christianity, according to a study presented yesterday by two Florida researchers." In addition to the Seattle Times article, there is also coverage on VNUnet. From the article: "The researchers, appearing at the RSA Conference of philosophers, discussed the findings in an event, 'Religion Showdown: Good vs. Evil.' One of them, a satanist, performs perverse human sacrifice rituals; the other volunteers at the local homeless shelter. They wanted to cut through the near-political arguments about which religion is less evil from a morality standpoint."
Knock Knock.
Who's there?
Knock Knock.
Who's there?
Knock Knock.
Who's there?
Knock Knock.
Who's there?
Knock Knock.
Who's there?
Knock Knock.
Who's there?
Knock Knock.
Who's there?
Phillip Glass
My 8 year old daughter, a great afficionado of knock knock jokes, didn't appreciate it.
"As God is my witness, I thought turkeys could fly." A. Carlson
"There are some people who are sceptical [of the results]," said Dr Thompson. "We would encourage them to replicate this type of study. If you see flaws please tell us."
Are they joking? Their metric (reported vulnerabilities) is absurd for a number of reasons.
1) Microsoft reports only a fraction of its vulnerabilities. Remember when Win2000 had over 65000 known (to Microsoft) flaws? No more than a handful were ever reported. Microsoft reports flaws only after bearing enormous public humiliation. Of course Microsoft's flaw count is going to be low. Microsoft hides them all until forced to disclose.
2) Linux vendors report every hair out of place. It doesn't matter if the flaw causes a D to look like an O on the third day of the Summer Solstice, but only if that day matches the 4th digit of PI, and only if the computer has calculated the cure for cancer at exactly 15 milliseconds after the user's orgasm.
3) Seriousness of vulnerabilities. Due to the nature of full disclosure under Linux, it will -always- have higher reported flaw counts than Windows. The vast majority of reported Linux flaws, however, are relatively benign, while the vast majority of reported Windows flaws hand over complete control of your computer to some third party.
4) Widespread Propagation. Windows, by its intended design, makes propagating exploits to these vulnerabilities trivially easy (automatic, actually), while this has yet to be accomplished on Linux (and likely won't be).
Sorry, but this "study" is complete nonsense.
One datapoint makes a terrible graph.
... and squint your eyes, you'll see the 'clear' results.
The researchers used reported vulnerabilites as their guideline, and 'days of risk;' quote: "the period from when a vulnerability is first reported to when a patch is issued."
Windows Server 2003 had 30 days of risk, Linux (Red Hat Enterprise Server 3) 71 days.
But which reports of vulns are they considering? Microsoft often provides their own reports, which are released WITH THE PATCH. I wouldn't give those reports the same weight, since the vuln could have been there (and unofficially known) for MONTHS.
I fully expect Linux to have MORE vulns in any case, since Linux ultimately is a collection of separate programs working together, each of which has their own potential insecurities. But, a vuln in sendmail is NOT going to affect my webserver, because I'm going to turn that OFF (if I'm a smart admin).
In fact, the researchers only used a "hypothetical" system to show "what an average system administrator may do." I'm sorry, but if an admin is using anything like a default setup he is BELOW average.
In conclusion, this really sounds like a comparison of how vulnerable the respective systems with a 'default' install. Wake me up when they go head-to-head with OpenBSD.
P.S. Hey researchers- RED HAT IS NOT LINUX.
I wish I could mod you up, bonch. I've experiened the head-in-the-sand Linux mentality too, and it is scary. It misses the whole point of linux.
Linux is awesome, this study doesn't change that but we always need to work to make it better and easier to secure. Critics of Linux are our best friends, because they do the work of finding out where we need to improve for free.
The best thing about linux is that when people have a legitimate complaint, it's well within our power to fix it! If Linux is temporarily less secure, so what? After reading this, everyone will adapt their linux distros to render the complaints moot.
This is part of why we love open source, right?
Slashdot. It's Not For Common Sense
Whew. For a second there, I was worried I'd have to get out the mat.
...is too hard to handle for most:
An OS is only as secure as it's admin is competent. This will NEVER change no matter what platform you are dealing with.
If you give some RedHat CDs to a complete goof off and have them install it on a system that is going to be directly exposed to the internet, that box is going to get rooted eventually. It might take longer to get rooted than a Windows box, but it will be cracked.
If you give Windows 2003 Server to a knowledgable admin, he will secure the box and make certain that the likelihood of it getting cracked is fairly low. He will know not to put the box on the internet until he's applied all SPs and critical updates. He will know to use an internal SUS or WUS to make sure that the box is updated without exposure to the internet.
If you give a complete moron who *thinks* he knows all about [insert platform] any installation media, you're going to have an insecure box.
It's been my experience that the best people to set up an internet exposed box using any OS are people who are most familiar with all OSes and have a good understanding of how to secure each one. It's not that hard to hit the main security points and still keep on top of all OSes. However, since egos aer so intrinsically tied to how secure a box is, people point the finger at the OS distributor. Sure, they are to blame in many cases, but the implementor is usually far more guilty of being lax. That's the hard truth and it cannot be refuted.
-"...bad old ideas look confusingly fresh when they are packaged as technology" - Jaron Lanier (Digital Maoism on Edge.o
I'm going to dump my Fedora Installation(TCO $0.00)
.....
and run to the store and buy me Server 2003(TCO $599-$3522 + Licencing).
Definitly not going for RHEL(TCO $349-$2499 + Licencing) because no matter how hard I try, I could never get as secure with up2date, SELinux, Pax and Firefox as I could be with Windows Update, Third party antivirus, Windows Firewall and Security Center. NEVER!
And I shouldn't even be comparing Fedora to Server 2003 because Fedora could never be used as a server of any kind. Neither could Slackware(TCO $0.00), Suse(TCO ~$100.00), Mandrake(TCO ~$100.00), Debian(TC0 $0.00) or any other of those insecure Linux distros! They're not SOLD as servers so they absolutely cannot be compared to server 2003. No way, never, uh-uh.
Wow! This study has really opened my eyes to the lie. Why did I abandon my XP installation(TCO $200.00) after only a few dozen major worm outbreaks? I could have done anything on XP that I can do in Linux. It would only have cost be a few thousand dollars, but I could have!
These researchers have really opened my eyes to the lies. I believe everything they say, even without the data to prove it they.....
Ok here my sarcasm must crack under the sheer enormity of the following statement.
The pair said that they lacked the funding to test other operating systems, such as the Apple OSX kernel(TCO $100.00), although they thought it was "amazingly" stable.
WTF!? Are these guys for real? Is this study just a troll? I mean... WTF!!?
I will however take a wild guess that their next server security study will have OpenBSD mysteriously absent.
May the Maths Be with you!
Funny thing that seems to be missing in the discussion so far: I don't see anyone pointing out that this is a "sample of one" study. So any generalization at all about which system (or admin ;-) is more secure is laughable at best.
It is useful as an anecdotal example. Especially in the area of security, where real security tends to mean knowing a lot of very specific examples of how things can go wrong. Documenting how these guys could have inadvertently left holes open would be useful. Then we need several hundred more such paired tests, with a more extensive report listing all the ways that admins of both systems can get it wrong.
But concluding that, because two guys didn't get it right in a single test, therefore one of the systems is more or less secure than the other, shows little other than a total lack of understanding what security is all about.
That, or intentional FUD on the part of either or both.
I'd go with the lack of understanding. People are really good at generalizing from a single case with no statistical significance.
Those who do study history are doomed to stand helplessly by while everyone else repeats it.
Hmm:
:(
Who's there?
Knock Knock.
Who's there?
nock Knock.K
Who's there?
ock Knock.Kn
Who's there?
ck Knock.Kno
Who's there?
kKnock. Knoc
Who's there?
Knock. Knock
Who's there?
Steve Reich
She probably wouldn't like that one any better.
It's unfortunate RedHat has acquired Windows' weak security posture in it's effort to attract Windows server market share. I've personally had to administer 3 compromised Redhat boxes, and this after converting that client over from Windows due to a compromise.
But, RH isn't Linux. Linux is many distributions, some good, some not so good, but if you take the pool of Linux administrators against the pool of Windows administrators, you'll find Linux administrators are more knowledgeable about their systems and do smarter things in securing them. This isn't as true as it was a few years ago before the reluctant Windows administrative masses took refuge in RedHat, but you won't see _any_, not even one Linux defector to Windows. Perhaps BSD, but definitely _not_ Windows!
I've never seen one of my Slackware servers (running sendmail, _even_ and FrontPage extensions with PHP on the Apache server) compromised. It's never happened in the 10 years I've been using them.
I've been wasting a lot of time lately poring through logs for a new project and it's ludicrous how much additional coding I've had to put into my Perl scripts to make allowances for compromised Windows boxes that have inundated my web server with traffic during their Code Red and Slammer compromises, not to mention all the other little oddities Windows clients do when downloading mp3s from the server, such as client caching and sending 32k+ search strings in the URL. It creates work to have these obnoxiously configured client machines on the Internet.
I'm not going to complain too loudly since without all these Windows users on the Internet surfing my site, there wouldn't be much of interest to process in these logs, but to assert Windows as more secure than Linux?! Really....
Could someone please post the name of which Micro$oft C?O's budget backed this study, so we can move on to a more interesting and valid discussion?
www.dedserius.com
VB != VisualBasic
Bruce Schneier
Posted on January 06, 2005 at 01:45 PM
------------
Different methodology, different results. My money's on Schneier.
"There are already a million monkeys on a million typewriters, and Usenet is NOTHING like Shakespeare." - Blair Houghton
I'm gonna give it a try and quote here what I read in the VNUnet article (which is the most informative one IMO since it contains a few details, in contrast to the other one) and try to express some reasoning. Until the real analysis is out we cannot be sure about anything though.
Classic strategy: minimize your enemy by defining it tightly as a dogma, then attack that dogma. I've seen this from Sun Microsystems as well. Basically, they ignore e.g. Novell. At least Novell is also a big player in terms of market share.
That said I remain interested in learning why they chose to compare to Red Hat and Red Hat alone.
Definition of 'vulnerability counts' and which vulnerabilities are counted. For example, lets say Red Hat has a patch for OpenLDAP while i run LAMP or LAPP then who cares about the fact that there's an OpenLDAP patch? Not me.
71 days is long! How they got to these numbers is also very interesting. For example, does this include e.g. the Mozilla bug which was alleged to be known (but not fixed) in 2001? It reminds me about MSIE for which vulnerabilities took long as well and remember 1 patch != 1 vulnerability either.
Statements like these may just as well be from astroturfers. Its also a classic strategy: basically, you play as if you're convinced by the study you conducted yourself while you expected a different result. In all honesty, why would you believe the judgement about the conclusion ("FUD!") from someone who hasn't read the study over the one from the person who's got convinced by his own study? This is why there's not much we can currently do except arguing over the existing details! This is why we need to stress about where the missing details are. This is why we cannot judge yet.
One last note:
With that last statement he Dr Ford basically says to take this study with a grain of salt because thats precisely what he hasn't researched!
WE DON'T NEED NO BLOG CONTROL.
Their contention was that for lower skill admins, Windows was more secure. Now, assuming the research was done correctly and the data does indeed support the conclusion, it's a good thing to know. That's something ot try and improve in Linux, espically since less competent admins are the real problem.
It's not all that useful to research how tight a competent admin can lock down a box because the answer for almost any OS is "very well". You get a good admin that knows their OS and is on top of things, they can keep anything secure, even Windows. So it's not of much use to say a compentent Linux admin can make a secure system, we already knew that.
It is useful, however, to know that a less competent admin will have trouble. More useful would be to know what specificly need to be done to fix it, but just knowing that it's a problem is a start. If Linux continues to gain in popularity, more people that are not as competent will be running it. While you can never truly protect someone from themselves, there are things you can do to make things more secure for those that don't know what they are doing, and that's a good thing for Linux developers to be looking in to.
Having read TFA, the "study" consisted of counting security flaws for RH and Windows, and comparing how long it took to issue patches -- from the date of the vulnerability being announced. This is really shallow; we've seen lots of such studies and laughed at them. I note the spin put on this is "One of them, a Linux fan, runs an open-source server at home..." which makes it look like a Linux zealot has been hacked in his own home, while the happy Windows guy is unscathed. In fact, it was all hypothetical, there were no trials of real servers (none mentioned anyway), just "potential" vulnerabilities in default setups.
As someone said, "extraordinary claims demand extraordinary evidence". In a lot of peoples' opinion, the claim that Windows is more secure than Linux is just that, an extraordinary claim.
How would the authors of their study reconcile it with something like this one, which showed that a default installation of Windows got infected with a virus within 20 minutes?
From TFA:That sounds good. A real comparision of real services running on real servers.
But wait!They aren't real setups.
And it gets worse.Hmmmm, I wonder if they included the info from www.eeye.com http://www.eeye.com/html/research/advisories/AD20
So, a "study" that doesn't test any real world criteria is somehow valid?
Oh, it's not that the study is not valid, it's that pointing out the flaws in the study shows the groupthink on
And pointing out that perceived groupthink gets you mod'ed up as "insightful".
That's a critical difference. So many people pour over the Apache source code that most vulnerabilities are discovered prior to when they actually become "in the wild" exploits. The same cannot be said about MS IIS. Worse, the odds are very good that many the IIS exploits were in the wild prior to when they were first publicly reported, while most of the Apache exploits were, in all likelihood, patched prior to the first exploit.
When viewed from that perspective, the Windows/IIS server was likely vulnerable to exploit for many, many more weeks than the Linux/Apache server. And that assumes that half the vulnerabilities are ever even reported. With a closed source product, there could be tons of security holes being subtly exploited by clever crackers every day and there would be no way to find out about it.
No, this article is pure and unadulterated FUD.
There are three kinds of lies: lies, damned lies, and statistics.
---Benjamin Disraeli
Check out my sci-fi/humor trilogy at PatriotsBooks.
Their analysis was based on number of patches and time it took to get patched from the time it was publically released. Microsoft stays quiet about most vulnerabilities until a patch is ready and will ship it some time that month, thus the average 30 days. In addition to this, there are still IE holes unpatched from last july. This didn't make the report because its a server. Also, Linux comes with *much* more software by default and much more functionality. They said that these were default setups. That means that if they were using a distro like Red Hat, every single program gets updated as necessary over 2000 programs judging from one of my boxes). Far fewer programs get updated from Windows Update (usually only core programs and utilities... or things that Microsoft deems necessary).
Also, many OSS exploits are theoretical in nature... if a strcpy() passes an unchecked ptr and some coder sees this... whether or not that code could have been exploited... he fixes it and out goes the patch. Its a patch for something that may have never been even able to be taken advantage of. That would never happen in a commercial project. All this study shows is that these researchers define security as the ability to hide security problems as long as possible until a patch is ready and if the patch never gets ready, just never tell anyone about the problem. Following the two above stated rules would easily make any software company "secure" by their standards. As stated previously, their criteria was # of patches and time to release. Time to release is shortened by waiting until the patch is ready (which Microsoft does) and # of patches is shortened by simply not releasing non-major patches and just rolling them out with the next version. The criteria these guys used was meaningless and if anything shows that linux is doing something right if they are updating several times more programs with only twice the delay (which i really doubt is the true delay time). One other thing worth noting, the Ford guy has been paid by Microsoft several times to do studies and release them in favor of MS, I'd hardly call him a true linux fan. Maybe this time they just covered it up better... you wouldn't want to bite the hand that feeds you.
Regards,
Steve
Knock Knock.
Who's there?
Who's there?
Is anyone there?
Who's there?!
- John Cage
Is that too obscure?
It is by the juice of the coffee bean that thoughts acquire speed, the teeth acquire stains. The stains become a warning