Slashdot Mirror
Ask the Author of the Latest MS-Funded Windows vs. Linux Study
Last week on Slashdot you saw a (Microsoft-funded) research
study on Windows vs. (Novell) Linux reliability by Dr.Herbert
Thompson. Novell disagreed
with the study's conclusions. So did most Slashdot readers.
Thompson's work been mentioned on Slashdot before, especially his
famous five-line
script that could change electronic voting machine results
and his novel, The
Mezonic Agenda: Hacking the Presidency. He's a real,
genuine-article computer security expert (and regular Slashdot reader)
who is happy to put on his flame-resistant
suit and discuss his Microsoft vs. Linux study with you. So
ask whatever you like, one question per post. We'll send him 10 of the highest-moderated questions and publish his
answers next Monday. He'll jump into the discussion then, which ought
to make it rather lively.
Dr. Thompson:
Admittedly, I don't know who you are and I haven't read any of your books. Worse, I didn't read your study itself, only its conclusions as reported second-hand by the press. However my lack of knowledge of your backgound is probably consistant with most Slashdot readers and the IT industry as a whole. I have to give you the benefit of the doubt and assume that you are a capable, respected researcher elsewise MS wouldn't have approached you in the first place.
Could you please explain why you decided to risk drawing your objectivity into question by undertaking this project? Your findings may be 100% valid. And MS may very well have straight-up told you: "Please print whatever you find, even if it casts Windows in a bad light." However, who's going to believe it, even if it were true? If I were in your shoes, I'd be affraid that making a deal like this would ruin my career. If I don't tell MS what they want to hear, word would get out that I don't play ball. If I do report what's in the sponsor's best interest, a lot of people start accusing me of being a shill. Seems like a lose-lose proposition.
Entrepreneur : (noun), French for "unemployed"
How can you stay neutral when one side is funding your research?
The study seemed to only compare comercial applications on the various platforms and not the alternatives. Its very common that comercial apps on Linux have poor support on Linux while the free alternatives blows most out of the water on Windows too. Its not especially hard to select a couple of apps with stellar support on Windows and SAP like support on Linux and blame Linux when the problem really lies in the lack of vendor support. Some vendors even support just one specific linux version without! any patches applied.
What care was taken in selecting applications with similar support offerings to not bias the study heavily to Microsofts advantage?
HTTP/1.1 400
Microsoft and Linux distros have had a policy for some time of including more and more functionality in the base operating system, the latest example is the inclusion of "Local Workflow" in Windows Vista.
As a security expert do you think that bundling more and more increases or decreases the risks, and should both Windows and Linux distros be doing more to create reduced platforms that just act as good operating systems.
An Eye for an Eye will make the whole world blind - Gandhi
How many Microsoft-funded studies have been buried because the conclusion was "incorrect"?
How would Dr. Thompson ever know that? Has he been in charge for a lot of MS-funded studies lately?
It seems that your study attempted to simulate the growth of an internet startup firm on Windows or Linux. One thing I did not see in the study was a good description of assumptions you made. What assumptions were made in both the design of the requirements and the analysis of the data? What limitations can we place on the conclusions as a result of these assumptions?
LedgerSMB: Open source Accounting/ERP
How many NDAs did you have to sign before starting the study? Did anyone pull you asside to "set the record streight" before the study began? How were you first asked about doing this study? Was it something like "hey, we need a study to boost our TCO stats, here's some cash..." or was it more altruistic like "hey, we need to see how we stack up agaist the competition .. heres some cash, and dont hold any punches!"
-GenTimJS
Everyone on /. likes to complain about microsoft security, and microsoft PR people like to point out their improvements. Here's a chance to give ammunition to both sides. What do you think are the three biggest security improvements microsoft has made in the past two years, and what are the three biggest security-related issues that still remain?
"Weapons should be hardy rather than decorative" - Miyamoto Musashi
I think that goes for OS's too
You tested six people on two different systems; how is that supposed to yield any substantial insight into the underlying OSes themselves?
[At best, your study seems to show that the GNU/Linux distribution you selected was not particularly good at this task. But why does that show that the ``monolithic" style of Windows is better per se than the ``modular" style of GNU/Linux distributions?]
"Every decent man is ashamed of the government he lives under." - H.L. Mencken
The Linux administrators faced some out of the ordinary challenges, not faced by most Linux admins, while the Windows admins faced none.
For example, most of the time difference between Windows and Linux was spent upgrading gLibC, something that you're really not supposed to do. It's comparable to trying to manually upgrade parts of a Windows 98 system to run a program that required XP, rather than actually upgrading to XP.
Then, you had the Linux admins getting updates from 4 different sources, rather than just from SuSE's repositories, which is also out of the ordinary, while the Windows admins only visited Windows Update, which only supplies patches to the base operating system, when in reality they'll have to get updates from many other sources if they wanted to keep their apps up to date.
Do you think this was a fair study?
Looking at your research report's appendices, it seems that the requirements for Windows Administrators were somewhat different than the Linux Administrators. For instance, you ask for 4-5 years sys admin experience minimum for Windows, whereas it's 3-4 years sys admin experience minimum for Linux.
Why wasn't it equal for both? And doesn't this sort of slight Windows favoring undermine your credibility?
How is it that Diebold can make ATM machines that will account for every last penny in a banking system, but they can't make secure electronic voting machines?
Also, does the flame-resistant suit come with its own matching tinfoil hat? (don't answer that one)
He who knows best knows how little he knows. - Thomas Jefferson
How do you sleep at night?
On top of a pile of money, surrounded by many beautiful ladies.
Kudos to you for braving the inevitable flames to answer people's questions here on Slashdot.
Read the EFF's Fair Use FAQ
From the study:
What I find lacking is the business case for upgrading the OS. And why on earth would any enterprise with even the tiniest amount of foresight and planning deploy Windows 2000/SuSE 8 knowing they will upgrade to the next gen just one year later? (Not that there aren't plenty of enterprises who fit your model, not to mention IT workers seeking to "power level" their skills...)
Now, certainly there is value in trouble-free installs. But can you say with confidence a better upgrade experience is really a fair test of value? Especially when the entire install/patch/upgrade philosophy between Windows and Linux is so disparate?
In other words: It's no surprise that Windows will perform better on the treadmill, constantly upgrading is at the very core of Microsoft's profitability.
--
If I understand the study correctly, the windows side had to do nothing but set up a server to do a few different tasks over time and run windows update. The linux side had to have have multiple incompatible versions of their database server running simultaneously on a single system and had to run unsupported versions of software to do it.
Why wasn't the windows side required to run multiple versions of IIS or SQL server simultaneously? In real life if you need to run multiple database versions you use virtualization or multiple systems, especially if one requires untested software. You don't run some hokie unstable branch on the same system as everything else. Why was a linux solution picked that required this level of work? My other related question is, did any of the unix administrators question why there were being asked to do such a thing? For example, did they come back and say they need a license for vmware? If they did not they do not seem like very competent administrators in my opinion.
Windows administrators are forced to wait until Windows releases a patch for known vulnerabilities to upgrade their systems. Why, then, were the Linux administrators told to attempt to upgrade their systems before Novell had released newly packaged versions of MySQL? The entire point of a package management system is that administrators rely on companies like Novell to correct dependencies prior to deployment. Since Windows administrators have the same constraint (i.e., waiting for security updates to be released), it is an unfair and arbitrary difference that caused a lot of troubles.
Why did you compare the number of patches required to apply between the systems? This is not a measure of security. Windows patches are bundled and affect many parts of the operating system while Linux patches affect individual components. The overtone in your paper implied that fewer windows patches was in some way easier or more secure; what justification do you have for this assertation?
What is the rationale behind this? Were the Linux administrators required to restart at this point? This is an incredibly contrived situation; one can simply stop and re-start the process in question after the upgrade has completed.
Furthermore, the upgrade methodology questionable. Real companies use development and production servers and don't upgrade the production server until a reproduceable upgrade trajectory has been tested on the development server. The actions of these administrators imply that they had no such access, and that there was no possibility for backtracking or restarting after a failed step. Normally, one would expect the ability to nuke the development server and start over, rather than following a bad plan to worse conclusions.
A quick read of the report shows that the real losers here seem to be the Administrators. Some of the Linux admins "could not meet business requirements", and some were judged as failures by not using vendor-supplied solutions.
Isn't one of the points of running Linux servers the freedom to use solutions NOT supplied by the vendor? Is it even possible for the Microsoft admins to make changes that aren't fed from the vendor?
When the only tool you have is the "Upgrade" button, and the button doesn't work, what then? The advantage of Linux in administration is the flexibility to Make It Happen, even if the vendor sends you something broken.
I know good admins on Microsoft, and good ones on UNIX. They seem to Make It Happen no matter what, because that is their job. Making It Happen sometimes include custom fixes, that are documented, so you can undo them when the vendor comes through (hopefully) later.
So the Final Question is, why was it bad for the Linux admins to stray from vendor-supplied fixes, and why is the lack of flexibility on the Microsoft side a "win"?