Slashdot Mirror
Ask the Author of the Latest MS-Funded Windows vs. Linux Study
Last week on Slashdot you saw a (Microsoft-funded) research
study on Windows vs. (Novell) Linux reliability by Dr.Herbert
Thompson. Novell disagreed
with the study's conclusions. So did most Slashdot readers.
Thompson's work been mentioned on Slashdot before, especially his
famous five-line
script that could change electronic voting machine results
and his novel, The
Mezonic Agenda: Hacking the Presidency. He's a real,
genuine-article computer security expert (and regular Slashdot reader)
who is happy to put on his flame-resistant
suit and discuss his Microsoft vs. Linux study with you. So
ask whatever you like, one question per post. We'll send him 10 of the highest-moderated questions and publish his
answers next Monday. He'll jump into the discussion then, which ought
to make it rather lively.
How can you stay neutral when one side is funding your research?
Good question. I'd be scared to post anything pro-microsoft on here, as a large number of /. users are pro-linux and anti-microsoft. I myself, am not too biased one way or another. I believe at this time that both linux and windows have their places, and aren't in 100% direct competition.
I find that there are too many variables plus unknowns to preemptively measure a TCO before a system has been installed and maintained and migrated to the next system. The maintenance is sometimes addressed, the end of life is rarely if ever addressed.
My personal bias is that Windows systems are good for being domain controllers and file servers for Windows clients, and the UNIX/Linux is better for your typical "headless" dull day to day server stuff like web servers, email, database servers, HPC machines, etc.
So my questions are: Are these studies worth anything more than pseudo-science advertisements, and if so why? And why is the end of life so rarely discussed?
How many Microsoft-funded studies have been buried because the conclusion was "incorrect"?
How would Dr. Thompson ever know that? Has he been in charge for a lot of MS-funded studies lately?
To be sarcastic, I'd ask "who the heck actually takes these studies seriously?", but obviously *somebody* does. Who are these people, and why do these people take these inudstry analyst firms/journals/reports seriously? Are they right or wrong to do so? This isn't an attack (or endorsement :) of your research -- I'm talking about the credibility gap in industry research, and my observation that it's an industry-wide problem.
The meta-credibility question is this: Given the amount of shoddy pay-for-play research out there, does being published in an analyst journal tend to cost (a researcher, his consulting company, his financial backers) more credibility than it can gains him/her/them? If not, why not -- and more importantly, if so, is there any way to reverse the trend?
What many of you miss to realize(Microsoft included), is that there are a large group of current Microsoft only customers that are unhappy with their current offerings. Just because someone is against Microsofts decisions doesnt meen they like Linux. Many just see Linux as a catalysator wich will free the market, push standards and make interopability more common between vendors. Its very rare with 100% Microsofts network still Microsoft refuses to support any standard that would make life for their customers easier. The constant steering towards 100% MS networks is pissing people off.
This really isnt about Linux its about making computers and their software be as standard as the internet.
HTTP/1.1 400
Do you think there is reasonable evidence of vote tampering in the 2004 US Presidential election? Do you think the current batch of Diebold machines in Ohio or other electronic voting machines in use for that election are trustworthy?
How is it that Diebold can make ATM machines that will account for every last penny in a banking system, but they can't make secure electronic voting machines?
Also, does the flame-resistant suit come with its own matching tinfoil hat? (don't answer that one)
He who knows best knows how little he knows. - Thomas Jefferson
The study had admins manually resolving dependency conflicts and borking their systems. I dont think 'right tool for the job' is even on the map if that's where the admins end up.
I mean, by whatever deitys protect sysadmins, _manually_ upgrade _glibc_??? I havent done that since before package systems were invented.
"If the conditions were different"
You mean, if the Windows admins spent most of their time manually copying files in dos shells from floppy disk because they for some inexplicable reason didnt want to use more modern methods for handling such problems?
If the conditions are to benchmark people doing things the wrong way then I rather doubt the value of the conclusions.
Is a Linux study funded by GNU/FSF/OSI/OSDL or whatever any more impartial? No.
I think many here would disagree. Nonprofits are not driven by motives which could be considered the mirrored opposite of commercial corporations. There is not the tremendous pressure to turn a profit (or some analog to monetary gain), and in your examples they're run by mere handfuls of individuals receiving very little compensation with only their reputations to fall back on. They represent what are largely hobbyists, almost to a maddening degree.
OTOH, in Microsoft we have a callow and selfish for-profit entity with a rather abusive track record right up through their financial, er, daliances with SCO.
Need I say more?
Given their dynamics and history, being so dismissive of FOSS organizations as to just say 'well, eveone's biased anyway' really doesn't seem like an acceptable attitude.
"Could you please explain why you decided to risk drawing your objectivity into question with insane paranoiac Slashdot readers . . ."
Corrected. I know it may seem like a troll, but I don't think it is. Something that a lot of the readers of this site don't understand is that not everyone thinks that Linux is the shit to the point of denying all evidence to the contrary. Don't get me wrong, I have one Linux-only computer that I use for work, my other is dual boot, and I like it. I love Linux both for its principles and because it allows me to do things that I can't normally do with Windows, BUT that does not mean that I believe its raw performance to be equal to that of a more heavily funded operating system. And you know what? That's OK. I'd still rather use it.
I think many here would disagree.
Hang on, you're saying you believe that you would trust a FSF or OSDL-funded study to be impartial? You're saying that if the FSF funded a study comparing GNU to Windows, and the study came back saying "Windows saves you money in the long term, and Microsoft's Shared Source is as good as Free Software for 99% of users", that the FSF would then be happy to publish that study?
I don't think so, and I suspect you won't either, if you pause to think about it.
Nonprofits are not driven by motives which could be considered the mirrored opposite of commercial corporations. There is not the tremendous pressure to turn a profit (or some analog to monetary gain), and in your examples they're run by mere handfuls of individuals receiving very little compensation with only their reputations to fall back on.
But that doesn't make them impartial! All it means is that the profit motive is replaced by other motives. And there are plenty.
Think about how much time the major contributors to free software projects put into those projects. Hours, days, months, years of personal time, freely given. Time that could have been spent earning money, or doing charitable work, or even just spending time with their families. Time that was wasted, if it turns out that the software they produced is not actually going to help many people do anything at all.
When you reach middle age, and the end starts to heave into sight on the horizon of your life, you start to get very, very uncomfortable about the idea that you might have devoted your precious time to an unworthy cause.
Being so dismissive of FOSS organizations as to just say 'well, eveone's biased anyway' really doesn't seem like an acceptable attitude.
What's dismissive about that? Microsoft really does think that everyone ought to use Microsoft software, and the FSF really does think that everyone ought to use free software. Everyone is biased. Pretty much everyone does have a pre-existing investment, either of time or money, in one of the options. And human nature does dictate that when you have an investment in something, you are biased towards accepting studies that support it and disregarding studies that don't.
What's wrong with telling the truth?
OK, I've found and read the report now, and this is just bollocks. From the report:
So the test involved installing on SuSE 8 two applications that (effectively) required SuSE 9. Rather than upgrade to SuSE 9, the test mechanism required the operators to hack their systems to make this work. Some of them did this by taking the ill-advised step of compiling their own glibc; doing this broke the vendor supplied version of 'rpm', leaving them unable to undo their changes. Others did it by partially upgrading their system to SuSE 9 by installing SuSE 9 rpms over their SuSE 8 equivalents.
The Windows equivalent test worked fine because the equivalent applications that the Windows operators were required to install were intended for use with the version of Windows they had installed.
Basically, the test wasn't fair. If SuSE-9 dependent applications were to be used, then SuSE 9 should have been used as the basis of the test. If SuSE 8 had to be tested, then equivalent applications that functioned on SuSE 8 should have been found (chances are, slightly older versions of the same 2 apps would have functioned fine).
So, no, glibc wasn't "mucked up because SUSE's YAST was broken". The operators broke YAST by trying to install a glibc upgrade in order to use an application that wasn't compatible with the system they were running. The test was unrealistic; they weren't given the option of upgrading the system properly. They were told, "make this application run on this system." It's not surprising that some of them failed.
This is utter bollocks. See my analysis of the report in this comment.
They broke RPM by hand compiling glibc, not the other way around. It says so quite explicitly. They hand compiled glibc because they were asked to install (without upgrading to SuSE 9) an application that wasn't compatible with the version in SuSE 8.
Dr. Thompson.
You note yourself, in your study that the sample is based upon 6 system administrators/systems. That number is, as you yourself note, too small to be considered definitive. That being the case I would argue that this makes the report viable not as a decisionmaking tool but a marketing tool. Were I a CIO I would feel unwilling to base my conclusions soley on a sample size of 6. What is your opinion on this? Do you expect further, more statistically-significant, work to take place? Or do you feel that this is not a problem?
Your study is interesting, but without knowing the 3rd party tools and applications that were used in the test how can we know the results are valid? Without disclosure the results are irreproducible. My hypothesis is that many of the applications were very poorly supported for linux and well-supported for Windows, but without knowing the applications I can't know if this is true or not.