Slashdot Mirror
Ask the Author of the Latest MS-Funded Windows vs. Linux Study
Last week on Slashdot you saw a (Microsoft-funded) research
study on Windows vs. (Novell) Linux reliability by Dr.Herbert
Thompson. Novell disagreed
with the study's conclusions. So did most Slashdot readers.
Thompson's work been mentioned on Slashdot before, especially his
famous five-line
script that could change electronic voting machine results
and his novel, The
Mezonic Agenda: Hacking the Presidency. He's a real,
genuine-article computer security expert (and regular Slashdot reader)
who is happy to put on his flame-resistant
suit and discuss his Microsoft vs. Linux study with you. So
ask whatever you like, one question per post. We'll send him 10 of the highest-moderated questions and publish his
answers next Monday. He'll jump into the discussion then, which ought
to make it rather lively.
How can you stay neutral when one side is funding your research?
Good question. I'd be scared to post anything pro-microsoft on here, as a large number of /. users are pro-linux and anti-microsoft. I myself, am not too biased one way or another. I believe at this time that both linux and windows have their places, and aren't in 100% direct competition.
I find that there are too many variables plus unknowns to preemptively measure a TCO before a system has been installed and maintained and migrated to the next system. The maintenance is sometimes addressed, the end of life is rarely if ever addressed.
My personal bias is that Windows systems are good for being domain controllers and file servers for Windows clients, and the UNIX/Linux is better for your typical "headless" dull day to day server stuff like web servers, email, database servers, HPC machines, etc.
So my questions are: Are these studies worth anything more than pseudo-science advertisements, and if so why? And why is the end of life so rarely discussed?
It seems to me that the "study" was a simulation or a model. Since such simulations are inherently simplifications of real-world environments, what conclusions should we draw from this? In other words, what are the limitations of your method regarding the conclusions we can draw?
LedgerSMB: Open source Accounting/ERP
How many Microsoft-funded studies have been buried because the conclusion was "incorrect"?
How would Dr. Thompson ever know that? Has he been in charge for a lot of MS-funded studies lately?
Could you please explain why you decided to risk drawing your objectivity into question by undertaking this project? Your findings may be 100% valid. And MS may very well have straight-up told you: "Please print whatever you find, even if it casts Windows in a bad light." However, who's going to believe it, even if it were true? If I were in your shoes, I'd be affraid that making a deal like this would ruin my career. If I don't tell MS what they want to hear, word would get out that I don't play ball. If I do report what's in the sponsor's best interest, a lot of people start accusing me of being a shill. Seems like a lose-lose proposition.
Contrary to popular belief, except in circles like slashdot few people have trouble accepting a report that favors a commercial product from the market leader over a distro based on a free product from a minority player. It is after all the way most reports go (which tend to make little fuzz because preserving status quo isn't very exciting), and there's nothing uncommon about being commercially funded by one side. You make it sound like this is something rare and dangerous. It's common and everyday, even when the reports go against Linux. Is a Linux study funded by GNU/FSF/OSI/OSDL or whatever any more impartial? No. Do you have problems finding people doing it? [sarcasm]What? Wouldn't they be afraid to be considered communist hippies?[/sarcasm]. Most people go out there and try to make a honest living (or at least not further than sensationalizing headlines to draw page hits). Of course there's a small group delivering FUD on demand, but they are far from the majority.
Live today, because you never know what tomorrow brings
I think that a better question leaves the whole Linux vs. Windows argument out of the question:
i.e. - Any idiot knows that a study becomes worthless when independent funding is lost (even if the results are truthful). Exactly how much did Microsoft pay you in order for you to submit to such a fiasco and lose your credibility for life?
To someone like me (who believes that Windows is much cheaper than Linux in many cases), this is something that I'd like answered.
More
To be sarcastic, I'd ask "who the heck actually takes these studies seriously?", but obviously *somebody* does. Who are these people, and why do these people take these inudstry analyst firms/journals/reports seriously? Are they right or wrong to do so? This isn't an attack (or endorsement :) of your research -- I'm talking about the credibility gap in industry research, and my observation that it's an industry-wide problem.
The meta-credibility question is this: Given the amount of shoddy pay-for-play research out there, does being published in an analyst journal tend to cost (a researcher, his consulting company, his financial backers) more credibility than it can gains him/her/them? If not, why not -- and more importantly, if so, is there any way to reverse the trend?
What many of you miss to realize(Microsoft included), is that there are a large group of current Microsoft only customers that are unhappy with their current offerings. Just because someone is against Microsofts decisions doesnt meen they like Linux. Many just see Linux as a catalysator wich will free the market, push standards and make interopability more common between vendors. Its very rare with 100% Microsofts network still Microsoft refuses to support any standard that would make life for their customers easier. The constant steering towards 100% MS networks is pissing people off.
This really isnt about Linux its about making computers and their software be as standard as the internet.
HTTP/1.1 400
Do you think there is reasonable evidence of vote tampering in the 2004 US Presidential election? Do you think the current batch of Diebold machines in Ohio or other electronic voting machines in use for that election are trustworthy?
How is it that Diebold can make ATM machines that will account for every last penny in a banking system, but they can't make secure electronic voting machines?
Also, does the flame-resistant suit come with its own matching tinfoil hat? (don't answer that one)
He who knows best knows how little he knows. - Thomas Jefferson
The study had admins manually resolving dependency conflicts and borking their systems. I dont think 'right tool for the job' is even on the map if that's where the admins end up.
I mean, by whatever deitys protect sysadmins, _manually_ upgrade _glibc_??? I havent done that since before package systems were invented.
"If the conditions were different"
You mean, if the Windows admins spent most of their time manually copying files in dos shells from floppy disk because they for some inexplicable reason didnt want to use more modern methods for handling such problems?
If the conditions are to benchmark people doing things the wrong way then I rather doubt the value of the conclusions.
What commercial apps on Linux did he use, exactly? I just looked over the report, and I saw Apache, PHP, GLIBC, and MySQL. I'd argue that comparing MySQL to MS SQL Server is like comparing a bicycle to a BMW, but still, MySQL, PHP, GLIBC, and Apache are probably the best supported Linux-based apps on the planet. Did you even read the report?
Is a Linux study funded by GNU/FSF/OSI/OSDL or whatever any more impartial? No. Do you have problems finding people doing it?
Yes. As far as I'm concerned, they are whores, just like this guy.
[sarcasm]What? Wouldn't they be afraid to be considered communist hippies?[/sarcasm].
Oh! You assumed I was going to answer some other way. Huh. So much for your whole damned post having any point at all.
Information wants to be anthropomorphized.
Its called integrity... I take it you've never done scientific research before (and if you have, shame on you)
-everphilski-
Is a Linux study funded by GNU/FSF/OSI/OSDL or whatever any more impartial? No.
I think many here would disagree. Nonprofits are not driven by motives which could be considered the mirrored opposite of commercial corporations. There is not the tremendous pressure to turn a profit (or some analog to monetary gain), and in your examples they're run by mere handfuls of individuals receiving very little compensation with only their reputations to fall back on. They represent what are largely hobbyists, almost to a maddening degree.
OTOH, in Microsoft we have a callow and selfish for-profit entity with a rather abusive track record right up through their financial, er, daliances with SCO.
Need I say more?
Given their dynamics and history, being so dismissive of FOSS organizations as to just say 'well, eveone's biased anyway' really doesn't seem like an acceptable attitude.
If you actually read the report, you'd see that GLIBC was all mucked up because SUSE's YAST was broken. And on top of that, part of the study was to see what the administrators would do. Part of the confusion for the sysadmins was WHERE to get the sources when the standard RPM manager broke. It's not clear where they should have gotten GLIBC, and that was part of the test.
"Could you please explain why you decided to risk drawing your objectivity into question with insane paranoiac Slashdot readers . . ."
Corrected. I know it may seem like a troll, but I don't think it is. Something that a lot of the readers of this site don't understand is that not everyone thinks that Linux is the shit to the point of denying all evidence to the contrary. Don't get me wrong, I have one Linux-only computer that I use for work, my other is dual boot, and I like it. I love Linux both for its principles and because it allows me to do things that I can't normally do with Windows, BUT that does not mean that I believe its raw performance to be equal to that of a more heavily funded operating system. And you know what? That's OK. I'd still rather use it.
First, let's recognize that anyone experienced enough with both operating systems will have their own experiences that will tell them which OS is better in various ways. These people are unlikely to be swayed by studies. Therefore, the first thing that is critical to understand is this: these studies are aimed at people who are NOT experienced with both OS's.
As such, it seems there are two potential groups who are targeted by such studies: 1) CIO or sysadmin types who are experienced with windows systems, and who were thinking of trying linux; and 2) PHBs. For the first type, the MS studies are meant to deter. For the second type, the MS studies are meant to indoctrinate.
For example, let's say MS saturates WSJ, Fortune, and similar newspapers/magazines likely to be read by PHBs. They read it enough times, and given they have no field knowledge of the various TCO variables, they believe what they read from seemingly "objective" sources. What MS then wants is this: when an intelligent CIO or sysadmin goes to the CEO and says "Let's try linux, it's great!" the CEO says no, and considers the CIO incompetent for even considering such a blatantly horrible idea.
So basically these studies are meant to influence decision makers who don't have hands-on knowledge. It's a very good idea, really. It will keep Linux adoption a lot lower than it would be otherwise.
I think many here would disagree.
Hang on, you're saying you believe that you would trust a FSF or OSDL-funded study to be impartial? You're saying that if the FSF funded a study comparing GNU to Windows, and the study came back saying "Windows saves you money in the long term, and Microsoft's Shared Source is as good as Free Software for 99% of users", that the FSF would then be happy to publish that study?
I don't think so, and I suspect you won't either, if you pause to think about it.
Nonprofits are not driven by motives which could be considered the mirrored opposite of commercial corporations. There is not the tremendous pressure to turn a profit (or some analog to monetary gain), and in your examples they're run by mere handfuls of individuals receiving very little compensation with only their reputations to fall back on.
But that doesn't make them impartial! All it means is that the profit motive is replaced by other motives. And there are plenty.
Think about how much time the major contributors to free software projects put into those projects. Hours, days, months, years of personal time, freely given. Time that could have been spent earning money, or doing charitable work, or even just spending time with their families. Time that was wasted, if it turns out that the software they produced is not actually going to help many people do anything at all.
When you reach middle age, and the end starts to heave into sight on the horizon of your life, you start to get very, very uncomfortable about the idea that you might have devoted your precious time to an unworthy cause.
Being so dismissive of FOSS organizations as to just say 'well, eveone's biased anyway' really doesn't seem like an acceptable attitude.
What's dismissive about that? Microsoft really does think that everyone ought to use Microsoft software, and the FSF really does think that everyone ought to use free software. Everyone is biased. Pretty much everyone does have a pre-existing investment, either of time or money, in one of the options. And human nature does dictate that when you have an investment in something, you are biased towards accepting studies that support it and disregarding studies that don't.
What's wrong with telling the truth?
Maybe he did it because HE understands that research should not be guided by popularity. Go ask Galileo or Pythagoras.
Fleur de Sel
Dr Thompson, Thanks for sticking your neck out. My question is: has your research given you enough data to provide feedback on other flavors of Linux? I suspect that Microsoft chose to pick on light-weight Novel since their flavor or Linux is one of the relative weakest. Any thought on that?
Horns are really just a broken halo.
This is utter bollocks. See my analysis of the report in this comment.
They broke RPM by hand compiling glibc, not the other way around. It says so quite explicitly. They hand compiled glibc because they were asked to install (without upgrading to SuSE 9) an application that wasn't compatible with the version in SuSE 8.
The two are largely equivalent.
I use emacs gdbsrc mode to debug my code, and I can set breakpoints, conditional breakpoints, step in, step over, print any expression, or call any function I want in the debugger. If I recall correctly, you cannot really manually call functions in the Visual Studio debugger, but correct me if I'm wrong.
There are also advantages to gdb frontends though:
Please explain what extra productivity or features you gain from the Windows debugger.
As for your selection of tools:
The point is not that Linux is inherently less powerful.
Its that for certain kinds of purposes, the current situation in the real world, is that, for no good technical reason, software only exists for Windows.
Due to this unfortunate situation, Windows is superior at achieving certain real world tasks.
People who just accept this and go through the path of ethical lazyness get bitten in the ass by the lockin they are themselves creating.
Copy: Ctrl-C
/etc. User config files are in ~, usually as .application or, perhaps, in the case of KDE, in .kde subdirectory. Again, in almost 8 years of use, finding a config file has never been an issue.
Cut: Ctrl-X
Paste: Ctrl-V
That uses one buffer (i.e. KDE's or Gnome's buffer). Not only is it standard, but it is the exact same shortcuts as those used in Windows!
X's buffer is used by:
Select: Copy
Middle-mouse : Paste
You don't need to know about one to use the other. They work independently of one another. I've been using Linux since 1998 and I've never has a problem cutting and pasting between application under Gnome, KDE and Ion3.
Config files? System wide are in
I find it odd you would choose these things to complain about. Now, if you want to talk about wireless drivers, it's a whole different ballgame... =)
Dr. Thompson.
You note yourself, in your study that the sample is based upon 6 system administrators/systems. That number is, as you yourself note, too small to be considered definitive. That being the case I would argue that this makes the report viable not as a decisionmaking tool but a marketing tool. Were I a CIO I would feel unwilling to base my conclusions soley on a sample size of 6. What is your opinion on this? Do you expect further, more statistically-significant, work to take place? Or do you feel that this is not a problem?
Your study is interesting, but without knowing the 3rd party tools and applications that were used in the test how can we know the results are valid? Without disclosure the results are irreproducible. My hypothesis is that many of the applications were very poorly supported for linux and well-supported for Windows, but without knowing the applications I can't know if this is true or not.
Hello Dr. Thompson
;)
First of all, thank you for participating in this flamefest
I read the Executive Summary of your report and skimmed the rest, so pardon me if I failed to notice something vital.
It seems to me that the demand that your Linux Admins were asked to upgrade Glibc led them to fail the majority of tasks, creating an artificial bias against Linux.
Any Admin worth his weight in pizza knows that you Just Don't Do That.
If you absolutely, positively need some component, you get the version which works with your Glibc. All hell will break loose as soon as you upgrade Glibc and especially if you don't recompile the rest of the system. For an organization which needs commercial support from the OS vendor, this is unacceptable and your Admin should have refused to comply. If your web programmers need a specific component, they should get the component which works with your system.
I understand that this induces "pain" on your organization, but that pain should be much milder than the one your Admins experienced, and as a result, your organization.
My question is therefore: How can you defend the demand to upgrade Glibc when it is so obviously designed to force the Admin to fail?
Thank you very much for your answer, I look forward to reading your reply.
Hi Hugh, Dr. Thompson, I really liked reading your study. I thought it was well written and setup a nice framework for studying Business Solution Reliabilty. I would like to as you these three questions: 1) When Novell bought Suse they got pretty late into the linux game; about 1-2 years ago (not sure). I am not familiar with Novell/Suse's offering but i am familiar with Red Hat, which has been in the Linux game for a much longer period. The RHN works very well to update key components smoothly... just as well.. if not better than Windows Update. Red Hat should have been picked, but instead Suse was picked, which i believe is like comparing apple to oranges, because Suse/Novell's offering is just too new for a fair comparison. I believe, the study would have been quite different if Red Hat had been picked. -> Why was Suse picked? 2) Study fails to mention the specific software components that were installed citing them as not being relevant. This is major source of bias, since the software components themselves could have been created by software manufacturers who had a higher priority on focusing compatibility with windows than with linux. Since the software vendors were not mentioned... it is impossible to verify if the software vendors were equally committed to create good software on both platforms. -> Why is it unimportant to include the Software vendors? 3) Study fails to measure # of reboots in reliability study. It is not an opinion but a fact that windows requires a lot more reboots than linux when making changes to the system, such as updating key components. A reboot should also be considered as downtime, but wasn't included in the study at all. For example if, if every reboot takes 2 minutes, and windows required 10 reboots and linux only 2; this should be added to the timeline. -> Why were reboots not considered? I am looking forward to a response from you. warmest regards, Daniel
There's a number of huge distinctions between the open source community, such as the FSF, and Microsoft. One of them is money: the other is that Microsoft has been caught tiime and time again lying in court, under oath, and breaking the clearest laws of intellectual property ownership, trade secret theft, coercion of witnesses, and fraud.
The FSF keeps its nose squeaky clean, because they know they have to to keep any respect from their members and from the world at large.
Dr. Thompson:
According to what I have read, you did not use the most recent version of Redhat Linux in your comparison. Also, the specific hardware build you selected was one that is publicly known to give very poor performance with Redhat Linux.
Why did you select this version of Linux, and why this particular hardware build? More to the point, what decision-making process do you use to ensure that the OS and hardware configurations you use for your studies are good matches for one another? I guess what I am really asking is, what measures do you take to prevent the "bad match" of which you have been accused here on slashdot? And please do be specific, we love the details.
Thank you.
When you reach middle age, and the end starts to heave into sight on the horizon of your life, you start to get very, very uncomfortable about the idea that you might have devoted your precious time to an unworthy cause.
Which is supportive evidence that your argument is too biased in itself to consider.
How is helping to produce freely given and very secure software for EVERYONE ELSE TO USE FREELY an unworthy cause? Indeed, one could (and I do) consider it "charitable work".
Oh, and I'm on the approach ramp to middle age; and my feelings about helping only get stronger year after year.
It's not the fact of bias that you're not considering, it's the *reasons* behind the bias. (Yeah, I've heard a lot of people say that altruism doesn't pay the bills. But as a blanket statement, that's bullshit).
SB
It's old. The more humans I meet, the more I like my cats. At least they are honest.
I didn't play my thoughts out to their conclusion in my post, and for that I apologize. Let me write the rest of what I was thinking here.
What I meant to say was that, hey, this guy has a pretty good reputation for putting out good studies. Insane paranoiac people will say, he is getting paid to deliver a certain conclusion! Normal people will say, this guy has a reputation for honesty, I trust what he is saying. That is what I was trying to convey in my post.