Slashdot Mirror
The Mindcraft Debacle: Part MCXVI
Quite a number of people wrote in to alert us to the Salon story about the Mindcraft survey. Andrew Leonard, does a great job of tracing the trail, including the information about how Mindcraft is asking for help from Linus, but is not apparently giving him the information that he needs. Ah, the joys of industry. Mindcraft has agred to re-do the tests, with Linus' help, as aforementioned, but is continuing to be...less then forthcoming about the tests. Meanwhile, Microsoft is trumpeting the original study.
Can anyone else picture Gates wringing his hands at the pure joy in seeing the results from this round of testing?
"Even with help from the top two Linux minds, namely kernel inventor Linus Torvalds and top lieutenant Alan Cox, Linux was unable to match the performance of the NT machine under test."
If Mindcraft is being as unhelpful in giving information to these two as they were in replying to Usenet posts for help, you're going to see a steaming pile of FUD right in the middle of that report.
Let's face it, the Mindcraft results are going to say EXACTLY what the Microsoft marketing forces want them to say. If the marketing people say there's a perception that Linux is difficult to get support for, you can damn well bet you're going to see forced substantiation of that claim in their report notes.
This second test, if it continues under the same sort of conditions that Salon notes, will NOT be any better than the previous. In fact, given the level of expertise being "consulted" in the new test, this result could hurt the Linux image much, much more.
I thought I wanted a career. It turns out I just wanted paychecks.
The original test used quad-Xeon machines and I'm
sorry but I think that's unrealistic -- how many
shops use quad-Xeon-processor machines for web serving
and file serving? I think dual-PII-processor is the
more common workhorse in this industry, and I'm
pretty confident Linux beats NT on dual-processor
hardware.
They should be testing "typical" hardware
platforms rather than using hardware so maxed-out
that hardly anyone would own such a machine
these days, or at least not for plain-ol'
web serving -- database serving maybe but not
for web serving and file serving -- that's
ridiculous!
...and I think the whole point of Linux is you
can run a pretty fast web server on *cheap*
hardware so you really don't need quad-Xeon
w/ 1 GB of RAM to serve some serious enterprise
level web traffic anyway -- a plain ol' pentium box
will work just fine.
Before this Linux testing disaster, Mindcraft had performed similar comparisons between NT4/NetWare 5, NT4/NetWare4.11 and NT4/Solaris. They are, historically, a professional, unbiased testing shop who can accurately perform tests that will produce whatever Microsoft wants - Microsoft seems to be their largest paying benchmark customer. They also did some tests for Netscape, too, "proving" that Netscape blows away NetWare for directory management.
You'd think that if a company pays for tainted benchmarks, they could at least ask Mindcraft to cook the numbers less blatantly.
THe best way to do this test, I think, would be to run the NT benchmarks, then let Linus, Alan, Brian, Andrew, and Dean (I apologize for any gurus I forgot) have their way with the same server for testing. Let the Mindcraft stooges watch. They may just learn something.
-- Josh Turiel
"2. Do not eat iPod Shuffle."
Since RedHat and and other Linux based companies have the most to lose from this, i suggest that they do a test on similar hardware and publish everything that they did, and what results they got.
However waiting for a company to do something is a stupid way to go. I suggest some non-profit linux organization find out how much it will cost to purchase one of these machines, and create a fund where people can contribute money towards its purchase. Then allow folks who know about configuring it contribute information on how to do it. Publish the report, and bam, end of story. The machine could then be used for other linux related things such as hardware for linux developers to work and optimize on, or other such things. The potential for this is great since if its alot of folks will be willing to contribute to it. hell i got $10 bucks to go towards this beast.
and i am sure that the money could be raised from slashdot users alone, much less the entire community, since once the project is started, companies will try to get some press coverage by saying they helped in the project.
Anywho this is my humble opinion, and what do i know?
Its spelt "L-I-N-U-X", but pronunced as "Free Beer"
It worries me to read in the Salon article than Linus and Alan Cox are trying to give advice to Mindcraft for a second test, but Mindcraft isn't giving them any real access.
It worries me because I think the Linux community did a very good job of refuting the ridiculous results of the test, but if they redo them and can claim that Linux lost even with the help of Linus and Alan Cox . . .
Allowing Mindcraft to do another test gives them a legitimacy that they don't deserve. I would much rather see VA research do a test, they would do an excellent job of tuning Linux, and in fact I would trust even trust them to be fair to Microsoft.
Summary: we debunked the Mindcraft benchmark, lets not play back into their hands by taking them seriously and giving them a renewed credibility. Please, VA research or someone of that ilk, come out with your own more legitimate tests.
Furthermore, his Gamasutra profile says that he is a Microsoft Performance Engineer. Now isn't that damning?
This Will character has been traced even further.
There's an old quote, attributed to Ghandi, and often cited in linux advocacy:
These four stages of victory map nicely onto what's happened and is happening between Microsoft and Linux:First they ignore you. Microsoft did this for years, not really surprising anybody.
Then they laugh at you This comes from from all levels, including Gates himself, and mostly takes the form of FUD. Do you really want an OS developed in some guy's garage? Is linux going to be here tomorrow? Linux has no roadmap. Can linux really be well-tested? I consider all this FUD to be a form of laughing at linux; it's really too indirect to be considered fighting.
Then they fight you The Mindcrap Affair places us full-square in stage three. Microsoft is actively fighting, trying to bloody the penguin's nose (beak?). Granted, they're fighting dirty, but they are fighting. And get used to it; we're probably going to see a lot more of this from Microsoft.
So why is this good news? Because it puts linux one step closer to stage four: Then you win. And it won't happen soon enough for me.
--JT
I can't believe the fuss.
Micrsoft so proud of the survey while everybody knows it's fake.
Saying that "NT is for mission critical applications" and is stable, is the stupidest thing ever.
(I wouldnt trust NT to feed my cat, and I don't even have a cat)
People know that they lie, and they become even less credible.
I think they make a joke of themselves, and it's quite funny, too.
In their struggle to survive Linux, they are being fools, and that will cost them costumers.
People are starting to realize what Microsoft is about.
Their crap bloatware is getting on people's nerves,
and the legal tactics are now more understood by the public (thanks to the DoJ).
But more of it, people are sick of being fed lies, and obviously bogus facts.
We don't want fake videotapes and false promises.
I'm glad to see Microsoft like that.
I'm just waiting for the big crash.
---
---
I'm going to live forever, or die in the attempt.
Between the overt attempt to fake a "groundswell" of support, the bungled videos in front of the DoJ and now the "benchmarks" that "prove" Linux is inferior to NT (as shown by an "independent" testing lab, no less), the Emperor's new clothes are revealing what Microsoft *really* means...
Chris
So Buddha walks into a pizza parlor and says: "Hey, make me one with everything."
All these benchmarks have done for me is question the integrity of both Microsoft and Mindcraft.
From now on I will take all claims made by Microsoft with a bigger grain of salt.
Microsoft will survive this debacle, but I'm sure that Mindcraft will take a serious blow. With their integrity questioned, who will take their tests seriously? I don't think that they'll be getting too many customers now.
Here is a suggestion for the Linux community: Someone should start a Linux hardware benchmarking and testing project. The role of this project would be to provide tips for tuning different configurations of computers for different purposes. Maybe they could get VAResearch to loan some hardware for the tests. The project team could challenge people at Microsoft to beat their results in head to head competition with impartial judges.
Microsoft has never made such serious mistakes in so short of a period of time with the rest of its competitors.
;)
you weren't paying attention to the doj trial, were you?
I think that pretty much sums up the whole situation. And here Microsoft is trying to tout this result as something valid when the whole Linux community is up in arms over it, the press is outright calling the results a sham, and even Mindcraft is now wanting to redo the tests, most likely to get some respect back. I hate to say it (actually I LOVE to say it) but Microsoft is again making themselves look stupid by demeaning Linux. Hell, I just started running Linux full time after playing around with it for a year, and I have to say I've never been happier with speed, stability, and performance. Let real world examples show Linux is better than NT.
Jack
Demona's Law - "User data expands to exceed available bandwidth." ("User data" being pr0n, mp3's, vob's,