Slashdot Mirror
Embedded Systems Study Rebutted
Gabba writes "LinuxDevices.com has a rebuttal to the Microsoft-funded report purporting to show Windows nearly 4X more efficient than Linux for developing embedded systems. The rebuttal shows the study to be full of flaws in both design and execution."
This is a tried and true method in the scientific community: if you disagree with the conclusions of a study, you can always call the methodology "flawed". That way, you never have to pay attention to results that are different from what you believe.
Toronto-area transit rider? Rate your ride.
...but needs PUBLICIZING. As in, to non-geeks. Specifically, pointy-haired bosses.
Wanna bet the pro-MS article will be the one most PHBs will have come across their desk?
Honey, I shrunk the Cygwin
The rebuttal shows the study to be full of flaws in both design and execution.
/. for fair reviews of items provided by stores they have no relation to.. now where can I order a Zero Blaster again? :D
;)
I'm not being funny here, but since when did anyone ever pay attention to Microsoft backed or funded tests such as this? They'd hardly be backing this comparative benchmarks and reviews only for their product to get slated. Every time I read a 'Windows 2003 Server is 2.3x cheaper than Linux!' type story (where they end up comparing to something like Solaris which.. duh.. isn't Linux!) it just bugs the heck out of me.
I'd much rather cast my attention to impartial, un-biased sites such as
And yes, I am just joking around before I get flamed to hell and back
"Hey! Unless this is a nude love-in, get the hell off my property!!"
after seeing ozzy trying out the iDrive on his BMW, I would guess it may not be more usable or more reliable
My Dad works with voice mail systems, if its loaded with winnt it can take up to 30 minutes to boot. With linux it takes maybee 5. One of the problems with windows is the inability to strip out stuff you don't need.
Windows XP = money Linux = free to download
I don't see where the money argument comes into play here? Before someone says something about TCO let me point out a humorous but true text on how Microsoft actually kills (link is a google cache).
The report includes data from a survey of 100 manufacturers using 32-bit processors in a range of embedded projects and applications -- 50 using various implementations of embedded Linux, and 50 using Microsoft's Windows Embedded platforms(Windows CE .NET and Windows XP Embedded).
Rubbish rubbish and more rubbish. They shouldn't have been so biased with the study. Which manufacturers were used? Give it a rest now MS. It's obvious for one if MS funded the study, it's bound to be swayed, however if they didn't fund the study, depending on the vendors, it's still bound to be swayed. Remember MS violated antitrust forcing companies to go MS or go to bankruptcy court. How is one supposed to believe any studies they'd do?
I'm sure someone else is going to post a very good thorough post but we all know this is nothing more than utter bs.
MoFscker
First, let me be honest. I just skimmed the LinuxDevices article and didn't read the Microsoft article.
One thing I've noticed among PHBs is an ever-broadening definition of "embedded systems". I've seen more than one project go down the road of using a cPCI system running Windows NT 3.51 (yes these are current systems running this old version) on a harddrive. These systems are calling themselves "embedded".
This has been especially in systems that had serious size, weight, and power needs. Had I designed the system, I guess I would've used something like QNX or Linux on a much smaller processor, compact flash card, etc.
I guess my point is that these days it seems like general-purpose computers are being called "embedded" when I see embedded as much, much smaller (e.g. no moving parts, a microcontroller, etc...).
I dunno, I'm rambling...
On a consumer level, there are no real benefits to using Windows on embedded controllers, or even developing for evaluation boards.
Although I deviate from Linux in this example, it is still relevant to all open source embedded solutions. A few questions: is there an implementation of Remote Terminal Services for embedded versions of Windows, for easy manipulation of the embedded device in question? If so, what sort of licensing costs are implied?
As demonstrated numerous times before, open standards such as VNC are superior in the aspects of platform-ubiquity, openness, freeness, and simplicity. A shining example of what would be a costly, if implemented, solution, under Windows would be the Ethernet board running Contiki.
Oh yeah, and how many simultaneous threads, per-process-threads, and processes, do embedded Windows products support?
One must also compare the existing products that can be compiled between embedded Linux and Windows. I'm willing to bet software written with POSIX in mind beats Windows.
Excuse me if these speculations seem a bit armchair.
-
And the Angel said unto me, "These are the cries of the carrots! The cries of the carrots!"
I hadn't seen the original report, but as soon as I saw it was funded by MS, I assumed it was marketting fluff and moved on. Now I see that the "rebuttal" is on LinuxDevices.com, and was meant to specifically prove the MS report wrong and that Linux is better. That means it's more marketting fluff so we can move on.
When will people relaize that MS is not the only people putting out biased reports. I put the same faith in a "Linux is great" report by a Linux group as I do in a "Windows is great" report by MS.
"Information wants to be expensive" - Stewart Brand, the same guy who said "Information wants to be free"
Linux-biased study finds that Microsoft-biased study was full of shit. Anti-Microsoft-Biased readers of Slashdot agree. Film at eleven.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
I didn't catch the original, and I don't know where it was publicized or anything, but I imagine it was in places that make it look legit. More importantly, it was in places that people who don't already use Linux might be likely to look. Rebuttals like this are nearly useless unless they are prominently placed in some widely read medium--and by "widely read" I mean someplace that at least an average techie, even if entirely M$-biased, would be likely to at least see the headline. I admit that I have no knowledge at all of the world of journalism, online or otherwise, but I think that people who write studies/rebuttals/articles/etc like this, showing up Microsoft and their precious status quo, should make significant efforts to get them in mainstream media.
Unless someone already likes Linux, they're not likely to frequent LinuxDevices.com. Someone who already likes Linux is not the target audience for such journalism, or shouldn't be. We need to target it at the others, the people who don't like Linux, because it's articles like this that might make them like it, and it's studies like the one it's rebutting that make them not like it.
Dan Aris
Fun. Free. Online. RPG. BattleMaster.
Microsoft says Windows better than Linux! Linux advocates disagree!
In other news, a communist revolution has shaken Russia, and Napoleon suffered a shocking defeat at Waterloo.
Cheers
-b
This is a tried and true method in the business community: If you can't get a fair study to show the conclusions you want, hold an unfair study. More people will pay attention to the results than the retraction.
At our company we use GNU/hurd on our embedded operating system. Everybody laughs about how gnu/hurd sucks but its perfect for embedded stuff. We produce Small kde-based web terminals for cyber cafe. On a Yontrix 386 compatible processor with 2 Mb of ram, it boots off a 1.44M flash rom. The KDE is a stripped down version of kde 1.4 with just Konqueror, kedit, and kicker installed. No Xfree86, a EGA frame buffer is used. If you thought KDE was bloated, just see our web termials . Since hurd is completly free and easy to code for, it took us only two hours of programming time to make this system compared to 3 weeks for a linux soloution..
Writing a rebuttal is nice, but like retractions in newspapers, they are not all that effective in undoing the impression created by the original report. Aren't there any studies out there showing Linux is cheaper? Relying on "Linux is free" is no longer sufficient.
The study goes on to prove that the concept of "friendly competition", which surprisingly many other companies follow, benefits end users the most.
Okay, a lot of pro-Linux studies have their own problems (frankly, I don't put much stock in "studies" any more, especially vendor-funded ones).
However, the numbers this one used are *ridiculous*. Total Cost of Development?
Okay, let's see. Of the parties surveyed, cancelled Linux projects cost more than CE projects. This Jerry guy (he's got a PhD, so he must just be dishonest, not stupid) then uses this as a basis to claim that Linux is more expensive than CE. He's got to be kidding me.
By the same metric, all C++ software should be replaced by bash equivalents. Why? Because the average cost of a bash-based project is much, much smaller than the average cost of a C++ project. Of course, there's the little additional detail that the sort of projects one uses bash on are much, much smaller and simpler. That is, of course, the factor that makes the huge difference. However, you can conveniently ignore that tidbit.
Somebody tried to do the same study with Windows and some Sun servers back in the day to show that Windows made a far cheaper server. Well...yes, but most of the servers being used to average out Windows cost in the study were small, departmental servers that nobody was spending much on. The Sun servers were the far more powerful and capable systems for things like eBay's back end that had technicians swarming all over 'em. Sure enough, the Windows boxes had a lower average maintenance cost.
Average total cost is *totally useless* without some additional constraints so that you're measuring average cost of *similar projects*. If you took all PVRs with roughly equivalent feature sets and examined cost based on embedded OS, *then* you might have a useful study. The current one is totally useless other than for FUD use.
May we never see th
One reason nowadays that most embedded developers host their development on Windows is that most embedded tools publishers only make their tools available to run on Windows. Often these tools, like compilers for arcane chips, are quite specialized, so the developer is left with no choice.
That said, I think whoever wrote the report is on crack if they think Windows is a better development environment than Linux. I have been doing embedded for a year now, and one of the main things I still dislike about it is that I have to do most of my development on Windows.
It is quite common for each compiler vendor to write their own integrated development environment, with an editor and integrated build system. But the market for these products are not as great as the market for IDEs for the development of desktop or server software, which means they can't invest in developing a more refined GUI for their IDEs, so their basic usability and quality is quite poor.
If you think Visual Studio is a lousy environment for development, you should try the ARM IDE or TI's Code Composer Studio. Using them is like pounding nails with your fists.
However, the situation is slowly starting to look up. GCC targets many embedded CPUs and is starting to become widely used for embedded development. The other GNU tools also form a more or less complete set of what you would need to develop embedded products, with GDB acting as both a debugger and simulator, LD able to function as an embedded linker, being able to do two-machine debugging with GDB and so on. Also there's GNU make, CVS and so on.
The result is that while I had to use the proprietary (and expensive) ARM compiler to develop for the Oxford Semiconductor ARM7TDMI-based 911 FireWire/IDE bridge chip (which allows you to hook up inexpensive IDE disk drives as firewire storage), they switched over to building their firmware with GCC for the 922 USB/FireWire/IDE 922 bridge chip.
I've been using GCC under Cygwin for my 922 development, but a CD with a new SDK on it is expected to arrive in the mail any day now. When it does, I will have a choice of Cygwin, Linux or Mac OS X development environments, all running the same version of GCC. And I'm very happy about that.
Most likely, though, I will use Mac OS X for my 922 development. I'd prefer using Linux to Windows, but if I can use Mac OS X, I'd prefer that to Linux, if for no other reason than the fact that the clipboard works correctly, as well as that I could use CodeWarrior to edit my source.
Maybe if I get real ambitious, I might write a CodeWarrior plugin so I can use the CodeWarrior IDE to compile my code with arm-elf-gcc.
(And don't give me crap about not using Emacs. I was an expert at Emacs when most of you were still in diapers. I still have my .emacs file which I first created in 1987. But I vastly prefer CodeWarrior's GUI text editor unless I have some reason to run a bunch of Emacs lisp code on my source file.)
Request your free CD of my piano music.
I guess if your definition of "embedded system" was a rack server set-up and your programmers were already Microsofted then you might make better progress with WinXP embedded.
Engineering is the art of compromise.
At my previous job, we were involved in several case studies that were paid for by Microsoft. They were called REJ reports (Rapid Economic Justification). Anyone who takes such reports at face value is a moron. Would anyone really expect Microsoft to fund a study that says their technology is inferior? I don't believe such reports really fool anyone. In my experience, they are primarily used by people who have already made up their minds, but need some technical "justification" for the executives.
But in this regard, I can't really blame Microsoft. Every big technical company has "white papers" and such on their web site that tell why their stuff is so much better than everyone elses. It's marketing material. So what? Everyone does it. It's just that Microsoft is damn good at it.
The linux community should not complain about how good Microsoft is at marketing, and just get better themselves!
Check here for a more thorough "factual" rebuttal, including my favorite quote from the original report...
No reasoning, no nothing, as to why Windows XP Embedded (which a lot of the reasoning of the rest of the report was based on). Why, might one ask, would someone do this? Might it have something to do with the fact that the royalty cost for Linux is $0, the royalty cost for Windows CE (in volume) is $2.60, and the royalty cost for Windows XP Embedded is approximately $100 per system?
Yah. OK. That's a bit like me saying I'm going to compare the reliability of Toyotas and Fords, but for the purpose of the study, only Toyota cars that don't actually run will be used.
I mean, really - the original report is so bad it's laughable. It really didn't even NEED a rebuttal.
I know software crashes occasionally, no matter who wrote it, but man, you ought to get your power checked or something, I mean holy shit man, My TV crash? NEVER! All three of my cell phones? NEVER! I don't have a DVD player, but my cable box? NEVER! My router? NEVER! I do turn it off when i leave for overnight or longer trips. WAP? I don't have one, but my cordless phone (landline?) NEVER! I don't have a Satellite Receiver either, but my microwave? NEVER! My automatic sprinkler system controller? NEVER! My coffee pot? NEVER! My VCR? NEVER! I could go on, but what's the point?
Yeah, he should check out his hardware/driver setup on his 2003 box, but you also need to remember 2003 has just been released, and is, for all intents and purposes,still BETA. Until it's been run in the field for a year or so, and the bug reports have been submitted back to Microsoft and fixed, it is BETA. Just try getting Exchange Server up and running properly, and with all the functionality of it running on 2000 server. It's a bitch, and even Microsoft admits that.
For those who describe their systems as 'boxen', do you order multiple 'boxen' of corn flakes also?
I know software crashes occasionally, no matter who wrote it, but man, you ought to get your power checked or something, I mean holy shit man, My TV crash? NEVER! All three of my cell phones? NEVER! I don't have a DVD player, but my cable box? NEVER! My router? NEVER! I do turn it off when i leave for overnight or longer trips. WAP? I don't have one, but my cordless phone (landline?) NEVER! I don't have a Satellite Receiver either, but my microwave? NEVER! My automatic sprinkler system controller? NEVER! My coffee pot? NEVER! My VCR? NEVER! I could go on, but what's the point?
I've had cable boxes crash too.
Anything with a CPU in it has the possibility of crashing. Particularly when that CPU is decompressing streaming data.
Cellphones? Yep, they crash. Do a websearch. Similarly for other devices. Power is not the issue; the complexity of the task is.
Your router hasn't crashed? Wow. That's lucky. Ever updated the firmware on it? Ever wondered why you had to? Heck, even cable modems and DSL modems can crash.
Do yourself a favor. Don't assume that because you've never seen it happen that it doesn't happen. Do a websearch for "Cable modem" +crash. Or any other similar device. They crash all the time.
As for your cordless phone? It's a simple retransmitter. Sprinkler system and/or coffee pot? Timers and/or simple sensors. Yet again, you're dealing with a state machine that can be described on a single sheet of paper. Microwave? Here's your microwave:
Set Power = 10
Set Counter = TimeToCook
While (Counter > 0)
{
Sleep 1 second
Decrement Counter
If Door Opened, Break
}
Set Power = 0
Not exactly an impressive bunch of code, is it?
What TV do you have? Mine's a Sony HDTV. It doesn't always wake up correctly. Yours is little more than a set of pulse generators - most of what it does is done with hardware, not software. Mine has a much more complex computer inside it than yours.
Yeah, he should check out his hardware/driver setup on his 2003 box, but you also need to remember 2003 has just been released, and is, for all intents and purposes,still BETA. Until it's been run in the field for a year or so, and the bug reports have been submitted back to Microsoft and fixed, it is BETA. Just try getting Exchange Server up and running properly, and with all the functionality of it running on 2000 server. It's a bitch, and even Microsoft admits that
Linux is therefore, still BETA, and always will be. Please.
Simon
Coming soon - pyrogyra