Mike Hall on Choosing Embedded Linux over Windows

prostoalex writes "Mike Hall from Microsoft asks the audience why they would choose Linux over Microsoft for embedded projects. He provides a point-by-point description of benefits of Microsoft's products (Windows CE and Windows XP Embedded) and points out that starting out on Windows-based embedded platform might save development and testing time."

Comments

In other news...

[mstefanus]

Linus Torvalds the creator of Linux asks the audience why they would choose Microsoft over Linux for desktop application projects. He provides a point-by-point description of benefits of Linux products (SuSe, RedHat, JDS, etc.) and points out that starting out on Linux-based platform might save development and testing time."

Even if it saves development time ...

[Basje]

... you still have to pay the windows licenses over the 250000 units you shipped last month. Linux can be shipped without having to pay those license costs.

Re:Even if it saves development time ...

[Spoing]

2. ... you still have to pay the windows licenses over the 250000 units you shipped last month. Linux can be shipped without having to pay those license costs.

No doubt, and I'll add;

...or pay for the extra hardware needed to run embedded Windows over embedded Linux.

...or, if you decide that even embedded Linux is too heavy, your Linux app can be much more easily ported to one of the other open or propriatory *nix clones out there (*bsd, QNX, ...)^

1. ^ mini rant: Except for Microsoft, it seems that *EVERYONE* is using unix-like operating systems. Palm was one of the last holdouts, and even they are switching. What's it going to take to have them cave in entirely and not just as a crufty though sometimes handy add-on layer?

Re:Even if it saves development time ...

[hey!]

Y'know, I don't create embedded systems, but if you create software this is a familiar kind of scenario.

Generally seapking, commercial software makes getting things off the ground easy -- it's an important competitive point and vendors pay attention to this. Open source stuff is usually poorly documented, has a fairly rough learning curve , but has many benefits that offset this. In a sense, open source sits somewhere between "make" and "buy" on the make/buy decision continuum. It takes a bit more sweat equity to get going, but it gives you more freedom down the road, provided you're still on the road.

Which software components you use is not purely a technical decision -- at least that's what I keep telling management. You have to look down the road and see the business scenario that you're expecting to be in. Supppose I could save two week on a twelve week development effort by using a commercial software product. If the software is only going to have a couple of dozen users, then from a business perspective the licensing costs and the impact of restrictions are relatively minimal. On the other hand, if I'm going to be equipping 2,500 users (much less 250,000), the two week difference is inconsequential.

Using software from a vendor means you are entering into a business relationship with that vendor. This has consequences. Small consequences for small projects, big consequences for big projects. One of the great things about open source is that you don't have to enter into a business relationship with a vendor; or if you do you can replace the vendor if you disagree. That said, Microsoft is probably one of the better software vendors to enter into a relationship with, leaving aside whatever you happen to think of their products. Their product strategies are way more stable than the industry norm (which isn't saying much) and as long as your product has no disruptive potential (disruptive to Microsoft's position or planned positions).

If, for example, I was developing a computerized milling machine, entering into a relationship with Microsoft wouldn't be a complicated business decision, and choosing/not choosing a Microsoft product could be done on a basis of pure technical merit. The number of units shipped are small, and the price of the units are high, so the cost of the license is hardly worth considering. What is most important, Microsoft has no strategic designs on the machine equipment market, so far as I know.

On the other hand, if I were entering the set-top PVR market, the decision would be driven almost entirely by business strategic concerns. The number of units shipped is large, and the margins tight. Furthermore, Microsoft has strategic designs in this area, and I could on one hand end up on the wrong end of the DRM stick if I don't play ball with them. On the other hand, I could find myself in the position of office suite vendors in the early 90s -- competing with very vendor who supplies the heart of my system.

In those kinds of situations, what you need really need to make the platform decision is a crystal ball. Which is why I avoid them. But that's the reason, I supppose, the world needs entrepreneurs.

Plain old FUD

[geirt]

He starts with:

I don't want this to turn into a "Linx is better than Windows is better than Linux" discussion, no throwing of mud or Fud

and ends with:

Much has been written about the pitfalls of incorporating GPL software into a product. An often overlooked consideration, however, are the costs of having to even worry about these licensing issues.

Re:Plain old FUD

[91degrees]

Yes, It is FUD isn't it. For certain applications it can be an issue, but all you need is to have a company web site with the Linux source code (including any modifications you made), freely available for download.

Also, he fails consider that Windows CE has licencing issues, and the SDK has its own licence. And you need a licence per developer for the OS that these run on.

Re:Plain old FUD

[jrumney]

Having to worry about licensing issues is a fixed cost for any development. Whether its the GPL or Microsoft's redistributable code license, the company lawyer needs to go through all the third party licenses and make sure that all terms are being complied with and it is not going to expose the company to unforseen risk.

Re:Plain old FUD

[arkanes]

It's true, but lets face facts - most business people are FAR more comfortable with ye olde "give me money" licensing model (I'm not familiar with CE licensing specifically most Windows licensing is of this type) than they are with anything else. There are people who would rather pay money and be on stable mental ground than even use a BSD license, much less the GPL.

It is not that simple

[curne]

A friend of mine works for a company that makes embedded systems and they chose a Linux kernel to drive it since they have to make hundreds of tweaks in the kernel code to compensate for custom hardware (that they build in-house).

IMO, choosing your software is not just a matter of point-by-point feature comparison. Some times you need the ability to modify the behavior, especially because embedded hardware is typically somewhat eccentric.

Re:It is not that simple

[Jondaley]

I don't know about XP embedded but for CE, we can modify a good portion of the kernel code. I don't know the exact percentages, but I would guess 80%.

Yes, there have been times that we wish we could modify other parts, but it hasn't been too bad.

WinCE 4.2 does seem to be more restrictive than previous versions -- haven't looked into it too much yet.

I don't know what the license agreements are in terms of modifying the so-called "private" code, as opposed to just the "public" code. If modifying the private code isn't allowed, then the parent has a pretty good point, as that probably shrinks the percentage of modifiable code to 50% or less.

Re:It is not that simple

[ClosedSource]

I think it's redundant to say an embedded system with custom hardware. What other kind is there?

It's not generally necessary to modify an OS just to support custom hardware. Historically, most embedded OS's were proprietary and not being able to modify the OS has never been a great limitation.

Perhaps the problem is that since general purpose OS's like Linux weren't designed for embedded systems, more tweaking is required.

The keyword here is...

[borfast]

The keyword here is "might".

[...] might save development and testing time.

Sure, if you're lucky...

What a crock ..

[naden]

The COTS article that the Microsoft guy keeps referring to is written is by Green Hills Software. You know the one that considers itself:

"the technology leader for real-time operating systems and software development tools for 32- and 64-bit embedded systems."

I'm sure they don't have an interest in picking holes in Linux.

Re:OT Unix - like OS

[the_greywolf]

in my understanding, a "Unix-like" operating system is one that complies either in part or in whole with the POSIX specification, but may or may not have been certified as such.

a "Unix" operating system is one that fully complies with POSIX and has been certified as such. in this case, it can legally carry the Unix trademark.

"modern" is probably the wrong word for anyone to call Unix. "mature" is more accurate, as it is a relatively old standard that has survived the test of time. Unix applications written in the early 80's would probably compile and run just fine on any modern Unix-like with few to no modifications (provided the compiler support exists, of course).

let's see...

[jeif1k]

Linux scales seamlessly from small embedded systems to high-end servers. Microsoft has two incompatible kernels and operating systems: Windows CE and Windows XP. In fact, Windows CE is so limited that it is nearing its end of life--maybe Microsoft will replace it with something else called "Windows CE", but you can probably throw away your software. Windows XP is a messy, bloated behemoth that really doesn't scale down well.

Linux natively uses POSIX-compatible APIs, widely used in the embedded world, Windows XP doesn't.

Linux has no per-copy licensing costs, Windows does.

Linux runs on a much wider range of hardware than Windows.

Linux has a mature, standard set of command line administration tools, invaluable for running and debugging a system over, say, a serial line. Windows XP doesn't.

Those are just off the top of my head. There are probably lots more reasons.

Re:OT Unix - like OS

[greed]

I think the important things about UNIX, aside from having the same libc.a APIs everywhere, is the filesystem.

To have a UNIX-like system, you need:

1. Single filesystem tree
2. Transparent mounting of filesystems into the tree
3. "Everything is a file" approach, all I/O types use the same APIs; so you only need special code for doing special things. (So "cat" can read from magnetic tape, raw disk, or a FIFO; but if you want to rewind the tape, you need a program that knows about it.)
   • Regular files
   • Directories
   • Sockets (TCP, UNIX, NetBEUI, AppleTalk, IPX/SPX--the difference is only in the names you use to open them)
   • FIFOs
   • Device-specials
4. fork(2)/exec(2)/wait(2) process model
5. "File mode" permission model (something that can be done with bitmasks), as opposed to an "Access Control List" model. (ACLs are available on many UNIXes, but very few programs support them. The OS still enforces them, but (say) "cp" won't preserve the ACL unless you update it.)

With those basic attributes, most batch UNIX programs stand a fair chance of working well.

I've worked on systems which implement only a few of them. Porting a UNIX combined file-and-network program to Windows is a massive pain, because the TCP sockets are not in the same identifier-space a s file handles--so you need to handle them separately. Similarly, any system which doesn't have "fork()" means you have to re-write all your co-processing code. AS/400's POSIX interface is particularly misleading--even more so than Windows. On Windows, you can at least run Windows .exe files fairly normally from the POSIX API; but on OS/400, non-POSIX stuff just doesn't work right (or didn't, back in 1997 when I last had to deal with it).

Even when there seems to be a good alternative, it may not work out well after all. So you can't fork()/exec() on Windows... well, so what, you just want to run a program and wait for it to exit, right? So spawn() is good enough, or CreateProcessEx() if you want more control.

But, launching a new process is very expensive on Windows; a make or shell program will be 10x to 100x slower on Windows than on a UNIX-ish system running on the same machine. When you write native Windows applications, you just don't code that way--you use threads, you use DLLs, you do everything you can in the same process image. You do NOT run external programs if you can help it. (You can, of course, do this on UNIX too if you like; but you don't have to....)

So even having [part of] the interface isn't going to save your code--if your application uses features which aren't handled well, or are simulated, it's going to suck on that system.

Control at Low Cost

[sirwnstn]

We had the unfortunate incident of having Windows CE and Advandtec runtime on one of our systems. The project manager was new, he wanted a "turn-key" system from the vendor, blah, blah, blah. At the time, it sounded like a good idea, and my boss relented. But now both he and I reget it terribly.

I will agree, the vendor had a really fast development time. I was happy about all the high level changes they could make in such a short time. It would have taken us longer to code something from the ground up. BUT, we ran into serious problems with the Advantec drivers, and runtime and then certain issues with Windows CE, memory leaks, and so on. The time it took for us to track down the problems and to deal with the vendor and Advantec costed us too much in man hours and late project schedules. The entire process was too painful and the worst of all, we had absolutely no control over the lower level software involved.

Bottom line: at our company, we like to have control over our software, down to the source code. Linux gives us that control and flexibility. That's why we will choose embedded Linux over WinCE any day.

Suffice it to say, we won't be using a WinCE solution any time soon, unless we are proved wrong later on.

Re:OT Unix - like OS

[Foolhardy]

1. The Object Manager provides a single tree for all named objects, including files. Win32 has its own directory in the OM of symbolic links that connect the DOS compatible drive letters to the actual devices. For files, Win32 just sticks \DosDevices\ on the front of the path before handing it to the native (syscall) api. \DosDevices\C: is commonly a symlink to \Device\HarddiskVolume1, equivalent to /dev/hda1.
2. The OM can be extended by any kernel mode component by adding a new object type. One thing that an object type can do is be reparsable; extending the namespace into an object as if it were a directory. One of the object types that the IO Manager exports are device objects. Device objects are reparsable and are used to extend the root namespace into filesystems that will return file objects.
3. Everything in UNIX that is a file is also a file object in NT, and many more kinds of things are also objects. Browsing through \Device with winobj I see a beep device, null, my cdrom, the gameport, my harddisk partitions, the keyboard, sysaudio, the video device, several protocols (IP, TCP, UDP, NwLnkIpx, NwLnkSpx), the named pipe filesystem (npfs.sys), the mailslot filesystem (msfs.sys), NFS and SMB server and client devices, and tons of other stuff. Each of these are device objects that produce file objects for client processes; objects under the IO Manager's domain. You can use the same read, write and ioctl functions on all of them. Some, like TCP rely heavily on ioctls but you can still use read and write on a socket handle (which is really a file open under \Device\TCP)
4. Win32 does not support fork directly (and neither do many popular shared Win32 libaries), but NT itself has no problems with any of those functions. Environment subsystems like SFU use the same syscall interface that Win32 does to make processes and such.
5. I don't think that this is really a requirement of UNIX. Yes, it should provide compatibility as needed for the old style permissions but there are UNIXes that use ACLs internally.

The programming style for UNIX or even a POSIX target is different from Win32. This isn't due to NT, but the Win32 subsystem and all the libraries that are initialized up startup for new Win32 processes. NT is perfectly happy to make a new process with a copy of the parent's address space with no new memory or threads or libraries or anything. fork() in SFU is as fast and efficent as it should be.

Porting woes

[Brandybuck]

At my company the Microsoft salesmen outmaneuvered out defensive blockers, and managed to talk to management. As a result, that thought that productivity could be vastly improved by porting a million line embedded LynxOS/Motif system to WinXPe. "Ten people should be able to do it in a year!" It's now three years later with fifty people and there is still no sign of a light ever being found at the end of this tunnel.

A minor anecdote to illustrate the problem. Yesterday I performed a code review on a trivial piece of code. I wrote the Motif version in three days. They wrote the Windows version in one month. My code (using the horribly "bloated" Motif) was 850 lines. Their code using MFC was 5200 lines of code. Before you blame an outdated MFC and recommend moving to .NET, the non-UI portion of my code was approximately 250 lines, while theirs was still over a five hundred.

Our experience with vxworks, XPe and Linux

[dmh20002]

we have been building radar systems with embedded processors for 15 years.

for parts of our systems that require submillisecond timings, we use vxWorks (and before that pSOS) because we can easily get to bare metal and control exactly what is happening with interrupts and threads. We use the very basic kernel of vxworks. its very expensive but we are relatively content for that type of application. We can't screw around with these applications so we bite the bullet.

we also have parts of the system that are real time but don't have such tight requirements (user interfaces, communications multiplexing etc). We used to use vxworks on those but the development system and licensing expense is horrendous. Then w tried windows XPe based on the false assumption that the commonality of our development systems (XP) and the target (XPe) would give us some savings. but Xpe was very hard to configure and get our applications running. It took a lot of training to get where our programmers could deploy applications. The development system was cheap ($1k) and the runtime costs were much less than VxWorks. But the development effort required negated those savings. So we ditched that after one program.

then we started using Linux on Diamond Systems EBX and PC-104 boards. Diamond Systems provided a stripped down slackware based flashdisk image with a configured 2.6 kernel, glibc and the basic executables. It worked like a dream. We develop on Fedora 2 workstations, and the executables move right over. We have written a couple of character mode drivers for some special hardware and that was easy too. It isn't 'embedded' linux, its just plain old linux. it runs easily on a 128MB flashdisk and 128MB RAM with lots of space left over. So we were hooked.

(it turns out we always deliver source with our systems so the GPL is no problem for us and our customer doesn't care either because they don't redistribute)

an personally, I used Sun UNix in the 80's and switched to windows in the 90's. Going back to a unix-like development environment kicked ass. I had forgotten how productive it was.

The momentum in our company is to switch to Linux for any new applications that aren't completely hardcore.

On a different note, however, I have a complaint with the 'embedded linux' vendors about their sales model. They don't publish prices, you have to 'negotiate' a price. They are cagey about 'open source'. They try to tell you that you have to buy a 'seat' for every developer. Why do that when the development systems they provide are open source? The most valuable service they provide isn't real time extensions, but IMHO it is the creation of turn-key cross-development environments and board support for embedded systems. Thats worth a lot, but from my experience they are unpleasant to deal with (maybe because I am a introverted engineer) and the expense of their tools negates any advantage unless you are doing a product that is going to be produced in the millions so that the elimination of runtime licensing expense makes it worthwhile.