Slashdot Mirror

... or you can program in CSP (communicating sequential processes) multiprogramming style which is an old technique even by UNIX epoch standards.

There are plenty of CSP-style large scale projects that really are assembled from discrete single-threaded programs; this works well for distribution across loosely-coupled clusters (and even topologically distant sets of those) whereas most multi-threaded programming paradigms make strong assumptions on things like expected delays and bandwidths and their derivatives (like uniformity). CSP is amenable to composition of single processes, much like discrete UNIX programs can be assembled into pipelines. (The QNX shell syntax for distributed computation and i/o is even more instructive as an analogy, or see the on(1) manpage and consider that "on" can form part of a pipeline.)

A very large example of CSP is Askemos which is a German Scheme-based multiprocessing/multiprogramming/concurrency system. Unfortunately for the wider world, finer-grained elucidations of Askemos's design in English are rare, in part because it's already in real, productive, profitable use in German-speaking agencies and organizations.

On smaller scales, many Mac OS X applications have separate processes (often written in different languages!) for the front-end user interface and back-end computation and persistence activities. Some of the most compute intensive software packages organize the back-ends into pipeline-like networks, which is classic CSP style.

The most significant drawback to CSP is that the inter-SP communication protocol must be appropriate to the job, and the risk is that it's too heavyweight compared to techniques like STM or lock-full shared memory. However, by analogy to transport layer techolonogy, TCP was considered far too heavyweight for LAN-based communications for many years, and nobody seriously considered using TCP for IPC on the same system -- nowadays, TCP is pervasive and cheap and is assisted by the low cost of compression and decompression (and even encryption) of the higher-layer data it carries.

In short, time-space tradeoffs probably will favour spending time to reduce the amount of inter-SP data exchange even given fairly tightly coupled multi-processor systems.

If you really want to see "slimming down the operating system", check out QNX, which is a true microkernel used mostly for embedded systems. The kernel just does memory, CPU, timer, and process management, plus interprocess communication. Everything else is optional. Networking, disk/file system support, display support, window management, etc. are all user-level processes that you can include, or not, when making a boot image.

The unusual feature here is that the components really are independent. You can have networking without a file system, or a file system without networking. If the machine has no display, you don't have to include any of the "console" stuff. Even error logging is an option, and can be connected to a display, a window, the network, or a file.

But this isn't what the original article meant by "just enough operating system". They're thinking more of bloated distros.

I hope "just enough operating system" means the ad-funded preloaded crap goes away. Remember Dell charging $50 extra to get rid of all that junk?

How about QNX?

Somehow, this operating system never became popular on the desktop, although we did try it out back in the days of DOS and XENIX...
.
Check out its entry on Wikipedia

However, I do know of a really good author, that is a "dead tree" author, for C: Herbert Schildt.

I used to recommend his books too, but he has a bad reputation among many developers:

Why do many experts not think very highly of Herbert Schildt's books?

A good answer to this question could fill a book by itself. While no book is perfect, Schildt's books, in the opinion of many gurus, seem to positively aim to mislead learners and encourage bad habits. Schildt's beautifully clear writing style only makes things worse by causing many "satisfied" learners to recommend his books to other learners.

Do take a look at the following scathing articles before deciding to buy a Schildt text.
http://www.lysator.liu.se/c/schildt.html
http://herd.plethora.net/~seebs/c/c_tcr.html

The above reviews are admittedly based on two of Schildt's older books. However, the language they describe has not changed in the intervening period, and several books written at around the same time remain highly regarded.

The following humorous post also illustrates the general feeling towards Schildt and his books.

http://www.qnx.com/~glen/deadbeef/2764.html

There is exactly one and ONLY one C book bearing Schildt's name on its cover that is at all recommended by many C experts - see Q 25.

I'm not aware of anybody actively selling/maintaining CP/M for customers but then again you never know.

However QnX seems to still have very active support and http://www.qnx.com/ is quite up to date (especially when contrasted with CP/M, which I would classify as 'mostly dead').

In the real time arena they are very well represented, and I think that without knowing it you probably use their products at least once or twice daily.

Think automotive, energy supply, industrial automation, medical equipment and such.

I disagree.. he wants a complete OS/environment.

http://www.qnx.com/

you can get it's complete kit free nfor non commercial use. is INSANE FAST at booting if you do it right and is small.

The OS has to be a microkernel, and it can't change much. The amount of trusted code must be minimized. IBM's VM has been stable for decades now, even though the applications have changed drastically.

Yup, OS/360^WOS/VS2^WMVS^Wz/OS has changed drastically over the decades. :-) ("VM" in the sense of "VM/CMS" is an "OS" that provides a virtual machines to the "applications" running on it; those "applications" are full-blown OSes, whether they're full-blown OSes that can also run on bare hardware, such as z/OS or Linux, or single-user OSes such as CMS.)

I don't know how "micro" the nucleus (kernel) of z/OS is.

The QNX kernel changes little from year to year

...and doesn't run on mainframes (the QNX Neutrino product brief lists x86, SH-4, PowerPC, ARM, and MIPS as processor architectures on which it runs; it doesn't list S/390 or z/Architecture, for example). It's an interesting system, but not really relevant to the engineering of mainframe OSes (real-time OSes, yes, but not mainframe OSes).

NT (like OSX) has a microkernel, but the operating system isn't just the microkernel. Most of OSX (for example) actually runs on UNIX which runs as a single application of the microkernel. NT also has an enormous number of kernel-entry points which means that it too is a monlithic-kernel-based system that happens to run on a microkernel.

A real microkernel-based system will have a lot of the userland facilities designed to take advantage of message passing and will probably look more like HURD or Squeak than it will like NT or NeXT. QNX and VxWorks are the only successful microkernel-based systems that I'm aware of, and frankly both of them are losing big to Linux, so we might have to say were the only successful systems in the future...

(2nd try.) The instructions for svn checkout of the qnx os source is here: http://community.qnx.com/sf/wiki/do/viewPage/projects.core_os/wiki/Os_source_guide

http://www.qnx.com/products/getmomentics/

The problem is, QNX management has said that before:

"The new QNX initiative consists of several key elements . . .

• In recognition of the ease with which developers can obtain and begin development with Linux, QNX has decided to make their new QNX Realtime Platform free "for non-commercial use". Developers can now download the software and its associated development directly from the QNX website.
• In response to the growing desire for source code that has resulted from the exploding popularity of open-source Linux, QNX will soon release the source code for many QNX applications, drivers, and libraries."

That's from a press release back in 2000.

Last time, the free version of QNX stayed around just long enough that free software developers ported their major packages to QNX. Then QNX management yanked it away.

Despite the new press releases, the QNX CVS source repository hasn't been updated in six years.

Here's an article that explain the license and model

http://www.qnx.com/download/feature.html?programid=16868

and an FAQ:

http://community.qnx.com/sf/wiki/do/viewPage/projects.community/wiki/FAQ

Here's an article that explain the license and model

http://www.qnx.com/download/feature.html?programid=16868

and an FAQ:

http://community.qnx.com/sf/wiki/do/viewPage/projects.community/wiki/FAQ

Yes, it's interesting that Larry Rosen is busy spreading FUD when he claims stuff like: Open Source licenses ... relinquish[..] all control over their intellectual property and giving it away for free (pg.2) or counterpoises OSS with ... the sustainability of a royalty-based business model for commercial projects {pg.2) A lawyer involved with licensing issues should be very aware that Intellectual Property is a garbage, meaningless term usually used by people that are seeking to assert greater control over information. He should also be aware that there are very many sustainable and highly profitable Open Source .... gah, let's call them Free Software ... projects. Red Hat springs to mind as the most succesful (their entire codebase, including their build-systems and bug-reporting systems are completely Free) as do MySQL AB (ever take a look at their dual-licensing model Larry?).

Basically, this guy is being paid to say "Hey, I'm an Open Sores guy too and I say it's OK, so drink up the tasty chocolate milk and ignore the funny taste from the proprietary ingredient".

QNX is awesome, but this isn't an Free or Open Source license.

Rosen has just disgraced himself.

The app itself must exist. Take QNX, for example - it has a GUI, but no apps.

I'd like a system where audio and video never glitch, and the GUI is always responsive.

Not only is it possible, but it's widely used in many RTOS. For example QNX lets you select scheduling algorithms on a per-thread basis: http://www.qnx.com/developers/docs/6.3.0SP3/neutri no/prog/overview.html#SCHEDS

QNX. Get a commercial license for it, use qcc and do whatever you want. It is perfect for the job. You can test development on the free evaluation which reverts to the non commercial (NC) version after a period of a month I believe. The "QNX Momentics 30-day Eval - QNX Neutrino x86 Host" would be the product to try. An application developed for QNX should also build on Linux\BSD\gcc with minimal configuration. Give it a try.

For starters, unix systems are not routers, they can be used as such, I use one at home. But for a backbone connection with millions of packets per second, they are a poor choice. They cannot keep up with a good cisco or foundry router.

For starters, unix systems are not routers, they can be used as such, I use one at home. But for a backbone connection with millions of packets per second, they are a poor choice. They cannot keep up with a good cisco or foundry router.

http://www.qnx.com/markets/security_defense.html

That is all, carry on.

No big deal. Someone should just show them the CLOCK_MONOTONIC trick (instead of CLOCK_REALTIME).

QNX does a good job of this, its package manager gui and package format get the job done quite well. Dependencies are handled and it can connect to multiple repositories, local or remote. I think its model could be well applied to a standardised Linux packaging strategy.

http://www.qnx.com/developers/docs/momentics621_do cs/neutrino/utilities/q/installer.jpg
http://www.qnx.com/developers/docs/momentics621_do cs/neutrino/utilities/q/qnxinstall.html
http://www.qnx.com/developers/articles/article_883 _1.html

Latest Docs:
http://www.qnx.com/developers/docs/6.3.0SP3/moment ics/bookset.html
(registration required)

QNX does a good job of this, its package manager gui and package format get the job done quite well. Dependencies are handled and it can connect to multiple repositories, local or remote. I think its model could be well applied to a standardised Linux packaging strategy.

http://www.qnx.com/developers/docs/momentics621_do cs/neutrino/utilities/q/installer.jpg
http://www.qnx.com/developers/docs/momentics621_do cs/neutrino/utilities/q/qnxinstall.html
http://www.qnx.com/developers/articles/article_883 _1.html

Latest Docs:
http://www.qnx.com/developers/docs/6.3.0SP3/moment ics/bookset.html
(registration required)

QNX does a good job of this, its package manager gui and package format get the job done quite well. Dependencies are handled and it can connect to multiple repositories, local or remote. I think its model could be well applied to a standardised Linux packaging strategy.

http://www.qnx.com/developers/docs/momentics621_do cs/neutrino/utilities/q/installer.jpg
http://www.qnx.com/developers/docs/momentics621_do cs/neutrino/utilities/q/qnxinstall.html
http://www.qnx.com/developers/articles/article_883 _1.html

Latest Docs:
http://www.qnx.com/developers/docs/6.3.0SP3/moment ics/bookset.html
(registration required)

QNX does a good job of this, its package manager gui and package format get the job done quite well. Dependencies are handled and it can connect to multiple repositories, local or remote. I think its model could be well applied to a standardised Linux packaging strategy.

http://www.qnx.com/developers/docs/momentics621_do cs/neutrino/utilities/q/installer.jpg
http://www.qnx.com/developers/docs/momentics621_do cs/neutrino/utilities/q/qnxinstall.html
http://www.qnx.com/developers/articles/article_883 _1.html

Latest Docs:
http://www.qnx.com/developers/docs/6.3.0SP3/moment ics/bookset.html
(registration required)

QNX's Neutrino is basically BSD layered on a microkernel and from what I've heard is the most highly regarded embedded OS out there. I don't know where you the got the idea you couldn't or shouldn't use a microkernel in an embedded system.

I've built a lot of web pages, and this has made me realize that it's incredibly quick and easy to whip up GUIs with HTML. The experience was much better than with the GUI builders I had used, and certainly beat coding GUIs by hand.

Of course, HTML is not intended as a language for describing native GUIs, so it has some limitations there. Fortunately, there is a variety of XML formats for describing real GUIs.

What makes XML so great for describing GUIs is that it's so good at describing nested objects. If you think about it, that's exactly what GUIs are: you've got your windows, with a bunch of widgets in it, one of which is a scrollable area with more widgets in it, etc. This is naturally described by an XML tree that contains all these widgets, with some attributes used for connecting them to the application; e.g. ids to allow the application to reference widgets, and embedded code to let the GUI respond to events (e.g. HTML's onclick).

Where many XML GUI languages fall short is in that they don't provide methods for building new abstractions. If you have a lot of subtrees that are all very similar (say, a frame, a title, a content window, and a hide and a show button), you'll completely have to code each of them in full. Any programming language worth its salt will provide a way to abstract over this (functions!), but I think the realization that XML GUI descriptions (and HTML documents!) are programs hasn't fully set in yet.

Next time I'm coding a GUI, I'll be generating the XML from a proper programming language. I've had good results with Lisp before...

I often see statements like "I'd wager 90% of the functionality for X is only used by 5% of end users.", with the implicit assumption is that you could remove 90% and only upset 5% of the users.

Unfortunately the math is not that simple. Quite often single users do only use 5% of the features, but it not always the same features for all users. So its possible that when you remove the least used features to get almost every user lose at least one feature they must have. What users really mean is: "I only use 5% of the features and I don't want to be affected by the ones I don't use." And "not effected" applies to UI design and security.

A system of minimal core functionality, plus opt-in, add-on plugins is a solution for this. The concept is applies to operating systems (e.g., microkernels like QNX) as well as applications. I'm not familiar with it, but I hope FireFox designed their plugin interface with the right balance of security and flexibility to achieve this.

Actually, if you rebuilt the i-Opener software in QNX with QNX 6.3, the current version, you'd get a new browser that supports more modern web sites. You just have to build a distro with the pieces you need, use "mkifs" to build a ROMable image with the components needed, and burn a new ROM.

It would cost about $8000 to buy a QNX development seat, but you could distribute the resulting software. You can get a free 30 day trial version, and after 30 days, the Eclipse IDE turns off, as does "QCC", the front end to gcc that makes it take POSIX command line options. Everything else still works.

Sadly, QNX backed away from their "open QNX" effort of 2002, and the system is now more proprietary and more expensive.

That is essentially what Apple has done.

No, it isn't. He's talking about a microkernel, like QNX or L4. Mach is also considered a microkernel, but OS X is not based on Mach; it is based on XNU, a hybrid kernel that consists of Mach bolted into the FreeBSD monolithic kernel, with I/O Kit thrown in for good measure.

You give performance arguments, but the speed of OX X does not imply that it uses a microkernel; quite the opposite, microkernels have been known to run significanly slower than monolithic ones, which is presumably why Apple went the hybrid route instead of using pure Mach. To be fair, Mach is rather outdated among microkernels, and "modern" microkernels like the aforementioned L4 have shown that the speed penalty might be as low as 5-10%, but the point still stands. Don't confuse simplicity of code with performance.

True, the fact that windows is bloated does not imply that all other operating systems are bloated. But the fact remains that they are.

Really ? What about QNX ? Then there are non-commercial projects like L4... How are they bloated, they're only a few KB ?

Not all developers write crappy unscalable, non-modular spaghetti code...

It's not really "image based". It's just another file archiving system, with one big file full of many compressed files. Like "zip" files. This is mostly a hype phrase, because "Microsoft announces new, incompatible compressed file format" would sound so stupid.

There are true "image based" systems. QNX has one. A QNX image, containing the microkernel, the servers, and any desired application programs, can be built, burned into ROM, and executed from ROM. This is how embedded systems start up, from copiers to routers to car navigation systems. Vista isn't doing that.

I was involved in a project using WinCE. All I was asked to do on day one was to set a breakpoint in the PS2 keyboard driver. Upon opening the ps2.c file I found a tangled mess of code, with sparse comments and little or no way of debugging, that had clearly been modified by 10+ programmers over the years, with each subsequent programmer adding patch after patch... Seeing the challenges we were up against.... We moved to an open source embedded OS.

Free (as in soda pop)

'Nuff said.

Many people talk about QNX [...] but we really shouldn't compare a general-purpose OS with real-time or special purpose OS.

but the fact is they have never had an OS that could replace Linux and other popular OS that everybody could run on their desktop with enough functionality.

Think about it.

If the disk subsystem crashes, how exactly do you get at the on disk program to start it with?

Oh, heck... that's easy. The code's already in memory, so all the kernel has to do is clear out the module's stack and heap space and restart it.

While it may not be possible to totally isolate every subsystem, with a microkernel subsystems should be more robust than in monolithic kernels.

The different between theory and reality is that, in theory, there is no difference.

Uh, OK. What's that got to do with this discussion? Every reliability-based OS on the market today is a micro-kernel. Because they're more reliable. That's not theory, that's practice.

Micro kernels are more robust. They aren't perfectly robust, but they're significantly, demonstrably more robust than monolithic kernels.

Likewise, you could say that OS X implements a microkernel poorly (or is it a poor microkernel?).

Yes, you could, although it is a bastard microkernel, rather than a microkernel impostor. Linux's microkernel adaptions provide nothing in the way of improved stability. OS-X is fundamentally stable, but has been altered in a way that reduces its stability. There's a difference.

is there another example of a microkernel based system that is much better? Wasn't NT4 microkernel based? Did that help it?

QNX.

And, yes, NT4 was so much more stable than any other, monolithic kernel Windows, I'm surprised that you brought that up. Companies used, and still use, NT far beyond its "life cycle" because of how fucking stable it was. It was so much more stable, that Microsoft integrated huge chunks of NT into its later OSes... and they got more stable because of it.

What makes you think that a microkernel would necessarily fair much better in this case? In my experience, SCSI problems are as much hardware as software.

Ok, let's say that SCSI problems are hardware related, for the sake of the argument. Let's say that the hardware is just flaky; the code on the SCSI chip is buggy, or something.

In a monolithic kernel, a SCSI hardware problem will cause the SCSI module to crap out, which will screw up all sorts of other things in the kernel. One drive dies, and all of a sudden -- because the SCSI module is borked -- you can't get to any of your SCSI devices. Your 'ls' hangs the terminal, and can't even be killed with a SIGHUP. In the worst case -- often the common case, in Linux -- your entire computer is dead in the water. In Linux, you reboot.

Let's assume the micro-kernel SCSI module is equally unable to gracefully handle bad situations like hardware failures. In a micro-kernel OS, if the kernel notices that the module is borked it just restarts the module, because it hasn't been corrupted by whatever screwed up the SCSI module. Or, if it doesn't, for some reason, you can tell it to.

The question remains, what is the value in running a system with one of its basic subsystems (disk I/O in the above example) crashed?

There is no value in running a system with a crashed subsystem, which is why, with a micro-kernel, you simply restart the subsystem and continue. Because mono-kernels don't segregate subsystems, if a subsystem crashes, you never know what else has been affected, and you reboot.

If you do a little research, you'll notice that monolithic kernel advocates never argue that monolithic kernels are as stable as or more stable than micro kernels. The argument is always that monolithic kernels are faster, not more stable. And that's true. It is equally true that microkernels are more robust than monolithic kernels. What Microkernel advocates argue is that the speed difference isn't as b

Go check http://www.qnx.com/ to see a true microkernelarchitecture. Very nice, very fast RTOS.

Furthermore, Richard Stallman still seems to take the microkernel seriously. Although Mach has fallen out of favor, the answer isn't to claim that Mach is all there is or could be to a microkernel and declare the concept an abject failure. RMS is moving on to greener pastures with L4. A bad implementation doesn't mean something is a bad idea.

The sheduler, for example, is real time only so for non-real time applications is questionable at best. A simple problem to address in the open source world but, apparently "not a high priority" for the manufacturer of this fine technology.

The problem with QNX is its marketing. There's no good entry level option. A development seat costs several thousand dollars, and there's not even an posted price for it. You have to talk to sales reps. Once you've bought a development seat, you can crank out vast numbers of embedded devices using QNX, so it's priced for the volume manufacturer. So few people learn QNX unless they have to.

Most Linux and QNX command line stuff has been ported to QNX. GCC is the main compiler. All the GNU command line tools work. You can even run Samba and Apache. X-Windows can be run, although it's not QNX's native graphics system. So conversion isn't that hard.

For a brief period of two years, QNX tried to open up and get more users. There was an "Open QNX" effort, a "No Charge" version for noncommercial use, and a reasonable level of interest from the open source community. Mozilla was ported to QNX. Then, with no real announcement, QNX cut that program off. You can still get a 30 day free evaluation version, but you have to ask. After 30 days it will still run, but the development environment (Eclipse) turns off.

Then QNX was acquired by Harmon International, the parent company of a range of audio companies. This resulted in 1) a focus on QNX for automotive "infotainment", and 2) abandonment of QNX by Harmon's competitors. Embedded users couldn't figure out where QNX was going, and many gave up. Lately, QNX seems to be making a modest comeback, but it's hard to tell.

To be fair, QNX has survived in a tough market. They've been selling a proprietary OS that can run on x86 desktops for two decades, against Microsoft. Nobody else has stayed in business doing that. Microsoft attempted to acquire QNX at one point, to use it as "Windows CE", but was turned down.

So it's a good technology, but the company drives you nuts.

I have yet to see a "proper" non-academic microkernel which lets one part fail while the rest remain.

QNX, but it isn't open source.

VxWorks and a few other would also fit.

QNX

http://www.qnx.com/developers/articles/article_920 _1.html

Have a look at their Package Filesystem.

You never have to install a think. Any changes to the package tree get saved in a special overspill area; and it all gets cleaned up if you kill off the package.

It's really quite neat.

Hum, finally a good comment on this. You know, you have quite a few alternatives in x86/amd64 operating systems, even from the Great Satan itself!

There's more to embedded operating systems than just Linux. QNX and BSD come to mind first. There are actually very many including Symbian, Virtuoso, VxWorks, Tron and dozens of others. Even MS tries to make one, called WinCE, though unlike the others it's only used only rarely and even then only for the humor value.

I used the Watcom C compiler back in the early- to mid-nineties on QNX, a real-time Unix-like OS. I assume QNX used it as their OS compiler because it produced fast code, but I'm not sure how it compares to Intel's compiler, or even gcc. Anyway, generated code performance would be an obvious reason to choose one compiler over another. Standards compliance might be another.

With this approach, there's no need to put drivers needed at boot time in the kernel. (Drivers are user programs under QNX.) The kernel doesn't need to know about disks. If you want a GUI during boot, you can have it. For embedded systems, the entire "OS file system" can be put in ROM, eliminating any need for a disk. For desktop x86 systems, there's a standard bootable "OS file system" which has all the usual disk and display drivers, the bus enumerators and plug-and-play handler, and the rest of the stuff needed to start an x86 PC. But all that startup stuff isn't in the kernel.

This is especially useful when your target is something that doesn't have a keyboard and screen. That's why QNX does this. Doing it this way cleans much startup-only junk out of the kernel.

The Minix 3 people, unfortunately, didn't get this, so their "microkernel" has more stuff in it than it really needs.

Also, read this excerpt of the alt.comp.lang.learn.c-c++ FAQ:

6: Why do many experts not think very highly of Herbert Schildt's books?

A good answer to this question could fill a book by itself. While no book is perfect, Schildt's books, in the opinion of many gurus, seem to positively aim to mislead learners and encourage bad habits. Schildt's beautifully clear writing style only makes things worse by causing many "satisfied" learners to recommend his books to other learners.

Do take a look at the following scathing articles before deciding to buy a Schildt text.

http://www.lysator.liu.se/c/schildt.html
http://herd.plethora.net/~seebs/c/c_tcr.html

The above reviews are admittedly based on two of Schildt's older books. However, the language they describe has not changed in the intervening period, and several books written at around the same time remain highly regarded.

The following humorous post also illustrates the general feeling towards Schildt and his books.

http://www.qnx.com/~glen/deadbeef/2764.html

There is exactly one and ONLY one C book bearing Schildt's name on its cover that is at all recommended by many C experts - see Q 25.