Linux Gains Native RTOS Emulation Layer

nerdyH writes to tell us that the Xenomai/SOLO project is attempting to deliver VxWorks and other RTOS emulation for any Linux kernel. "Some weeks ago, I started laying the groundwork for porting the Xenomai emulators natively over the PREEMPT_RT kernel. Unlike the co-kernel based Xenomai version, SOLO does not require any kernel support from additional modules or patches. It is fully based on the standard POSIX library, and runs as a regular process controlled by a single image Linux kernel. As a first step, a VxWorks emulator has just been rebuilt over this new framework."

Comments

Realtime, VxWorks, Dolla Dolla Bill Yall

[Epistax]

The benchmarks that are really expected by real time in my area are things like consistency. For example if we set a task/thread to execute every 125 milliseconds, the closer it hits the mark the better. Time lag in either direction puts that OS in the "No" category. Another important asset of an RTOS is well defined task preemption: No task gets preempted by one with worse priority. Time slicing might be enabled so that a task gets preempted by one of the same priority, and better priority tasks always preempt if they are ready to go. Also if a high priority task is waiting on a resource owned by a low priority task, that low priority task gets an elevated priority equal to the high priority task. As a last ditch effort to provide mutual exclusion / data protection, threads/tasks need to be able to disable system interrupts. Remember kids, in the RTOS world one task can take down the whole system.

VxWorks is the only OS I've played with so far that allows this, but I'm VERY curious to see what people can inject into the Linux kernel. VxWorks is.. shall we say... NOT CHEAP. And inter-version migration is a pain... and god help you if you aren't using off the shelf hardware...

Re:Realtime, VxWorks, Dolla Dolla Bill Yall

[JohnFluxx]

> Another important asset of an RTOS is well defined task preemption: No task gets preempted by one with worse priority. Time slicing might be enabled so that a task gets preempted by one of the same priority, and better priority tasks always preempt if they are ready to go.

Any normal distro Linux kernel can do this particular part. Just set the scheduler to round robin. (You can do this in KDE4 btw. Press ctrl-esc to bring up the task manager, right click a process, change priority, and chose round robin.)

Re:Realtime, VxWorks, Dolla Dolla Bill Yall

[pla]

VxWorks is.. shall we say... NOT CHEAP.

Have you ever actually had to code for Vx?

You couldn't pay me enough (literally - I've turned down jobs that wanted me to work with it... I should probably take it off my resume) to deal with that POS (by which I don't mean "Point Of Sale") on a regular basis.

Actually, in fairness, as an OS, it doesn't suck too much. But the build tools... Let's just say WindRiver clearly subscribes to the "firmware should hurt" coding paradigm. The IDE made OutLook look stable and friendly, the command line build tools simply didn't work (literally - WR couldn't even have tested them, because they failed phenomenally even on a clean install and a "hello world" module), the revision control had no objection to overwriting parent revisions without forcing a new fork... Ugh. I'll probably have nightmares tonight just from thinking about it.

Oddly enough, it surprises me to see it still talked about. When I suffered with it nearly a decade ago, it looked like a near-certainty that Linux would tap that last nail in its coffin. How ironic, that Linux should now give it new life via emulation.

Re:Realtime, VxWorks, Dolla Dolla Bill Yall

[Not Invented Here]

The problem with VxWorks is that the scheduling accuracy is its only positive feature. Device drivers are rare and expensive, probably because it doesn't have a sensible hardware abstraction layer like Linux, Windows (NT family), and QNX. If you want to use the MMU on the processor you get charged again, and you get even more incompatibilities. The documentation doesn't go into much depth, and Google isn't very helpful, as hardly anybody talks about it on the Internet.

Have you tried QNX or RTEMS? I don't have any data on their scheduling accuracy, but they claim to support the same real-time features. I've also found the QNX documentation much easier to follow, and I managed to turn out a BSP and a custom device driver within a week of first receiving the software.

Re:Realtime, VxWorks, Dolla Dolla Bill Yall

[faragon]

VxWorks is the only OS I've played with so far that allows this (...)

May be you need to play more :-) . LynxOS is a full-fledged POSIX RTOS, with memory protection, far better than VxWorks if disk and network is involved (I was developing toll systems real time applications for 4 years). I hope someday Linux reach RTOS habilities without having to use "cokernels", as although I'm pretty happy with current kernel preemptiveness, I would love to get true RTOS Linux (echo 1 > /proc/rtos).

Re:Realtime, VxWorks, Dolla Dolla Bill Yall

[Epistax]

Oh yes I've coded for it. I'm on my second BSP and my third kernel. It's not as bad as it used to be in some respects. The Eclipse is the IDE now. The documentation is still horrendous though. WindRiver is working hard on their own RTOS Linux solution. I don't know if they are planning on eventually digging a hole and throwing VxWorks in it, or just trying to combat the free RTOS Linux market with something they can sell to managers.

Re:Realtime, VxWorks, Dolla Dolla Bill Yall

[K. S. Kyosuke]

VxWorks is the only OS I've played with so far that allows this

and god help you if you aren't using off the shelf hardware

Well, and what does happen if you do use off-the-shelf hardware? I am not exactly "well educated" in this area, but I am not quite sure whether the current typical OTS hardware is suitable for RTOS at all. Well, maybe some soft real time apps may run just fine...but I once spoke after-hours with Vojtech Pavlik (of the linux-joystick-drivers-and-other-input-devices fame) and he introduced me to the wonders of the System Management Mode, a wonderful creation of Intel capable of destroying any precise timing in ordinary PC hardware that just cannot be overridden by any normal means. Does industrial hardware that does not exhibit such features and is designed for RTOS and time-sensitive applications still qualify for the designation "off-the-shelf"?

Re:Realtime, VxWorks, Dolla Dolla Bill Yall

[Epistax]

I haven't had any experience with other systems yet really. VxWorks did add an abstraction layer for hardware drivers called "VxBus". So far the only advantage I've found is that.. Okay well I haven't found one but I did eventually get it to work. The advantage is they didn't break everything when they introduced it.

I don't understand why WindRiver hates documentation to much. They actually have a policy of not sending hard copies of their manuals anymore. Their man pages are kind of half-assed. I usually have to go straight to the kernel source tree to figure out what a function really does. I really must still state, it's a real pain to use, but their stuff DOES work. Once you get it set up right, I've found 99.99% of what goes wrong is application code's fault.

I've learned that if it takes more than a day to figure out how a specific something works, I pass it along as a service request to them. It's not my fault if the didn't document well enough!

Re:Realtime, VxWorks, Dolla Dolla Bill Yall

[Epistax]

Well, you can change to see if there is processor time being stolen from the OS by toggling a pin on a fast cycle and watching it with an oscilloscope, or perhaps polling the internal pentium timer. Someone before me had checked with our specific setup to see if this was happening (specifically we were concerned if USB hookups caused the BIOS to inject work) but nothing was detected. We use custom PC boards that could have had it disabled at the BIOS level, but I'm just not sure,

Off-the-shelf is where BSP's and drivers come in. The BSP (board support package) needs to be configured for the specific board. WindRiver provides a kajillion default packages, and if you use a off-the-shelf-board, the BSP and driver set should require little or no modification at all so you can just go straight to customizing your OS. The more customized your board is, the more you might have to do, such as writing VME, clock, PCI, or Ethernet drivers, do custom memory management, etc. I guess it's not fair to peg this complication on WindRiver. I haven't tried a Linux kernel on a custom board, but just as much configuring must go on at some point.

Re:Realtime, VxWorks, Dolla Dolla Bill Yall

[joelsherrill]

RTEMS is a free, open source alternative to VxWorks that provides POSIX threads in addition to an API we call the "Classic API" that was based upon an old dead proposed VITA standard from pSOS+ folks. RTEMS is single process, multi-threaded in the POSIX sense and has almost every POSIX 1003.1b features that is possible without being able to exec a new process. We use GNU tools and maintain an APT/Yum repository so GNU/Linux users have an easy situation with the tools. Check out some of the places it has been used at http://www.rtems.org/flyers.html and in the Wiki Applications page. You can get predictable tasking and algorithmic behavior while still staying in the free software world.

Re:Realtime, VxWorks, Dolla Dolla Bill Yall

[Esben]

Linux with PREEMPT_RT is as predictable (but not as fast) as VxWorks in the mentioned categories, you just have to make sure that all the RT tasks have SCHED_FIFO or SCHED_RR to behave correctly. VxWorks doesn't have (or at least didn't in version 5.4-5.5 which I used to code for) SCHED_OTHER. And that is a huge drawback for VxWorks because very often you have a lot of application threads running with no hard deadlines and might fluctuate a lot in how much CPU they use (webserver and other user interfaces). In VxWorks you have to give all such tasks priority 255 and use time slicing. Otherwise one task suddenly taking a lot of CPU will starve all tasks with lower priority until it is done.

Another problem I have seen in VxWorks is the priority inheritance: It doesn't work correctly for nested locks: A low priority task takes a lock A, runs it's own code for and then it calls into say a driver and takes lock B. Now some high-priority task wants to access the same driver and tries to lock B. The first tasks is then boosted in priority and soon finishes the driver call and unlocks B. But it is not unboosted before it release A. That is a huge problem in a RTOS: Suddenly the timing of the high priority task depends on how long a low priority tasks keeps the completely irrelevant lock A. You don't get the decoupling, that the timing of a high priority tasks only depends on the tasks of equal or higher priority, which is so important in a RTOS.

Basicly, I think core of Linux with PREEMPT_RT is a better (more deterministic) RTOS than VxWorks, but slower (longer maximum latencies for especially interrupts); but all subsystems around the scheduler etc. aren't coded for RT use - i.e. it has non-deterministic code-paths and might even make blocking calls. So an RT application on Linux/PREEMPT_RT can basicly only use basic highres timers, simple serial ports and memory mapped devices. Use of the netstack, disk IO is a no-go. But then again, the same applies to VxWorks...

Re:Realtime, VxWorks, Dolla Dolla Bill Yall

[spaanoft]

QNX used to be intel-only, but has been ported over to other chipsets. I know I've used it on an arm and some ppc before.

Re:Realtime, VxWorks, Dolla Dolla Bill Yall

[PCM2]

I have asked Wind River representatives whether their Linux product is meant to replace VxWorks and they emphatically said no. (If that means anything.)

Re:A quick search reveals

[Anonymous Coward]

don't click that link, mod parent down.

Re:A quick search reveals

[Tony Hoyle]

FFS don't click that link - it's some sort of browser spawner/malware/virus.

I had to power cycle my machine to shut it down as it managed to completely saturate the machine.
As far as I can tell it:

1. Tried to log me onto a gay porn site
2. Tried to open up IRC and do something (failed, luckily, since osx won't let such things happen automatically.. my screen just filled boxes asking if I wanted to start colloquy)
3. Tried to run a .exe - luckily on my mac that did nothing.. that's the virus payload I guess

I reckon if you clicked that button on a windows machine you'd be crying right now - and your passwords would be all over IRC too...

Re:A quick search reveals

[Teun]

No, it's Lord Haw Haw's Yahoo link that's attacking windows machines.
But then, who in is right mind would admit on /. to running such a thing :)

Re:A quick search reveals

[Anonymous Coward]

What's worse, the link (that Slashdot shows as a Yahoo link) is actually an only slightly obfuscated Nimp link.

Aside, I find it amazing that a 4 letter TLD is allowed to be used this way as long as it has. Nimp isn't just a shock site, it's got to break enough criminal laws to put it's owners and people that link to it in jail.

Re:POS needs realtime? hahahahhaha

[richpm]

"Why not just get a faster computer," Power consumption? Heat? Noise levels? Size? "... because if they had a clue, they would leverage Eclipse ..." I guess they've got a clue then, they've been shipping an Eclipse based IDE for 2+ years now and Wind River are /very/ active in the development of CDT. In fact the CDT project lead works for them now. Admittedly they do some stuff in a slightly non-standard way (e.g. not the way vanilla CDT does it) but that's with good reason and allows them a far more flexible build system than the standard CDT project model allows.

Re:A quick search reveals

[ozmanjusri]

Thanks for the spam sunshine.

How 'bout you reformat and reinstall so the rest of us don't pay for your "everything appears fine." system?

Re:POS needs realtime? hahahahhaha

[billcopc]

I don't think you should be speaking against RTOS without any actual RTOS experience. It just makes everyone else assume you're an ass.

These things typically run on embedded devices, not a friggin' Dell midtower. They do one job and they do it with exacting accuracy, on minified motherboards and fanless CPUs, hooked up to custom-built controllers and monitoring equipment.

RTOS tasks are typically things we used to do in solid state with simple feedback logic, but the RTOS allows it to be done in software at a lower cost, plus allowing easy updates or adjustments without a complete redesign.

Oh dear god no!

[malevolentjelly]

Why would you ever need to emulate an RTOS in linux? Linux does not do what ThreadX, VxWorks, INTEGRITY, uVelocity, or any of the sort do- imagine you're an embedded device manufacturer and suddenly you need to bump your device up from 64k chip of ram to 8mb. This is completely retarded. In the embedded world, true RTOS are used for things that can never fail, lag, or be insecure in any way. Linux is generally used to fill in a cheap userland. Like on Sony TV's, the RTOS junk is handled by uVelocity, but they use linux for the OSD, etc. You DO NOT PUT LINUX in places you put VxWorks. Imagine having that ludicrous monolithic server OS in a PACE MAKER. Or a NUCLEAR MISSILE.

I could see people wanting to hypervise linux in a secure RTOS but emulate an RTOS in linux? Please tell me this is for development purposes... further still- are they completely insane?

Re:Oh dear god no!

[malevolentjelly]

Linux is not realtime. "Realtime linux" is a non-linux realtime OS with a non-realtime linux compatibility layer... emulating a REAL RTOS?... Why would you emulate a realtime system in a non-realtime system? Doesn't that defeat the purpose of a realtime OS?

Do you have any idea why people use realtime systems?

With VxWorks you pay license fees... FOR A REALTIME SYSTEM. You're also able to do a lot more with a lot less resources. In the embedded world, less is more! You save millions cutting kb's (or mb's) of ram out of hardware. And this emulation layer saves what, some programmer a week or two tops porting one POSIX-compliant VxWorks application to a semi-POSIX compliant linux device. The licensing fees don't seem so bad when you think about the extra hardware necessary to use a make-believe-realtime OS like "realtime" linux.

I've got a better idea- use a real, tried, trusted RTOS and simply use an emulated UNIX layer if you need that sort of support (most decent RTOS support this). This pairing of linux and Xenomai's RTOS just sounds awkward. Software costs are just miniscule compared with the cost of making a bulky device with hardware that outclasses its functionality.

Re:POS needs realtime? hahahahhaha

[Anonymous Coward]

A RTOS isn't about speed, it is about predictability. It is about guaranteed response.

There are some things that a 200MHz machine with the right software can do quicker and more accurately than a 4GHz machine with a standard OS.

What it comes down to is they built an expensive mansion, but built it on sand. Doesn't matter how expensive or big the house is, if it is built on a poor foundation, the house will be unstable.

I have seen process control applications where an 8MHz 8086 was fine. No OS, just application coded in ROM. Fast enough, stable. When newer, faster processors came available, the bosses decided, "We have power to burn, let give the application a GUI." So now they run with dual processor motherboards. But... motherboards today are not as fully debugged as the old days. Some companies never kills the bugs, they just release a new Motherboard with a new (buggy) chipset. Same for the OS. A well written application becomes unstable.

If you drop a scratched DVD in you DVD drive, does your windows machine kind of hang while windows tried to decide what is there? I've experienced that quite a few times.

BTW, this is similar to Linus' exchange with Tannenbaum on monolithic v.s. microkernel.

Re:A quick search reveals

[neumayr]

After reading the comments further down the thread (lol) I take back the part about the generally harmless.
But there is some educational use for those links. Keep them coming, I say.

Re:A quick search reveals

[krbvroc1]

I'm running NoScript Firefox extention under Linux and while my retina hurts, there was no moving screens or whatnot. By default, I do not trust javascript/activex. Get NoScript and configure it!

http://noscript.net/

Re:A quick search reveals

[Teun]

Hehe, I see it the other way around.
I don't like having to fend off thousands of malwares with an OS that implemented networking as an afterthought.