Ask Kevin Lawton About Plex86

Kevin Lawton, currently employed by MandrakeSoft, is the creator and driving force behind Plex86, the Open Source X86 virtualization software which runs under Linux and now boots several operating systems, including QNX and Windows 98. This is a tricky endeavor, because (among other things), as the plex86 site states: "The x86 processor is not 'naturally' virtualizable. That is to say, it was not designed to run multiple operating systems concurrently." But with enough feints and jabs, Kevin and company have cajoled it into doing so anyhow. He's agreed to answer your questions about virtualization (and / or emulation -- he is the guy behind the also-Open Source bochs project to emulate X86 processors), so please post your stumpers below. Make sure to check out the Plex86 website first, and perhaps read other things about Plex86 on Slashdot. (And "What's the point?" is not a stumper.)We'll forward the top questions on to Kevin, and hear back from him soon.

Comments

Dynamic Code Generation for more speed

[Sludge]

What do you think about the idea of dynamically generating native code in places where emulation is necessary instead of repetitively doing a loop that follows the below sequence:

1. Can the code already be run natively?
2. No, the code does some hardware specific tricks. Turn the emulation on.
3. Pass code and data off to proper kernel abstractions in the current OS.
4. Repeat for next opcode

Re:Plex86, Bochs, Platform Independence

[GrenDel Fuego]

I think he's referring to the fact that plex86 requires a linux kernel module to be loaded before it will work.

FreeBSD on x86 cannot load this module, and so it will not work.

Re:Are you endangering commercial software on Linu

[mfterman]

I tend to break software development into two areas, computer science and software engineering. Voice recognition, code morphing, cracking mathematical problems and so forth are computer science. Writing a word processor or a monolithic operating system or even a virtualizer is more software engineering. It's not like new ground is being broken in writing an x86 virtualizer, the art is documented already. As the previous post commented, MATLAB is based on well known algorithms.

Open source projects tend to be better at software engineering projects rather than computer science projects. The GNU project did very well creating the GNU tools because that was software engineering. Building a serious operating system based on a microkernel is closer to comptuter science and so got bogged down. Monolithic kernels are software engineering and so Linux took off. Even Ogg Vorbis is really just taking the well studied field of compression and the focus is on sidestepping patents than trying for something oh wow and state of the art.

To my mind, this is where the difference between commercial and open source software should lie. The commercial companies should be pushing the state of the art and working on all this gee wow stuff that people will be willing to make money on. The Open Source people should see this and then promptly reverse engineer it as the state of the art trickles down to the level that it becomes software engineering and the commercial people should be plowing on ahead and finding better things to focus their money on.

Apple is trying to do it with OS X and their rewriting of the BSD kernel. They've done some fascinating things there with Quartz and display PDF that I really hope the GNOME/KDE folks are looking at to implement in their own systems, not to mention the GNU/Linux crowd with the innovations on folder bundling and moving all system resources into XML files. Apple is steadily innovating in their products. Microsoft's .NET has a few interesting ideas and whatever good pieces are in it hopefully will be appropriated by Open Source people to be put into GNU/Linux.

Personally, I want to have a personal computer based on GPL'd software that has all of the functionality of a mainframe down to the virtual machines and scalable multiprocessor support, with all of the newer innovations that have cropped up like XML resource files, Display PDF for graphics, CORBA for handling widgets and so on. All of that is really more software engineering (yes, expensive in time and expertise software engineering but still that). The media players on the system should all be dealing with well defined media formats that anyone can write a free encoder or decoder to in GPL'd software.

All the money I should be spending on the system should be for things like voice recognition and things that I know are cutting edge and state of the art and require money to finance. My biggest beef with Microsoft is that they gouge out huge amounts of money and frankly there are very few technical innovations that they have spearheaded themselves. At least Apple goes and raises the standard for what one should expect out of a personal computer.

In the ideal world, the universities develop new ideas, the commercial companies take these ideas and turn them into money-making projects and after the newness of the ideas has worn off and their genuine value has sunk in, the open source people move in and incorporate it into widely available open source software, forcing the commercial companies to move on to something new. This is rough on the commercial software companies but no one said that life was going to be or should be easy on them.

Trying to keep on topic with the discussion, even if I am not asking a question, virtualization is a solved problem and goes back decades. Now that personal computers have reached the right level of performance to handle such things (as with other features of mainframes that trickle down when PCs get fast enough) it is only natural to start writing Open Source projects to implement it. VMWare folks need to find somewhere else to move onto, or to simply conceed to plex86 and start working on selling support of the project. They could look to see if they could combine something like VMWare with Transmeta's code morphing, perhaps even to virtualizing different processors on one machine. There are directions they can still go, there is still room for improvement. Or they can compete the way the rivals to XFree86 compete, with a wider range of driver support. In short, I feel little sympathy for VMWare. If they can't keep ahead of plex86, then they're doing something wrong.

Re:Plex86 vs. VMWare

[Bruce Perens]

If a commercial vendor can't do better than the free project, the commercial vendor should pack up their tents and find something else to do. Free enterprise includes the concept of competition, you know :-) Nobody guarantees you a living.

Thanks

Bruce

"Host-less" VM software?

[IGnatius T Foobar]

IBM's virtual environment (VM-ESA) works without the aid of a "host operating system." In other words, if you want to load (for example) Linux/390 into one of your VM's, you don't have to have the VM running inside OS/390 (the operating system formerly known as MVS). The whole mainframe is virtualized by VM-ESA before you start getting into operating systems at all.

What would be involved in making something like this run on the x86 environment? Could, for example, a thin VM management layer run on the bottom, with Linux and Windows running in parallel, instead of one on top of the other as both plex86 and VMware do?

Or is the PC so badly designed that there's no hope of doing VM's without a host OS?
--

BOCHS + WINE in MacOS X?

[VValdo]

An interesting idea came up in this Slashdot thread this week about the possibility of using BOCH's x86 emulation with WINE's Windows-like libraries to create Windows-binary compatability on the upcoming MacOS X.

Would this approach work? If nothing else, could BOCHS used in OS X to install/run Windows from the new Apple OS a la VirtualPC?
-------------------

Timetable?

[einstein]

Hi Kevin,
I've been lurking on the development mailing list for quite a few months now, and I'm wondering where do you see Plex86 going in the future? virtualization of as many OSes as possible? graphics virtualization? or some other tangent that hasn't been discussed yet?
---

What makes a processor virtualization-friendly?

[Christopher Thomas]

What characteristics make a processor difficult to virtualize? What characteristics make it easy? I have more than a passing interest in this, as I'm currently a graduate student studying IC design.

The more detailed the answer, the better.

Re:What makes a processor virtualization-friendly?

[jms]

On a processor, you have facilities like general registers, that the emulated operating system is supposed to use, and that will be saved and restored by the control program when you switch contexts.

You also have certain special registers and facilities, like control registers, and segment registers. The actual contents of these registers are going to always remain under the control of the supervisor. Part of the job of a virtualization supervisor is to emulate the function of these registers, and also to reflect their virtual state to the virtualized operating system when requested.

So basically, in order for the architecture to be virtualizable, all of the instructions that expose the presence of the emulator must be privileged.

This includes things like control registers, status words, I/O instructions, etc. If you're going to "fake out" an operating system into believing that it is running on bare iron, you need to be able to generate an exception whenever the operating system attempts to execute an instruction that exposes the fact that the actual state of the processor is different from the state of the virtual machine.

For instance, one of the mistakes in the IBM System/370 architecture was that the instruction for accessing the hardware time-of-day clock (STCK -- STore ClocK) was a non-privileged instruction. As a consequence, it was impossible to set a different time of day for a virtual machine -- the supervisor couldn't trap the instruction.

Some architectures (I believe that this is one of the X86 sticking points) require privileged state in order to modify certain control registers, but provide non-privileged instructions for accessing the contents of those registers.

For non-virtualizing operating systems, this makes sense. By making the instructions to examine the system state non-privileged, you both simplify the job of writing the operating system (because you don't have to emulate those instructions), and you also receive a performance boost (because executing those instructions don't result in exceptions). The tradeoff comes when you try and implement a virtual machine environment, and find that you can't, because even in protected mode, a program can find out "too much" about the actual state of the processor.

Where would you like to go tomorrow?

[ClayJar]

Some apps will "never" be able to run in Wine (Windows lets you do things that Linux knows are *bad* ideas). With this in mind, are there any applications that you would really like to see running in Windows (or whatever) on Plex86?

Everyone mentions running Microsoft Office when it comes to Wine; do you have any "milestone events" in mind for Plex86 (as the list of booting OS's continues to climb)?

Resources ?

[f5426]

Do you beleive that large scale virtualisation could be possible, like emulating 100 linux hosts on a big SMP box, getting the kind of virtualisation that the Mainframe guys enjoys ?

Cheers,

--fred

(And yes, we could have a beowulf on a single host)

What controls the direction Plex86

[CyberKnet]

I understand that you cant control open source developers, and tell them "You must work on xyz aspect of the code", but I was wondering this one question: Is the driving force to get a perfect Virtual CPU, is it to boot OS XYZ at any given time, or is it to boot as many OS as possible?

CK

---

Issues with Debian

[David Greene]

Recently there has been much discussion on debian-devel about where plex86 (or parts of it) can be placed in the archive. The main issue is the licensing of the VGA BIOS from Elpin , which states that the BIOS is free for use in plex86, but may not be modified or used for any other purpose. Because of the dependency of plex86 on this BIOS, it my be forced into contrib instead of main. The BIOS itself would have to go into non-free, which raises questions about whether this would be a legal distribution of the BIOS, since it would not be packaged with plex86. Moreover, with the rumblings of eliminating non-free, it may not be distributed through Debian at all, regardless of legal issues.

Are there any plans in the near future to replace the Elpin BIOS with a Free implementation?

--

Native parition support

[David Greene]

What are the plans for native (raw) partition support in plex86? Bochs has had this ability for some time and I find it to be essential in VMware.

The need for large partitions in a Windows environment coupled with the file size limits in Linux and the more restrictive Windows licensing means this sort of support is critical to anyone wanting to run Windows on Linux with plex86.

--

How could a virtualized OS be made more efficient?

[Deven]

If the authors of an OS want to cooperate with Plex86, so it runs more efficiently when virtualized, what would you recommend? What (expensive) operations would be best avoided? Will Plex86 have a API available for a faster alternative to emulation?

Plex86, Bochs, Platform Independence

[zanzar]

I'm an avid FreeBSD enthusiast, and I'm disappointed to see that Plex86 doesn't look like it will run under FreeBSD any time soon. Bochs, on the other hand, works fine under FreeBSD, but lacks many essential features that Plex86 appears to have. I was wondering what you will be doing (if anything) to ensure the platform independence of Plex86. Will you be leaving Bochs as the multi-platform solution? Will the Bochs be abandoned in favor of Plex86?

How can the rest of us help?

[Anonymous Colin]

Hi Kevin,

What are the greatest needs in the Plex86 project? How can the /. cast of thousands help out?

(Ok Kevin, here's your chance - you'll never get a better one to recruit new helpers!:-)

DirectX support

[sheckard]

I know this is a way off, but do you see any reason why DirectX support would not be possible? Or would it just require too much manpower that it wouldn't be feasable?

Plex86 vs. VMWare

[Chester K]

A topic brought up on Slashdot a couple days ago had some interesting discussion that if Free solutions like Plex86 took off, it would destroy VMWare's business model, and show other businesses that you can't make money developing software for Linux because someone will undercut you with a Free solution. How do you respond to these fears?

Development Priorities

[jzoetewey]

While the ideal is that Plex86 will support all operating systems, I assume that you've made choices as to which ones you're going to work on first. Your site mentioned Dos, NT, Win95, Linux and QNX as OSes that PLex86 currently runs. As someone who tends to use FreeBSD when I use a free OS, I'm curious as to where the BSDs rate in your priorities and what about BeOS, Plan 9, etc...