Slashdot Mirror


Docker Moves Beyond Containers With Unikernel Systems Purchase (thenewstack.io)

joabj writes: Earlier today, Docker announced that it had purchased the Cambridge, U.K.-based Unikernel Systems, makers of the OCaml-based MirageOS, a unikernel or "virtual library-based operating system." Unikernels go beyond containers in stripping virtualization down to the bare essentials in that they only include the specific OS functionality that the application actually needs. Their design builds on decades of research into modular OS design. Although unikernels can be complex to deploy for developers, Docker aims to make the process as standardized as possible, for easier deployment.

69 comments

  1. Buzzword bingo! by Anonymous Coward · · Score: 0

    Buzzword company buys other buzzword company! Woohoo!!

    1. Re:Buzzword bingo! by Anonymous Coward · · Score: 0

      I made bingo with just this one article!

  2. Complicated by 110010001000 · · Score: 0

    Could we make things a bit more complicated? Virtualization I understand. I don't understand the need for Docker. It is just another layer that will break. Maybe I am just getting old, but this seems very complicated and prone to breakage with all the layers.

    1. Re:Complicated by Anonymous Coward · · Score: 0

      Docker actually makes things simpler!
      Since you understand virtualization, then think of docker as lightweight virtualization. The main differences to virtual machines are - docker shares host kernel and can't boot different OS, virtual machines do. Docker runs at bare metal speed while virtual machines don't (they can come close with hvm technique).
      Docker container almost starts immediately while virtual machines need to boot which takes longer time. There are ton of resources on web on this subject if you google! But the bottom line is that docker or containerization is the future.
      The way things went - unix servers -> cgi -> java EE -> virtual machines -> docker/container/lxc -> MirageOS/Ubuntu Snappy Core
      Things are certainly getting interesting and not complicated :-)

    2. Re:Complicated by jiriw · · Score: 5, Insightful

      Virtualization is 'expensive', as each virtual server running on the host operating system has it's own operating system, each running their own kernel, having their own generic support libraries, doing its own memory management, hardware access and interrupt management (to/on virtual devices emulated by the host operating system), etc.

      Chroot is 'inexpensive', but it only offers a thin veneer of file system separation.

      Docker lies somewhere in between. It has its self contained file system with all the generic support libraries (user land) needed for the application, but hardware resources are managed by the single kernel of the host operating system. This does give rise to a restriction not present for true virtual machines; all Docker containers on one host system must use the same kernel (interfaces): those of the kernel of the host system. Actually, the kernel has some special modified interfaces to make certain the applications in the docker container can't access (data or processes of) other docker containers (unless permitted) or the host operating system, and for those applications it still 'feels' like they are running their own copy of the operating system. But, for example, all processes running in all Docker containers on one host system are part of the processes list of the host kernel, there is only one memory manager; that of the host system, etc.

      Now there is that newfangled Unikernel kid... What I understand of it is that, in comparison with a Docker container, the support libraries / userland is stripped bare so only the symbols/functions remain that are actually in use by the applications that run in it. But the 'kernel' bit in Unikernel would suggest also parts of kernel functionality is transferred to the container and I would suspect parts not in use by the actual applications in there would not be included. The question is, how much of the host kernel can you transfer to the containers? Certain things should be done 'at the top', if only to prevent containers from hogging critical system resources and such and still being able to do certain system diagnostics at the host os level...

      I should read more about it. It seems to be interesting stuff.

    3. Re:Complicated by Anonymous Coward · · Score: 1

      "Expensive" is a loaded word. Unused CPU cycles and memory bytes are worthless, i.e. a system running at 5% load costs the same as a system running at 10% load.

    4. Re:Complicated by Shane_Optima · · Score: 2

      The layers add abstraction, compartmentalization, portability and the ability to roll back snapshots without rebooting. If you understand virtualization then I'm sure you can understand how this can greatly increase security and robustness in general, at least in principle.

      Now, if you don't understand why someone would use an LXC-based solution like Docker instead of a fully virtualized HVM machine then you probably haven't seen them in practice. They are native speed, and you don't have to mess around with inefficient memory allocation. The performance difference is critical in many cases and even when it isn't, it's still damn nice to have everything being as smooth as normal. Unikernel is a different approach, but offers similar performance advantages over traditional HVMs.

      As I mention below, paravirtualization is also pretty damn fast and is more secure and flexible compared to LXC solutions since the kernel isn't being shared, although for that reason you are stuck with potentially inefficient RAM allocation issues.

    5. Re:Complicated by swb · · Score: 1

      It sounds almost like someone used a smart linker to link in the operating system with the application to create a standalone, appliance-like application with its own integrated kernel.

      It kind of makes some sense, if you think about the sort of general trend of "appliance" servers like FreeNAS or Pfsense that are actually customized OSs with applications on top but have functionally complete user interfaces that don't really expose the underlying operating system.

      It would be great if there was a way to automate creation of an application appliance, some way to point to an application package, scan it for kernel dependencies and produce a bootable VM or ISO that would just run the application.

    6. Re:Complicated by Anonymous Coward · · Score: 0

      I think I saw Solaris do this like 10 years ago.

    7. Re:Complicated by Anonymous Coward · · Score: 1

      It's BSD jails or Solaris containers all over again. Imagine a chroot with its own IP address and you're basically there.

      Docker also has its own system for autogenerating the content of a container - you can write something vaguely related to a Makefile saying e.g. "start with a minimal debian stable, copy these files into it, and run these commands as root inside". The result of that can be stored as an image; those images can optionally be used as the base for yet more specialized images. What you use them for is typically "start the image called X, run this command with these switches inside, and open/forward these network ports into the image". Starting one takes mere moments, which is a large part of the appeal. There's also a fairly large repository of these spec files, so you can grab a config file, skim through it if you're paranoid (as you should be), and go from there to an installed and running service with very few keypresses. (Or add your own specialization to it first.)

      There's also some appeal in having these be automatically started (on e.g. amazon EC2) when needed and stopped when you don't need them anymore, if you can boil your serving needs down to a docker image plus some suitable load balancing. My needs have never involved cloud servers, so I have never touched this part.

    8. Re:Complicated by Anonymous Coward · · Score: 0

      Cycles are expensive, if you need more than you have. Power is expensive when you're using more than you need. Cycles take power.

    9. Re:Complicated by Shane_Optima · · Score: 5, Insightful

      I don't doubt it. There should be a mandatory "Reinventing the Wheel" course for all CS majors. Chapter 1, The Amiga. Chapter 2, Lisp. Etcetera.

    10. Re:Complicated by Immerman · · Score: 1

      If your server is running at 5% load under peak conditions, then you wasted money buying a lot more server than you needed.

      Reducing overhead is relevant precisely when system demand is at it's peak - for a large operation a reduction in overhead may translate fairly directly to a similar reduction in the number of physical servers needed, with offer proportional cost savings.

      --
      --- Most topics have many sides worth arguing, allow me to take one opposite you.
    11. Re:Complicated by Anonymous Coward · · Score: 0

      Depends what your server is used for.

      I manage many domain controller/file servers that average less then 3% utilization and might peak to 50% once a month when updates are installed. These are entry level servers on top of that.

      Honestly you could probably use a 10 year old dell desktop as a DC in a lot of my customers situations. I wouldn't recommend it of course but the point is the processing load is not great.

      Sometimes the entry level server is more then plenty for your tasks.

    12. Re: Complicated by Anonymous Coward · · Score: 0

      Except that cotainerization was the past - virtuozzo/openvz/parallels started as a glorified chroot back in the late 90s, and containerization to this day remain just that - chroot with process isolation and cpu time accounting and quantization. This has been developed ages ago by Sun (zones) and IBM (lpar)

    13. Re:Complicated by 0100010001010011 · · Score: 0

      So it's a poorly implemented jail.

    14. Re:Complicated by 0100010001010011 · · Score: 1

      FreeNAS has that. It's jails. I run almost everything in its own jail on my FreeNAS box.

      I still haven't seen what Docker brings to the table other than "Hey, I don't like that wheel, look at mine".

      JID IP Address Hostname Path
                1 - armory_1 /mnt/keg/jails/armory_1
                2 - couchpotato_1 /mnt/keg/jails/couchpotato_1
                3 - gogs_1 /mnt/keg/jails/gogs_1
                4 - mysql_1 /mnt/keg/jails/mysql_1
                5 - nginx_1 /mnt/keg/jails/nginx_1
                6 - nikola_1 /mnt/keg/jails/nikola_1
                7 - notex_1 /mnt/keg/jails/notex_1
                8 - nzbget_1 /mnt/keg/jails/nzbget_1
                9 - owncloud_1 /mnt/keg/jails/owncloud_1
              10 - pgp_1 /mnt/keg/jails/pgp_1
              11 - pixelserv_1 /mnt/keg/jails/pixelserv_1
              12 - python34_1 /mnt/keg/jails/python34_1
              13 - rabbitmq_1 /mnt/keg/jails/rabbitmq_1
              14 - redis_1 /mnt/keg/jails/redis_1
              15 - rst_1 /mnt/keg/jails/rst_1
              16 - sabnzbd_2 /mnt/keg/jails/sabnzbd_2
              17 - sickrage_1 /mnt/keg/jails/sickrage_1
              18 - syncthing_2 /mnt/keg/jails/syncthing_2
              19 - transmission_1 /mnt/keg/jails/transmission_1
              20 - usenet_1 /mnt/keg/jails/usenet_1
              21 - managethis_1 /mnt/keg/jails/managethis_1
              22 - template_1 /mnt/keg/jails/template_1

    15. Re:Complicated by 0100010001010011 · · Score: 1

      So what about it couldn't have been accomplished with tcsh script and a jail or even better PC-BSD plugins?

    16. Re:Complicated by Anonymous Coward · · Score: 1, Interesting

      Docker allows you to run an executable in a container with no setup and provides resource allocation, two features which jails don't support. Like jails, docker can also use an existing container as an overlay on the base system. Docker can dynamically limit CPU utilization and memory, and it supports prioritization including disk IO. You have full control over the container including stopping execution, freezing the contents, and restarting later.

      At this point, jail is a poor man's docker, and FreeBSD suffers from people in the community (like yourself) that dismiss docker.

    17. Re:Complicated by Anonymous Coward · · Score: 0

      No, jail is a poorly implemented docker.

      You do the FreeBSD community a disservice by being ignorant and dismissive of technology from other platforms.

    18. Re:Complicated by dns_server · · Score: 4, Informative

      This is something different.
      Take the linux kernel, split it into modules such as tcp/ip stack etc.
      Now you can create a binary that includes the part of the os that you need that you can include with your binary.
      You then link the kernel and your application so you have one binary with the bits of the os you need as well as your program and nothing else.
      No init system, no other processes just the os.
      You can then run this inside xen as the target so you don't need the hardware support.

    19. Re:Complicated by greenfruitsalad · · Score: 2

      Docker actually makes things simpler!
      Since you understand virtualization, then think of docker as lightweight virtualization.

      how is adding stuff on top of LXC making things simpler? i followed docker from its inception and to me and my colleagues, they are the microsoft of os-level virtualisation.

    20. Re: Complicated by Anonymous Coward · · Score: 0

      So what virtuozzo was doing in 2003? Laughable!

    21. Re:Complicated by serviscope_minor · · Score: 1

      Actually it's not complicated it's just the cycle of reinvention and calling old concepts new things.

      If you fire up a VM to do one task, why bother to run a full OS under that VM, why not run a single program. And, well if you're running one program on the OS, why bother with a whole kernel, why not just run the program on the "bare metal".

      And, er, well it's not really bare metal. It sort of looks like bare metal except the hypervisors provide a uniform interface to pseudo-hadware on the whole so you don't have to do excessive screwing with device drivers as you move your virtualised stuff from one machine to another.

      So, in other words, a hypervisor is now a full OS running a bunch of programs which are now called "unikernels".

      Sooner or later someone will decide that the unikernel stuff are sharing a lot of memory regions and isn't that inefficient and maybe they should be able to share memory regions locked to read only by the hypervisor. And thus, .so files will be reinvented, but probably poorly.

      Someone will also decide they need shared access to named blocks of storage.

      --
      SJW n. One who posts facts.
    22. Re:Complicated by serviscope_minor · · Score: 1

      You can then run this inside xen as the target so you don't need the hardware support.

      Indeed! Of course you can just write it in something with a C API underneath and make calls to socket() and connect(). No need to link in the whole of Chrome for example, just to get an editor...

      It's just funny that the OS is being reinvented one level down and will inevitably grow back up to where it is now, but looking different.

      --
      SJW n. One who posts facts.
    23. Re:Complicated by 0100010001010011 · · Score: 1

      Which one has logged more hours in the field?

    24. Re:Complicated by Anonymous Coward · · Score: 0

      So what about it couldn't have been accomplished with tcsh script and a jail or even better PC-BSD plugins?

      Nobody's going to throw VC money at tcsh.

    25. Re:Complicated by Anonymous Coward · · Score: 0

      > "Expensive" is a loaded word. Unused CPU cycles and memory bytes are worthless, i.e. a system running at 5% load costs the same as a system running at 10% load.

      This is something I have heared many times over the years, and I still do not agree.
      When I have a system with a load-avergage of 470 ( like I had yesterday ) then these kind of statements goes out the window.

      I could, program like I'm the only user of the Server.
      Or, I could program like I'm probably sharing the Server with _way_ to _much_ software.

      Saying goes: Hardware is cheap
      Truth is: Yes it is, but managers do not buy it, cause it costs in administration in the IT department.

      So, my conclution, yes CPU cycles is expensive.

    26. Re:Complicated by Anonymous Coward · · Score: 0

      Current VMs have the horrible issue that they need all CPU cores at the same time, for any VM tech out there. Because of this, the recommendation is to not give more than 4 CPUs to a guest. What if I have a guest that needs 8, 12, 24 cores? Too bad, so sad, VMs are not recommended for this case.

    27. Re:Complicated by Anonymous Coward · · Score: 0

      Jails in FreeBSD can do everything Docker does and more. Some of the things you mentioned are not directly exposed via Jails in FreeBSD, but can be done and most have been done with some basic scripts. There are several projects trying to make Jails fully Docker compatible, and they're just collections of scripts that re-use what already exists. FreeBSD can do what ever Docket can, it just isn't exposed in as nice a way as Docker has, and FreeBSD has been able to do it 15+ years.

    28. Re:Complicated by Anonymous Coward · · Score: 1

      Internet Explorer probably has more hours in the field than both combined. How's that working out?

    29. Re:Complicated by Znork · · Score: 1

      And if docker means you're going to spend more time managing OS level instances, then any savings on hardware are often eaten within weeks of deployment. Shared systems are a PITA if there's any need to coordinate between multiple users of those systems at all.

      The fact is, I suspect the only way to avoid triggering manpower costs while implementing docker currently in a large company is basically to deploy it on a one-container-per-OS-image basis. Basically as an application packaging method. Which of course means there won't be any cost reduction on hardware at all. But then again, nobody cares about that, which should be obvious from the lack of optimization in the actual software running.

    30. Re: Complicated by TheRaven64 · · Score: 1

      FreeBSD introduced jails in 2000. Solaris gained support for Zones (jails, but rebranded) in 2004. Linux still has a load of competing solution and no sensible management interface, hence Docker exists to add them as a third party thing.

      --
      I am TheRaven on Soylent News
    31. Re:Complicated by swb · · Score: 1

      I think that's mostly what I described as "linking the executable with the operating system" although perhaps less succinct. It's what I meant to say. :-).

      The problem with this, though is the Unix nature of not having a single application + OS, but having functionality spread over many applications. This may make it more complicated to use this type of a system unless it can somehow accommodate multiple applications into a single binary.

      It goes against Unix religion, but I always kind of wished applications with heavy dependency on other applications could have just imported their dependencies into the main application versus library calls or worse, actual execution of other executables for some task.

    32. Re:Complicated by Anonymous Coward · · Score: 0

      Nothing; the underlying tech is quite similar. The management and setup tools in docker feels a bit more polished than ezJail and the like, and it's convenient to have a large selection of prebuilt stuff - but if you're running a large operation you would presumably write your own management scripts anyway and render those differences moot. The target for docker seems (to me) to be single users and small groups - people who like being able to grab MySQL-in-a-box or whatever and fire it up as easily as possible, so they can spend their limited time on something else.

      As for me, I'm testing docker on a data analysis server. We've got a fairly unusual workload: Researchers ssh in and run R and perl and random bioinformatics tools. These can be brutally RAM-heavy, and while a lot is single-threaded, other things are embarrassingly parallel. We just bought a 256GB RAM / 16-physical-core machine for them to play with; so far so good. The complication is that I'd like to sandbox them in a bit, and to have another sandbox with stricter access control (yubikey auth, whitelist of users instead of anyone in the right domain group) for more sensitive data. Both sandboxes should be able to expand up to use all cores and most of the RAM, ideally while handling overcommits in a slightly sensible way, so virtualisation doesn't seem like an ideal fit.

      Personally, I'd put them in FreeBSD jails ... but some of the tools are linux-only, and many of the scripts (and instructions) they use presume you're on ubuntu or something similar. I guess I could install Debian/kFreeBSD, since it supports jails - but just to test the waters I'm about to try manhandling Docker into providing what I want instead. It might be a horrible mismatch, but I'm just curious to see what will happen.

    33. Re:Complicated by Junta · · Score: 1

      It depends on your perspective. Understanding direct manipulation of cgroups and namespaces is sometimes the simplest thing, depending on what you are doing. Sometimes lxc is. Docker is less about running the containers as it is about building and sharing filesystem snapshots to be used in containers (of course it can manage running of containers, but in that specific respect it doesn't do anything to commend it over directly using lxc). Without dockerhub, I don't think anyone would be paying docker much mind.

      If you just want a q&d 'super chroot', then unshare is probably the most straightforward, and lxc just makes things more complex.

      --
      XML is like violence. If it doesn't solve the problem, use more.
    34. Re:Complicated by Etcetera · · Score: 1

      I don't doubt it. There should be a mandatory "Reinventing the Wheel" course for all CS majors. Chapter 1, The Amiga. Chapter 2, Lisp. Etcetera.

      I wish I had mod points to give to this.

    35. Re:Complicated by Junta · · Score: 1

      Mainly convenience and networking effects.

      The instructions to start a new instance of a docker image someone else has shared is one line: docker pull ; docker run
      The instructions to create a new stock docker image are relatively short, like a simple makefile.

      Those short instructions, the existence of dockerhub, and hype combine to have a lot of samples of images to run.

      Of course, that image infrastructure is pretty fast and loose, limited ability to verify that the image arrived intact from the uploader, and a sea of mostly anonymous uploaders mean such verification is of limited value anyway. I might pull down a docker image to help clarify documentation or when documentation is not available, but I'd produce my own rather than rely upon 'randomguy45' to correctly maintain their image.

      --
      XML is like violence. If it doesn't solve the problem, use more.
    36. Re:Complicated by Penguinisto · · Score: 1

      It's simpler... for product owners/developers. Not so much for the sysadmins who have to t-shoot the stuff, but there's benefits on the ops side too.

      I should explain. It has benefits:

      * Upgrading something high up on the stack (say, a custom node.js app) is as easy as whipping up a docker container for the new version, dropping it on the host server, then swapping out old with new, leaving you with an instant and pristine fallback should something go wonky.

      * outage due to code-side corruption or similar garbage? No problem - barf up a new container w/ non-corrupt code and demolish the old. Takes less than a minute for the small stuff.

      * scaling? again, no problem - spew out new containers to handle the load, then scale back when you don't need them... the time it takes is astoundingly fast compared to cloning VMs, and if you're using EC2 or the like to provide the expansion hardware, it won't require a huge CapEx spend sitting idle in your datacenter. Note that this requires a lot of infrastructure work underneath, but totally doable - containers just give it the speed to make it happen in short order. Put this way - a spike in user traffic is a cheaper solution when you pay for temporary cloud usage, than it would to have a massive VM farm hanging around doing mostly nothing in-between spikes.

      Now it does have drawbacks:

      * you really don't want to use this for things like databases. No, really - containerizing your DBs can be done with enough effort, but it's really stupid and expensive with little-to-no benefit.

      * if you have a massively dynamic CI/CD environment, finding/t-shooting the problem is going to be a bit more convoluted.

      * Security may be a bit harder to keep if you're even a little sloppy about it - now someone just has to pop the underlying server and swipe the entire container, instead of having to loiter on your equipment to find what they want. It's much easier to run that container at home on their own machinery, at leisure, with way less chance of detection.

      --
      Quo usque tandem abutere, Nimbus, patientia nostra?
    37. Re:Complicated by luis_a_espinal · · Score: 1

      Could we make things a bit more complicated? Virtualization I understand. I don't understand the need for Docker.

      Docker allows you to "packetize" your applications into their own secure containers in a standard format, with standard lifecycles. It makes it easier to distribute and deploy systems. And in cases when you want to provision environments, it gives you greater granularity.

      In terms of provisioning virtualized environments (say a server that collocates systems A, B and C), without something like Docker, you have to provision an entire system for each combination/version of A, B, and C. That creates a burden in terms of backups, snapshots and storage.

      With Docker, all you need is to virtualize the base system, the host, onto which you can deploy the Docker image of the systems in question.

      It is just another layer that will break.

      Every layer has a potential to brake. And by the same token, systems with less layers than they need will also break. How much layers you need, that is specific to the work at hand.

      Maybe I am just getting old, but this seems very complicated and prone to breakage with all the layers.

      One could argue with separating authentication from authorization (instead of cobbling them together into a single "security" layer). And why separate security from application. Let's couple the shit out of them.

      Again, it all depends on the work requirements at hand.

    38. Re:Complicated by luis_a_espinal · · Score: 1

      Docker actually makes things simpler! Since you understand virtualization, then think of docker as lightweight virtualization.

      how is adding stuff on top of LXC making things simpler?

      i followed docker from its inception and to me and my colleagues, they are the microsoft of os-level virtualisation.

      What's complicated about Docker?

    39. Re:Complicated by luis_a_espinal · · Score: 1

      To add to what I said, thinks BSD jails or Solaris "zones". If you see the reasoning behind any of these two, you get the reasoning behind Docker. Again, it all depends on the work requirements.

    40. Re:Complicated by Anonymous Coward · · Score: 0

      > Nobody's going to throw VC money at tcsh.

      Nope.

      *Sigh* I miss the days of the Dot-Com Boom...

  3. Market size? by Anonymous Coward · · Score: 0

    It might be unikernel

    It might be OCaml based

    It might have to do with container

    It might even be linked to virtualization and the ability to strip out to the bare essential

    But the point is, what is the market size?

    I might be wrong, but I am looking at a niche market, a very narrowly defined niche market

    If I am wrong, please correct me, and thanks !

    1. Re:Market size? by Anonymous Coward · · Score: 0

      Docker containers are a major thing now. They promise security, ease of movement between compute nodes, abstraction, and all of this on the cheap, especially with an OpenStack backend. It also is the "vogue" thing, and in a previous job, there was a push to put Oracle RAC active/active nodes in Docker containers.

      Realistically, Micro-VMs solve one major issue -- the fact that root in a Docker container meant root in the OS. However, it is still a solution looking for a problem.

    2. Re:Market size? by omfgnosis · · Score: 3

      They promise security

      No they don't.

  4. No more DLL hell then by Anonymous Coward · · Score: 0

    No more DLL hell then, just compile the entire OS into the application :)

    Works, indeed :)

    1. Re:No more DLL hell then by phantomfive · · Score: 1

      That's basically what Forth applications do......

      --
      "First they came for the slanderers and i said nothing."
  5. Paravirtualization by Shane_Optima · · Score: 1

    You skipped over paravirtualization, which can be damned fast (though not quite as snappy as LXC-based) and is theoretically in a much more secure container since the kernel is not being shared. QubesOS provides a good demonstration of how responsive a properly configured PVM can be. A "cold" boot can render a Firefox window in perhaps three seconds.

    The latest version of VirtualBox supports some level of paravirtualization as well, though I haven't had time to tinker with it yet.

    1. Re:Paravirtualization by Anonymous Coward · · Score: 0

      No I did not skip paravirt. hvm (hardware assisted virtualization) is now preferred over paravirt for the speed.
      All amazon ec2 instances now offer hvm over paravirt.
      But all said and done, there's this huge overhead of OS abstraction for VMs if you were to just run one server in the VM.
      Docker is best suited for such application oriented virtualization.

    2. Re:Paravirtualization by Shane_Optima · · Score: 1

      HVM is not is not faster than PVM. You might be confusing PVM for the software-driven virtualization that VMWare/QEMU/Virtualbox/etc. used prior to the introduction of VT-x.

      From my understanding, paravirtualization works by running a modified kernel in the guest so that knows it's really in userspace of the host OS (Ring 3) instead of where it would normally live (Ring 0). VT-x basically adds additional namespaces that allow guest operating systems to run in Ring 0, but this process adds some overhead. And for some reason I've noticed this leads to some massive slowdowns in some applications, especially anything that does a lot of FLOPS.

      I'm not an expert in these matters, but everything I've seen says (and my own experience verifies) that paravirtualization is generally faster than HVM. I'm running Qubes right now and all of my paravirtualized applications respond normally. Guest OSes in VirtualBox with VT-x enabled (on this same exact hardware) could get quite sluggish.

      I've never used Docker but it is (so I gather) geared largely towards LXC virtualization. This is radically different from either PVM or HVM as it is basically creating multiple namespaces within a single kernel. It should be faster than either HVM or PVM, though still not as fast as a chroot. Depending on your goals, I don't necessarily agree that it is "best suited for application-oriented virtualization." It's fast and it removes the problem of RAM management, but from a security standpoint the compartmentalization isn't nearly as airtight and you lose the flexibility of being able to utilize different kernels. The overhead of the alternatives can be non-trivial, but silicon is cheap. Like I said, it depends on your goals.

    3. Re:Paravirtualization by Anonymous Coward · · Score: 0

      Silicon is cheap, but a VM image these days often wants a gigabyte or more of RAM. A docker container may only require a few hundred MB. If you start stuffing boxes full of application systems, it pays off pretty fast.

    4. Re: Paravirtualization by bill_mcgonigle · · Score: 1

      nobody is quite right here. HVM is slower than PV that uses hardware assist for virt (aka PVH in Xen terms). HVM can be faster than soft-PV with some hardware. Empirically, HVM has been less secure than PV (that floppy driver ...). AMD is left behind with PVH, so YMMV.

      --
      My God, it's Full of Source!
      OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
    5. Re: Paravirtualization by Shane_Optima · · Score: 1

      Interesting. I hadn't quite got that far in my own research yet; I had assumed that PVH was just some kind of halfassed hybrid stopgap until a full PV solution for the guest in question could be created. But no, you're absolutely correct... it looks like the reason why paravirtualization is so much faster than vanilla HVM in practice is because of all the kernel routines and drivers in the guest OS that were replaced with special calls to the hypervisor (which I knew), but some of the context switching is still significantly slower than HVM (which I did not know.)

      It's too bad none of the terminology is standardized across different virtualization solutions platforms. I could do what I previously did and use VT-x as a synonym for HV but as you highlight that's not entirely fair to poor old AMD (or VIA, if they're still around.) Even if we stay within the Xen universe, it's more complicated than that because they've gone at it from two opposite directions: PVHVM ("Hey, you spilled some PV in my HVM!") and PVH ("Hey, you spilled some HV in my PVM!")... and it seems like the phrase "PV on HVM" can refer to either depending on the speaker.

    6. Re: Paravirtualization by Shane_Optima · · Score: 1
      Oh and one more thing:

      Empirically, HVM has been less secure than PV (that floppy driver ...).

      Is that true even when using VT-d (or whatever AMD's equivalent is) ?

  6. Oz the Wise and Powerful by Required+Snark · · Score: 4, Funny
    Yep, unikernels will solve all you system problems. It will be just as wonderful as living in the Emerald City in the Land of Oz.

    A unikernel, therefore, is an indivisible unit of computing logic. As a microservice, it carries the promise of unlimited scalability. ... And as an architecture, there’s a certain elegance in unity.

    Not only will it solve your deployment problems, provide scalability for free, and eliminate all system security issues, it will do your shopping, wash your car, clothes and dishes, pay your bills and taxes, balance your checkbook, and walk the dog, even if you don'thave a dog!

    Overhype much?

    --
    Why is Snark Required?
    1. Re:Oz the Wise and Powerful by Anonymous Coward · · Score: 0

      Shut UP! and take my money!

    2. Re:Oz the Wise and Powerful by Anonymous Coward · · Score: 0

      It sounds like it would have been better off just porting to GNU Hurd.

  7. Slowly building a decent OS, ass-backwards. by Shane_Optima · · Score: 1

    All of our commonly used, general purpose, end user operating systems suck. Separation, compartmentalization simply isn't there. Everyone knows the advantages of proper compartmentalization: easy portability, unparalleled security, a mighty robustness against system-crashing errors and reboots, configuration flexibility and avoidance of dependency conflicts, the ability to easily prioritize system resource usage, the ability to roll back snapshots without rebooting the entire system, sharing or cloning base images, etc. But our OSes can't do any of this out of the box.

    So people slowly added it back in, first with file permissions and then mandatory access control (Apparmor, SELinux), chroots, FreeBSD chroots, traditional virtualization (at first entirely software-based, then hardware-assisted), PVMs, LXC, Unikernel, and probably a dozen more approaches I'm unaware of. All of these things have advantages and disadvantages, but the important thing to keep in mind is this: they're all trying to fix something that is fundamentally broken. And ultimately all of the best solutions involve figuring out different ways of emulating multiple OSes because the applications we want to run only run on "modern" OSes... none of which are built around a concept of strict compartmentalization.

    Enter the Genode aficionados and the hordes of older microkernel partisans who've been screaming about this stuff for decades. They will scoff and say a bunch of undeniably true things... but of course, last time I checked everyone who actually used Genode for everyday tasks was using it to boot VirtualBox or something similar. It's inescapable: we have a bunch existing applications we want to run and we simply can't run it without (in some fashion) running the crappy OS it was designed to run on.

    And so we have a dozen different varieties of duct tape to choose from, each with their own advantages and disadvantages. It's interesting. It's fascinating. Currently, I'm leaning towards paravirtalization as the sweet spot of performance and flexibility and security (Qubes OS was the decisive influence on me here), but LXC is faster and simpler and Unikernel, while not simple, sounds like it could be very fast indeed. And I'm sure the dirt-simple appeal of vanilla HVMs will endure regardless of the advances Docker makes.

    It's an interesting debate with no single right answer because of the long list of different pros and cons. But it's also interesting to consider how and when all of this is likely to be rendered moot. At some point, a new OS (probably microkernel based) will finally gain enough traction and get enough native end user applications to make it usable without in any fashion having to emulate another OS on top of it. People will start porting other applications to run on it, and eventually the Windows and Debian and Red Hat and OS X and Android and iOS of today will finally be abandoned as the insecure, unstable, kludge-ridden clusterf___s they are.

    The question is... will this revolution be driven by an OSS project? Will the microkernel world finally get their Linus? Or will this be the ultimate proprietary walled garden, a massive investment by Apple or Microsoft or Google, some heart-stoppingly expensive super secret project to finally offer their customers a UI that never gets laggy or unresponsive, a core system that is virtually immune to malware, app developer APIs that rarely need changing, real time performance when necessary, and fine-grained app permissions that actually work properly?

    Unfortunately, I think my money has to be on the walled garden, and even more unfortunately I'd have to say that Apple stands the best chance of pulling it off given their strict control over hardware and their titanic cashflows. Maybe Google will get there first and maybe, just maybe someone will be able to convince them to open source it.

    It's fascinating to watch all of these brilliant people come up with these interesting and inventive kludges. And useful, undeniably useful in the here and now. But at the end of the day, I can't help but think... dust in the wind.

    1. Re:Slowly building a decent OS, ass-backwards. by swb · · Score: 1

      I think HVM will be here to stay and continue to be widely adopted because its one of the few ways to solve the problems associated with dependencies on the crappy OS underneath. And sometimes the dependency on the crappy OS underneath is really about supporting the crappy application on top which doesn't support a newer version of the OS.

      I almost never see a heterogeneous OS environment, at best variation within a given OS family's version (eg, Windows 2003, 2008, 2012) and quite often multiple operating systems, even if the non-Windows ones are just appliance-type instances for supporting other products.

      And the appliance piece seems to be a growing, not declining, trend, with some traditionally hardware-based network appliance vendors shipping VM appliances as an option. These don't work at all outside of HVM.

      If operating systems would ever stabilize their release cycles and get more into long-term version releases that include bug fixes and non-destructive feature additions, maybe paravirtualization would gain more traction because you wouldn't have the vicious cycle of new apps needing new OSes and old apps being incompatible with new OSes.

  8. Hurds of unicorns. by Anonymous Coward · · Score: 0

    If these are unikernels, GNU Hurd is a mythicaal unicorn.

  9. Wtf is docker? by Anonymous Coward · · Score: 0

    Wtf is docker?

    1. Re: Wtf is docker? by Anonymous Coward · · Score: 0

      Something my CEO thinks we need because he's heard people talking about it.

  10. Yeah whatever by drinkypoo · · Score: 1

    Docker aims to make the process as standardized as possible, for easier deployment.

    You know what we actually need? Tools to make using selinux easier, "for easier deployment". I'm given to understand that there are tools that are supposed to watch activity and build you a profile, but I couldn't even get the tools to work.

    --
    "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
  11. unikernel? by Anonymous Coward · · Score: 0

    More like DOS in a can.

    Ok, so it is somewhat an improvement on that as it has memory protection and can properly multitask if need be.

    But these unikernels are beyond that single user, and you need extra libs for anything beyond barebones bootstrap.

  12. Meanwhile at Intel... by tarpitcod · · Score: 1

    The dusty file-cabinet creaks open revealing the secret plan.

    Shadowy figure one: "It's taken a few more years than we originally expected, but the day is drawing near. All this virtualization, exo-kernel, uni-kernel crap is just garbage compared to this baby!"
    Shadowy figure two: "But we will need to make it 64 bits..."

    Shadowy figure one: "No problem, that'll take a few weeks, plus we can fit ALL of it on one die with plenty of cache. GDP, IP, plus we always planned the IO Processor to be x86 compatible, so we have x86 - no problem! It'll be fault-tolerant and run circles around the competition plus we can bolt on that fast vector unit"

    Shadowy figure two: "But what about marketing?"
    Shadowy figure one: "Are you kidding me,they'll love it, they can re-use some of the the original stuff too."

    iAPX 864 - MICRO SUPER COMPUTER.

    In a stunning development Intel has released a new 64 bit microprocessor architecture designed to radically address the serious issues with virtualization, containers and other security mechanisms.