Slashdot Mirror
Virtualizing a Supercomputer
bridges writes "The V3VEE project has announced the release of version 1.2 of the Palacios virtual machine monitor following the successful testing of Palacios on 4096 nodes of the Sandia Red Storm supercomputer, the 17th-fastest in the world. The added overhead of virtualization is often a show-stopper, but the researchers observed less than 5% overhead for two real, communication-intensive applications running in a virtual machine on Red Storm. Palacios 1.2 supports virtualization of both desktop x86 hardware and Cray XT supercomputers using either AMD SVM or Intel VT hardware virtualization extensions, and is an active open source OS research platform supporting projects at multiple institutions. Palacios is being jointly developed by researchers at Northwestern University, the University of New Mexico, and Sandia National Labs." The ACM's writeup has more details of the work at Sandia.
Now we'll never need to build another expensive supercomputer. We'll just "virtualize" them on cheap desktops.
Oh. Wait...
Warning: this article may contain humor, sarcasm, parody, and perhaps even irony. Read at your own risk.
This is virtualization... Imagine someone Imagining a beowulf cluster of those!
-Matt
--- Need web hosting?
5% may not sound like mubh, cut with 4096 nodes that's over 200 nodes that they are wasting.
What is the point of virtualizing a supercomputer? A 5% performance loss is a pretty big loss, in say a cluster of 100 computers, 5 of them would be wasted translating to thousands of dollars lost with little to show for it.
Taxation is legalized theft, no more, no less.
It is really pleasant to see more and more OSS projects which are being deployed at national level and large infrastructures.
Hopefully some less greedy company who benefit from such projects will start paying the volunteer developers. But then again, I have found that a lot of times if you are doing something as a hobby/interest/challenge, rather than because you were employed to do it, the outcome will be more refined and efficient. Though I have yet to experience the latter part first hand.
Imagine a Beowulf cluster...
Warning: this article may contain humor, sarcasm, parody, and perhaps even irony. Read at your own risk.
Virtualizing a Supercomputer is never the correct solution. supercomputers have in their nature a system of managing lesser processes. that system could be extended rather than adding another virtual management system to run parallel to the existing management system burdened with maintaining it as another running process.
The way virtualization works is it is a virtual layer spread across many nodes to avoid any down time when you get
one node that fails, the rest pick up the slack, and without having to stop the running systems. This is using linux architecture to
cluster many computers on the bottom layer, so as to have the look of one mega computer, when it actually is 100 computers or more...etc...
Then we get into supercomputing, which again uses clusters and usually uses linux, to be able to make all the computers act as if it was one big computer, giving the advantage of multi-processors to be able to calculate much faster common operations, etc....
Now combining the 2, we could ....what is the advantage again, of putting a cluster on top of a cluster, I need to understand, because I don't see it, either one of these are used to make a supercomputer per se, but one is virtual, the other is physical....
either case, the advantage is the same from both, but merging the 2, would have too much of a slow down if you ask me, with all the
backend needing to monitor the other backend to load balance , raid, etc.... it just seems like it is a test to see if you can do it, but would you get any real advantage out of it, I am not so sure....someone with knowledge of vms, and supercomputers , please enlighten me.
We're not trying to hide anything, and so I will admit to being surprised by this (anonymous) accusation. To address the anonymous coward's concerns, however:
1. Actual users of supercomputers care most about application run time because applications are what scientists run, not micro-benchmarks. As a result, our paper and research more generally focuses on the runtime penalty to real applications (e.g. Sandia's CTH code) as opposed to focusing on optimizing micro-benchmarks that aren't what real users of these systems care about.
2. Micro-benchmarks do provide useful information about the exact costs of various low-level operations, however, to the extent that they can show you what is causing the application slowdowns you do see. They also can potentially help understand how proposed changes might impact applications other than the ones we were able to run in our limited access to the production Red Storm system. Because of this, the paper the anonymous coward above refers to explicitly measures and presents micro-benchmark latency and bandwidth overheads. Specifically, it cites the latency cost on both Red Storm's SeaStar NIC (5 or 11 microseconds, depending on how you virtualize paging) and QDR Infiniband (0.01 microseconds). It also presents a bandwidth curve to fully characterize virtualization's cost over the full range of potential message sizes on SeaStar. (IB is less expensive to virtualize than SeaStar, because IB doesn't have interrupts that Palacios must virtualize on the messaging fast path where as SeaStar does, at least when running Cray's production firmware).
We're very up front about the costs of virtualization because we are well aware that there is no such thing as a free lunch. Virtualization provides a number of potential advantages in supercomputing systems, for example in terms of dealing with node failures, providing a small open-source platform for OS research and innovation on supercomputing systems, handling applications with different OS feature and performance requirements, and a variety of other things. However, it does come with a cost to applications and application scientists that has to be weighed against its potential benefits.
Palacios can run on real x86 hardware or on QEMU. In fact, most of our development is done on QEMU, which is open source. The VMWare image was something we did on the original 1.0 release just to help people get started running it and haven't done since, but VMware has *never* been required for development.