Slashdot Mirror

← Back to Stories (view on slashdot.org)

Ask Slashdot: Safe Learning Environment For VMs?

Posted by timothy on from the first-do-no-harm dept.

First time accepted submitter rarkian writes "I am the teacher in this story. I teach Python and C++ to high school students: grades 9-12. I use CentOS 6 with DRBL to run my computer lab. Some of my students have become Linux experts. Next year I'm planning on allowing students to create and run their own VMs in a segregated LAN. Any advice on which virtualization technology to use and security concerns with allowing students to be root in a VM?"

42 of 212 comments (clear)

  1. Set up VLANs by Anonymous Coward · · Score: 4, Insightful

    for each of the students and don't allow any interface between them...and certainly no main network/internet access.

    1. Re:Set up VLANs by ttucker · · Score: 2, Insightful

      for each of the students and don't allow any interface between them...and certainly no main network/internet access.

      VLANs are not for security! Any two things plugged into the same switch, whether virtual or real, can talk to each other if sufficiently motivated.

    2. Re:Set up VLANs by TheCarp · · Score: 4, Interesting

      > This is Linux were talking about. What could you possibly teach them without an internet
      > connection? The only use I see is teaching shell scripting or something, since other tasks like
      > package management, and sane server configuration kinda require an active internet connection.

      Well Python, that is what he said he wanted to teach them :)

      Beyond that, you are right, I say, the key is not security but recovery. Template boxes so that a new VM can be spun up effortlessly, then let them have at it. Segment the lab off from the rest of the network, maybe let the lab out to the internet, but not to the schools internal network.

      However the key is, you can always blow away a machine and reimage it to get class moving, so there is no real danger in letting them play. Hell, I took a class that taught us administration and hacking eachother's machines was explicitly allowed, as long as it stayed in the lab. (we were also required to create a guest account as one of the exercises)

      Gives the advanced kids who are bored with the class something to do that lets them learn too; I certanly had fun.

      --
      "I opened my eyes, and everything went dark again"
    3. Re:Set up VLANs by LordLimecat · · Score: 4, Informative

      Not if they are on private VLANs (still not clear if it is a standard or not). VMWare supports this; the idea is roughly analogous to AP isolation on a wifi AP-- "isolated" nodes can all talk to the gateway (the community / public node), but cannot talk to each other.

      And VLANs actually are for security, and can provide far superior security than ACLs, since unless you have a trunk port or layer 3 switches, it is impossible for two devices on different VLANs to communicate, short of a switch misconfiguration. Its probably second to air-gapping in terms of security-- its sort of a logical implementation of "air gapped switches", except that they CAN be joined together if someone gets onto the switch.

    4. Re:Set up VLANs by ttucker · · Score: 3, Informative

      VLANs are not for security! Any two things plugged into the same switch, whether virtual or real, can talk to each other if sufficiently motivated.

      This is simply not true. You're probably referring to 802.1q tag hopping attacks, which are not particularly difficult to prevent.

      Do you really think tag hopping is the only attack on VLAN? Perhaps you should read what Cisco says about the matter, if your job depends on it: http://www.cisco.com/en/US/products/hw/switches/ps708/products_white_paper09186a008013159f.shtml . They point out that with an adequately secure switch, and good configuration practices, that the security is adequate. Without all of the correct practices, VLANs provide a very dangerous false sense of security.

    5. Re:Set up VLANs by MattW · · Score: 3, Interesting

      VLANs are not for security! Any two things plugged into the same switch, whether virtual or real, can talk to each other if sufficiently motivated.

      As you pointed out below, VLANs in general are trustworthy when properly configured with a proper switch. I did nothing but netsec work in the late 90s, and everything was airgapped; we'd never have frames from two networks on the same wire. If you wanted to cross security zones, it was at L3 on a firewall and to different wires and switches.

      On the other hand, it seemed like back then a new practical way to defeat VLANs was coming out every other week, so this was a wise precaution.

      That said, keep in mind that VMware also affords some additional security in terms of VLANs. Physical switches have to connect to virtual switches to interact with the VMware layers (either the hypervisor for control traffic, or with the VMs for VM traffic), and the hypervisor itself will enforce a lot of things. On a VMware vSwitch properly configured:

      - VMs can't enter promiscuous mode, change their MAC address, or forge transmits with the wrong L2 address
      - QinQ frames are discarded
      - The hypervisor itself will determine which virtual nics on a vswitch should receive copies of a frame, depending on which VLAN tag is on a portgroup
      - Guests can't send tagged frames if their portgroup is set with a VLAN; you have to specifically configure a trunk on a portgroup to pass VLAN tags in and out of the guest environment

      If the network was homogeneously ESX nodes and administratively controlled network equipment, you could likely enforce security between VMs with VLANs even with a dumb hub.

      Obviously, airgapping and single-role wires will create better security than VLANs, because there always remains a chance that an undiscovered bug will allow breaching that L2 barrier, but that's true for everything.

    6. Re:Set up VLANs by AlphaWolf_HK · · Score: 2

      With a cheaper consumer grade switch, perhaps. However enterprise grade switches (which I'm going to assume he is using) there are all kinds of features that can prevent anything you described above, such as MAC flooding, VLAN hopping, broadcast storming, deliberate switching loops, and more.

      --
      Careful with names containing L slashdot.org/~AiphaWolf_HK slashdot.org/~AlphaWoif_HK slashdot.org/~AiphaWoif_HK
    7. Re:Set up VLANs by evilviper · · Score: 3, Interesting

      VLANs are not for security! Any two things plugged into the same switch, whether virtual or real, can talk to each other if sufficiently motivated.

      Umm, no. Not unless your switch is defective, or massively misconfigured. VLANs are very secure, when done properly. And the same security measures needed to protect VLANs are the same ones you need to protect switching in general (see CAM overflows, arp spoofing, and such).

      http://www.cisco.com/en/US/products/hw/switches/ps708/products_white_paper09186a008013159f.shtml

      If you leave your trunk/native VLAN at 1, you're in trouble. If you configure user-facing ports as auto-negotiate, or trunk without explicitly specifying allowed VLANs, you're in trouble.

      --
      Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
    8. Re:Set up VLANs by ttucker · · Score: 2

      That's not true at all. Yes there exists a VLAN hopping exploit, but it is easily prevented by modern switches. While VLANs weren't intended for security in the beginning, that has become one of their new purposes. Otherwise, layer 3 switches would probably never be used in any environment where security was major a concern, but that's simply not the case.

      Another extension of the VLAN concept is PVLANs, whose purpose is for nothing else other than security, primarily used in hotels to prevent hacking, but has other uses as well, nearly all of them security related.

      Some further research indicates that my initial reaction to VLAN and security is somewhat dated, and VLAN tech has improved drastically since. Still, there does seem to be a few places where poor configuration could lead to a spectacular breach, simply because any exploit of the switch allows an attacker to access any VLAN segment.

    9. Re:Set up VLANs by cas2000 · · Score: 2

      then there's no good reason for them to have root.

      depending on the work environment, devs may or may not have root on their own workstations, but definitely shouldn't have it on the development, testing, or production servers unless part of their job is systems administration (either because they're an assistant sysadmin or because their employer is a cheapskate and thinks that sysadmins and programmers are the same).

      actually, i'm partly wrong - there's one good reason, to teach them how to cope with the mess caused by the kind of bad systems adminstration done by a programmer who sees it as a boring chore that uses up valuable dev time. but that would only work if they only had root on another students' VM rather than their own.

      another good reason is that some students will realise that they prefer or are better skilled at sysadmin work than at programming, and devote their future studies and work to further that....programming and systems administration require similar skill sets, but quite different attitudes and aptitudes.

    10. Re:Set up VLANs by Drakonblayde · · Score: 2

      Blatantly incorrect. The vlan is what defines the broadcast domain. A broadcast (by default anyway) will reach every single port in a given vlan, and no other.

      You really should do your research, this is absurdly easy to disprove.

  2. Vagrant by Pinhedd · · Score: 4, Informative

    Vagrant is a wrapper for Virtualbox and VMWare Workstation that accelerates the deployment of development environments.

    http://www.vagrantup.com/

    1. Re:Vagrant by Pinhedd · · Score: 2

      It automates the setup of the isolated development environment. A company can put a Vagrantfile into a development repository along with any associated setup scripts. As long as the developer has VirtualBox/VMWare Workstation and Vagrant installed, they can type `Vagrant up` from within the repository and it will automatically setup the development environment within the development VM. This is a really easy way to ensure that everyone is working on the same page.

    2. Re:Vagrant by Anonymous Coward · · Score: 2, Insightful

      You can check in a Vagrantfile and Chef/Puppet scripts that create a working development, test, or production environment from a base OS install and Vagrant can spin them up with a single command. It simplifies creating a new development, build, or test environment, makes everything repeatable, and gives you assurance your stuff will run again when installed on a fresh box instead of a hand-crafted VM image. The Chef/Puppet stuff is applicable to production too, so through the course of dev/test you build up a cookbook that lets you deploy anything at the push of a button. No more huge, out-of-date documents saying "install these packages, configure this, create users, blah blah blah" that you spend hours scratching your head over. No more "works on my machine" puzzles. It's really the way to go.

    3. Re:Vagrant by FlipperPA · · Score: 2

      +1 to Vagrant for local development. If there are any problems, you can easily just blow away the VM and start again. Vagrant scripts all of this with the same Chef repos you use for production. I modified and updated a Django Vagrant run-through I found on the net. It is free to the public on Github:

      https://github.com/FlipperPA/djangovagrant

      Naturally, this includes Python as well. I hope this helps gets you started, and thanks for going above and beyond with your students!

    4. Re:Vagrant by styrotech · · Score: 2

      I was going to suggest that too.

      Turn the problem on its head. Instead of supplying them with an actual virtual sandbox to play in (lots of work) - give them the ability to spin up a bunch of different virtual network/server configurations on their own machines.

      Vagrant configs can specify multiple servers and networks etc.

      They can easily be blown away and rebuilt, they can start sharing their own custom environments.

      BTW: checkout salty vagrant. Salt is a Python based configuration management tool like Chef and Puppet but much simpler and lighter weight. And because you're teaching them Python and most Linux distros already have Python builtin, you don't need to install much else to bootstrap it.

  3. Re:Safety? by Anonymous Coward · · Score: 2, Interesting

    Unless they take down the network, e.g. running a rogue DHCP server. Or they use it to hack other systems on the network, e.g. password-sniffing the other student's credentials when they log in from their VMs.

  4. Re:Safety? by fwice · · Score: 2

    if the VM has a full root account, with a network address on the global network at large, then it has the ability to, for example, run a priviledged NMAP scan on the entire network. Which can expose open ports or vulnerabilities on another machine that can then be used to leverage access.

  5. VM is irrelevant by onyxruby · · Score: 4, Interesting

    The fact that your using VM's is largely moot and goes back to the line of thought that VM's are somehow not 'real' computers. VM's run the same operating systems, software, have the same bugs, vulnerabilities and everything else as a physical computer. You need to patch them just like any other computer and you need to license them just like a regular computer. The fact that they are VM's really only makes two differences practical differences that matter, fist is that is easy to roll them back and second is that your aren't running on bare metal.

    In other words you have a core issue that needs addressed of giving students root access to a computer. In an isolated environment this isn't necessarily a bad thing. Understand that they exploit root and see what they can do with it, however they are there to learn and if you can do so safely and without disruption of what your trying to teach then let them. Your focus needs to be on making it safe for those around them and that means making sure your VLAN and any related Internet access are properly setup. The lab is a lab and as long as you can make sure they aren't getting access to anyone persons computer than let them have at it.

    A good rule of thumb is to roll your sessions back prior to the start of every single class. This always gives a fresh machine and the students will quickly learn how to set their VM just the way they want it.

    1. Re:VM is irrelevant by Archangel+Michael · · Score: 4, Informative

      VMs have one advantage that non-virtualized systems don't have. The ability to put several machines in their own sandboxed network, all managed by a single student who needs to demonstrate cooperating systems. Give every student a template of needed machines and a VM server and you have a small lab on every computer. One that is easily setup, cleared and re-setup for every class, and as needed.

      VMs are a perfect solution for advanced computer systems management training.

      --
      Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
  6. SELinux on the host by hpa · · Score: 3, Interesting

    Make sure you have SELinux enabled (and enforcing!) on the VM host, and keep the VMM software updated... there sometimes are security holes in VMM software which can be exploited. SELinux can help contain a breached VMM.

  7. VLANs are your friend by StoneyMahoney · · Score: 2

    Just in case anyone gets a bit... shall we say "Adventurous" and tries to use their root access boxen to attack something they shouldn't, it might be worth isolating the VMs on their own VLAN away from the rest of the network, if you haven't already.

  8. 'Create' is the tricky part by bill_mcgonigle · · Score: 4, Informative

    Next year I'm planning on allowing students to create and run their own VMs

    Running their own VM's is straightforward. Allowing the students to create their own VM's implies that they'll be root on the hypervisor.

    Do you intend to run the hypervisor on the client machines of the DRBL system, or run a single hypervisor on the server and deploy the VM's there as DRBL clients?

    To satisfy your requirements you probably want to run the hypervisor on the clients so they students can each have their own root on the hypervisor. This would require a hypervisor compatible with DRBL. I don't know how it works, but just from reading the description on the webpage, it sounds like it's geared to PXE booting a host OS.

    If you go with Xen, you'll have to probably separately PXE boot Xen and then DRBL boot the Dom0. Which would probably work fine and get you decent performance, but it will expose the students to DRBL (is this what you want?)

    If you go with KVM, the performance is a bit slower, but for a student shop that's probably OK, and you'll be able to DRBL-deploy the hypervisor and then let the students create their own non-DRBL (or DRBL) guests. This probably fits your model the best unless you have old hardware that KVM does not support - then you might need to go with the Xen-PXE-Boot model (because it can paravirtualize without hardware assistance).

    You could also use VirtualBox, and while it offers a nice GUI, it's probably too simple for teaching your students about virtualization (it just feels like an app).

    BTW, it sounds like you're doing great work based on that article. Kudos on your accomplishments and being an inspiration for others in your field.

    --
    My God, it's Full of Source!
    OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
  9. Re:Virtualbox by h4rr4r · · Score: 3, Informative

    And is utter trash for anything that needs to be scalable.

    It is fine for a desktop VM system, but it simply does not offer the management interfaces that other solutions have. Basically the options here are VMware and KVM. The first if you want a shiny GUI the latter if you are ok without one. Both will let you script everything they do, which will be very handy when you need to reset 100 VMs for the next batch of students.

  10. VMWare, Ubuntu and Puppet by i_want_you_to_throw_ · · Score: 2

    I see this as being similar to when we needed to have all of our developers in my company working in an environment that absolutely matched the production environment. Just use VMWare on each individual machine, run an Ubuntu image in that and best of all use a Puppet script to customize it and give 'em the goodies they need. The beauty of this is once the kids screw it up (and let's hope they do, they're learning after all) then you can rebuild this back to a pristine machine in no time. Good luck!

  11. Re:Safety? by Nerdfest · · Score: 2

    They can always plug in their own laptop and do that anyway.

  12. oVirt by knarfling · · Score: 2

    Depending on your equipment and the time you want to spend, oVirt might be an answer.

    Although it is still fairly new and is in development, it runs on CentOS6, is free, can handle multiple guest OSes, can create VM's from a template, and has a power users portal page where trusted students/employees can create their own VM from supplied templates. This way, no student would have access to the host OS, but could create a VM as needed. The downside is that it can get quite complicated to set up the system, and could take a bit of time to learn and set it up properly. Since it is free, you are also dependent upon community support.

    You can access more info here.

    --
    Great civilizations have lived and died on false theories. Don't mess up mine with a few facts.
  13. Re:Safety? by Archangel+Michael · · Score: 3, Informative

    Which is why they need to setup their own VLAN to isolate the VMs to the classroom. VM traffic is isolated to non-routing VLANS. They call this setup a "sandbox", and it is generally a good practice for classroom work.

    As for which VM technology to use ... VMWare, or ZEN or even Microsoft's version are usable. VMWare is sort of free, Xen definitely is. I'm not familiar with pricing on Microsoft's versions but schools tend to get steep discounts for server licenses. Look at OpenStack for management, I hear it is decent when it works.

    --
    Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
  14. ESXi by meowgoesthecat · · Score: 2

    ESXi. Its free, powerful, and offers a lot of pre-built appliances. I don't see any safety concerns if the network is segregated. If you have specific VM's that you want the students to learn within, keep screenshots of those so that you may roll anything back that gets damaged. This is great because it allows them do pretty much anything they want without creating a maintenance headache for you.

    If you want to teach them about specific technologies using VMs that go hand and hand with programming (like source control, bugzilla, configuring web servers, etc), turnkeylinux.org offers many free linux appliances that will make your job easy.

    --
    Meow
  15. Re:Network Security by ttucker · · Score: 2

    I don't think you need to worry about OS security, since that is the point of using VMs. However, the "key" to this question is the definition of "segmented." There are host of nefarious and simple mistakes you can make to completely trash the network of the of the VMs. I would recommend disabling multicast.

    Banning the use of fork() can't hurt either.

    Yes, banning fork() can hurt, because how else are you supposed to learn about it. Also, running a forkbomb in a VM would have no effect at all on the VM host.

  16. VLANs, RH Virtualization Security manual, virt-man by raymorris · · Score: 5, Informative

    Thanks for going the extra mile with your students.

    As AC said, a separate LAN or VLAN, or multiple separate LANs/VLANs handles most of what's posted below. For example, a rogue DHCP server would only be visible on that VLAN.

    Red Hat has a Virtualization Security section in their manual:
    https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Administration_Guide/chap-Virtualization-Security_for_virtualization.html

    CentOS/RHEL includes comprehensive support for KVM with virt-manager. While VirtualBox et al are fine for running one or two virtual machines on your desktop, for many VMs, with new ones created and removed each semester, the enterprise level support of KVM built into the distro is more appropriate. That support includes creating VLANs within the same management interface, for example, and integrates with the built in storage stack administration tools. Again, VirtualBox may be simpler to set up for one or to two machines, so I'm not saying it's not good - it's just not the best tool in this particular scenario. In this type of scenario, the KVM / virt-manager / virsh stack that RH baked in is probably a better match to the needs.

  17. Re:VirtualBOX by Archangel+Michael · · Score: 3, Informative

    Forkbomb is only successful if you don't have limits on your VM environment. You have put limits on your environment, right?

    --
    Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
  18. Re:VirtualBOX by NatasRevol · · Score: 2

    A nice forkbomb in a single VM can cause headaches for the rest of the environment.

    Then it's a very poor environment.

    We're talking about one or many classes of students. If it's not built out to handle several VMs using their max CPU concurrently, then it's a very poor environment.

    Heck, everyone compiling at the same time would shut things down if the environment is built poorly.

    --
    There are two types of people in the world: Those who crave closure
  19. APT-Cacher, Squid by SgtChaireBourne · · Score: 3, Interesting

    A good rule of thumb is to roll your sessions back prior to the start of every single class. This always gives a fresh machine and the students will quickly learn how to set their VM just the way they want it.

    They can start each class with a fresh snapshot. In effect they would be restoring from backups. The configuration files from some other networked storage or their thumb drives and the applications themselves from the repositories. I've done something similar, but on bare metal, and after about half a dozen times they don't notice -- it had become such second nature to install and restore applications. Heck you might even have them practice installing the whole system from scratch. If you go that route, they can become quite proficient with installation and resource allocation. PXE booting a netinstall image helps there.

    However, once you start to load packages from the net things can really slow down unless you prepare. The best way is to have a cache like APT-Cacher or Squid on your LAN or host system and have them configure their systems to use it for APT. For the cache to be most effective, you have to pre-load it before each class. That's easy and can be done while doing other things. It only takes time not attention. But once you have the cache loaded, installation will fly and can be done in 15 - 20 minutes. After that they weren't shy about installing on their own computers at home or helping their friends.

    --
    Beta is broken and the link to classic doesn't work. Stop wasting our time or there won't be anybody left here.
  20. It depends on what you're trying to protect by dankney · · Score: 3, Interesting

    So far, I see lots of advice about VM breakouts and network isolation. If this were a production datacenter where uptime was a criteria, this is all well and good. I suspect that this isn't what you need to hear, however.

    I see three things you could be attempting to protect:

    1) The larger school network.
    2) The VM host infrastructure.
    3) The VMs themselves.

    1) A student on a VM is no more dangerous to the network than one who can connect to the school wireless with a laptop or smartphone. If the lab uplinks to the same network as the broader access, your risk profile is unchanged.

    2) Make sure the VMs can't route to the host and keep it patched. If a student managed to break out of a VM in a patched hosting environment, do some forensics and find the bug then sell it. It's probably worth more than you make in a year. Seriously, if they can do this, they deserve to win. You might as well worry about protecting against nation-state sponsored attacks.

    3) Make sure that the class work is backed up (a git server, perhaps) and then don't worry about it. Seriously, just throw the VMs away after each class (or every night, etc) and start with a clean one the next time they log in. Don't spend time trying to outsmart a classroom full of bored highschoolers. Instead, make it so it doesn't matter when they break something.

  21. Re:Virtualbox by Anonymous Coward · · Score: 2

    Wow, two whole servers. However did you scale so high?

  22. Re:Safety? by Joce640k · · Score: 3, Interesting

    Unless they take down the network, e.g. running a rogue DHCP server. Or they use it to hack other systems on the network, e.g. password-sniffing the other student's credentials when they log in from their VMs.

    So... nothing they couldn't do much easier/more safely by just pulling the network cable out of the physical machine and connecting it to their netbook?

    --
    No sig today...
  23. Re:Safety? by Joce640k · · Score: 2

    You know how I know you didn't even read the summary...?

    --
    No sig today...
  24. Re:Safety? by LordLimecat · · Score: 2

    I am not familiar with KVM or other Linux VM solutions.

    I do know that during my VCP cert course, all students were provided with a VMWare ESXi infrastructure that was entirely virtual, contained in a vApp on a parent vSphere infrastructure. We all had our own connection on our own vSwitch, but no uplinks to everyone else, so there really wasnt much anyone could do to interfere with other students.

    I suppose one of the students could try to defeat the vSwitch segregation via an exploit, but I think if they pull that off they dont need the class and deserve an instant A (assuming responsible disclosure).

  25. Re:Virtualbox by greg1104 · · Score: 2

    VirtualBox provides the VBoxManage tool for automating operations. It works perfectly fine for this sort of thing. One of my small servers at home is running 28 VMs with all management happening through the command line, and that hasn't even gotten close to whatever the upper limit is. You certainly can run a classroom worth of VMs on a modestly sized box.

    The only major management feature that's much easier on VMWare than VirtualBox is moving VMs to new systems. That is very useful for large production VM deployments, but it doesn't sound necessary for this situation.

    I use VirtualBox because there is an open source release that works across multiple platforms. VMWare is all closed, and Xen only works on UNIX-ish systems. Students in particular can benefit from running a VM copy of Linux on another host OS, because it provides a way to get familiar with the software on either a Windows or Mac laptop (which they probably own already). That requirement rules out Xen as a good example. And if you're going to introduce students to open source software via Linux, it's nice if you can present that lesson on an open source stack too.

  26. Re:Network Security by Andy+Dodd · · Score: 2

    So - have a lesson on forkbombs prepared for when that happens.

    --
    retrorocket.o not found, launch anyway?
  27. A bit of cloud security author advice by MattW · · Score: 2

    So, I co-wrote this book on virtual security and am a former VMware Cloud Solutions Architect. And I'll preface this advice by saying that, if you want to talk more in depth, feel free to ping me. First initial, last name at gmail will work. (The email I have attached to slashdot I glance at occasionally, but it gets almost purely spam and so I'd likely miss anything.)

    From my perspective, the first question is which hypervisor to use:
    - VMware is mature, you can get a free license for the base hypervisor (which is quite feature rich; this is no trial product) for up to 32GB per physical box, is widely used. If VMware remains as relevant in the future as it is now, it's actually a very solid skillset to have.
    - If you have physical hosts over 32GB, VMware ceases to be free
    - Some features require more advanced VMware stuff, including vCenter server, which isn't free - for example, VMware's live vm migration feature (vMotion)
    - VMware is almost entirely closed on the internals; hypervisor is closed source (other than a not-useful-for-your-purposes "open source" bundle that contains their modified GPL code only); they have a bunch of APIs for internal functions (ie, tracking changed blocks on the virtual iscsi devices, for example), but those are generally restricted to partners; so if your students want to actually hack the virtualization layer, they can't. Then again, letting them do so wouldn't really be safe.
    - On the other hand, VMware layers do have nice APIs that are reasonably accessible for doing non-internals stuff; things like powering VMs on and off, changing their allocated RAM and cpus, etc
    - VMware has a nice set of tools, including CLI tools, which work well even with the free versions, that can allow you to move virtual machines in and out of specific hypervisors (not while the VMs are powered on), and into and out of VMware's desktop products (Workstation for Windows and Linux, Fusion for Mac). (google ovftool for the cross-platform CLI tool, for example; it can import/export to/from ESX, vCenter Server, Workstation, Fusion, and vCloud instances)
    - VMware has a nice set of tools for snapshots and backups, even on the base hypervisor; for example, I have a personal ESX box at a provider and I use this tool to back up the VMs back and forth, which can be done from outside the OS without powering the VM down, and it's free.
    - I found using some things I'd think of as mandatory for a lab environment (ie, thin provisioning) were just built-in on the VMware side and required a fair bit of extra work and added extra wrinkles

    The virtual networking on VMware is dramatically more mature from my experience; my experience with Xen & KVM is now dated (it's been 2 years since I was in the thick of writing that book, which was the last time I was really in the thick of exploring the open-source hypervisor networking bits). I found that depending on the version of the hypervisor OS, which hypervisor, which kernel, which guest, etc, you could fall into all sorts of traps. I had some examples in the book where I showed, for example, generating and applying ebtables configurations to the host OS (the Xen Linux hypervisor OS) to block forged frames from coming across the bridge from one of the guest Linuxes, for example.

    Compare that to the VMware side, you could in theory wire up everything to dumb hubs, even, and enforce network separation at the hypervisor layer with VLAN tags applied to the portgroups where you attach VMs. (Warning: not suggesting you blindly do that; but VLAN enforcement on the VMware side is fairly rigid if configured in a good way.)

    My own book is a fun read for some of these concerns, although Haletky's book is probably the canonical work on the subject. (Although it is -slightly- dated from bein