Slashdot Mirror
Ubuntu Gets Container-Friendly "Snappy" Core
judgecorp writes: Canonical just announced Ubuntu Core, which uses containers instead of packages. It's the biggest Ubuntu shakeup for 20 years, says Canonical's Mark Shuttleworth, and is based on a tiny core, which will run Docker and other container technology better, quicker and with greater security than other Linux distros. Delivered as alpha code today, it's going to become a supported product, designed to compete with both CoreOS and Red Hat Atomic, the two leading container-friendly Linux approaches. Shuttleworth says it came about because Canonical found it had solved the "cloud" problems (delivering and updating apps and keeping security) by accident — in its work on a mobile version of Ubuntu.
biggest shakeup for 20 years? how? maybe I'm getting old ... that would be 1996 right... have they really been around that long?
No dependency management or fooling around packages that require conflicting library versions, possibly near-instant "installation" (depending on if they're distributing Dockerfile-equivalents* or containers directly). Sounds good to me - I'll have to take a look sometime.
*Yes, I know that Docker is not the only way to do containers, but it's easy to imagine they could be using a similar "build" step.
And here we go again, adding yet another layer to an already wobbling stack of layers.
First we have hardware. Then we're running Xen or some other supervisor on that hardware, so we can have numerous VMs running Linux running on one physical system. Then each of these Linux VMs is in turn running VirtualBox, which in turn is running Linux, which in turn is running some container system. Then each of these containers is running some set of software. In some cases these containers are running something like the Java VM, which is, of course, another layer. Then in some truly idiotic cases, we have something like JRuby running on this JVM. There's some half-baked Ruby code running on JRuby.
Let's visualize this stack:
- Ruby code
- JRuby
- JVM
- Container
- Linux
- VirtualBox
- Linux
- Xen
- Hardware
Now that there's all this compartmentalization, it becomes a royal pain in the ass to share data between the apps running in the containers running in the VMs running on the actual hardware. So we start seeing numerous networking hacks to try and make it all into something barely usable. So throw on Apache, Varnish, and other software into the mix, too.
I'm sure that within a few years, we'll start seeing containers within containers, if that isn't already being done. Then those will need sandboxing, so there will be sandboxes for the containers that contain the containers.
Meanwhile, it's just one hack after another to intentionally get around all of this isolation, in order to do something minimally useful with this stack. The performance of the system goes swirling down the shitter as a result of all of the layers, and all of the effort needed to bypass these layers.
What a fucking mess!
I haven't used Docker before. Does this mean if I have two (or more) servers running on a JVM, that each container will have its own JAVA_HOME? If so, wouldn't that make maintenance a nightmare? Similar for python (or other language) based services? Or items running a database? Each will have it's own MySQL or PostgreSQL instead of just adding another DB to an existing server? Or do the containers sit on top of traditional mode of installing these things?
-- I ignore anonymous replies to my comments and postings.
Because you don't look to containers for security.
Soo... that's a pretty artificially sadistic case. Does anybody really run hypervisors under hypervisors commonly? Virtualbox under Xen? Really?
"You know, Hobbes, some days even my lucky rocketship underpants don't help" -- Calvin
It may not be systemD but does this mean it gets a free pass?
http://saveie6.com/
Each container would contain all of the stuff it needs to run - in this case, Java + associated modules.
It simplifies stuff, because if one server requires Foo v1.11.4 but another needs Foo v1.10.8, neither server "sees" the other. You simply configure each container separately, without worrying what the other container's doing. When distributing the container, all you have to do is send out one image. If you want to run 12 containers on a host, that's cool. If you want to run only 1, that's fine too. And that same container will work just fine whether it's running on the server or the new kid's development laptop.
It's not an all-or-nothing approach, so you can choose if you want the database to live in a container of its own, on the host, in the app container, or somewhere distant.
I'm just a casual user, not a sysadmin.
But I thought containers were kind of like VMs, not like packages.
What's the difference between a VM, a container, a chroot jail, and packages?
Auto analogies are always welcome.
General Relativity: Space-time tells matter where to go; Matter tells space-time what shape to be.
Why is everyone trying to turn Linux into Windows?
"You can update transactionally!!" Great. What does that mean? Is it like git add newapp; git commit -a? If so, how do I back out a program I installed three installations ago?
dpkg -l
dpkg -i <previous_version>
#include <cheap_shots/systemd.h>
debsums
...Did this guy just say he brought DLL Hell to Linux? Help me to understand how he didn't just say that.
No, it isn't!! What the hell is OwnCloud pulling in? What's it using as an HTTP server? As an SSL/TLS stack? Is it the one with the Heartbleed bug, the POODLE bug, or some new bug kluged in by the app vendor to add some pet feature that was rejected from upstream because it was plainly stupid?
Honestly, I'm really not getting this. It just sounds like they created a pile of tools that lets "cloud" administrators be supremely lazy. What am I missing here?
Editor, A1-AAA AmeriCaptions
No car analogies...it's a train wreck.
Broadly speeking a VM is a virtual environment that runs a separate kernel.
Containers are like a chroot jails in that they provide virtualization of the user environment for processes that execute under the parent kernel. Containers generally provide more sophisticated control over system resources (CPU, RAM, network i/o) than a simple chroot jail.
This wikipedia article provides a comparison of different types of container: http://en.wikipedia.org/wiki/O...
A VM runs its own kernel.
Containers share a kernel, but have their own namespace so e.g. they see only their own process table
chroot jails just really control permissions and environment/libs
On the one hand all of them have some pretty compelling use cases like ease of moving machines to new/backup hardware. On the other hand they all lend themselves to horrible abuses and serve to keep crummy, buggy, code in service way past when it should be flushed down the toilet with extreme prejudice, and attract a swarm of people with awful instincts to create heinously bad UIs to administer them. And as someone noted above, often require network contortions after the fact when folks realize that yes, they needed some IPC between containers.
Someone had to do it.
They're not. Windows has and makes use of dlls. They're trying to turn linux into dos.
Developers of snappy apps get much more freedom to bundle the exact versions of libraries that they want to use with their apps. It’s much easier to make a snappy package than a traditional Ubuntu package – just bundle up everything you want in one place, and ship it.
So when a library needs a security / other update, I'll possibly have to update several snappy packages that all contain the affected library? Ya, that's sooo much better Mark.
It must have been something you assimilated. . . .
Does this mean if I have two (or more) servers running on a JVM, that each container will have its own JAVA_HOME?
Isn't this already the case with large Java stuff? That despite the "run everywhere" mantra, people package it with large apps because in reality, it's not going to work?
Ezekiel 23:20
In a way they are, though. Win7 keeps every distinct version of every dll that each installed program requires, so you can quickly wind up with a folder under Windows/ filled with 20 different versions of the same library, even if most of the programs requiring them could run just as well on one of them.
Ok let's get you up to speed on containers in 7 paragraphs and there is some pottering hiding somewhere in here to keep folks interested. A VM emulates the entire hardware layer. A container depends on cgroups and namespaces support in the Linux kernel to create a lightweight isolated OS environment with network support. So you could be running a Debian host and multiple Redhat, Centos, Ubuntu, Fedora etc containers and vice versa.
The advantage is because containers are not emulating a hardware layer you get near bare-metal performance, easier access to hosts filesystem via simple bind mounts and easy workload portability across any Linux system. The limitation is you can only run Linux OS containers as it depends on the Linux kernel.
The Linux containers (LXC) project has been baking since 2009, and has been supported by Ubuntu since 2012. However its been a low profile project. A lot of hardcore Linux users have not heard about LXC let alone used it. LXC gives you system containers, ie you get a near complete Linux environment like a lightweight VM.
Docker was using LXC in its previous avatar of dotcloud to run a public PAAS platform and experimented with LXCs support for overlay filesystems like overlayfs and aufs. They then released this as Docker in 2013. Docker basically used LXC as a base to abstract the container away to an app, which is composed of layers of aufs filesystems. Compared to LXC Docker gives you a more constrained OS environment, mainly a single app execution environment made up of aufs read only layers. Running multiple apps, daemons, cron, services will requirments will require workarounds, like a bash script launching multiple apps or processes, or using a process manager like runit or supervisor. And any data produced by the container in the top most aufs read-write layer is transient and has to be commited to a layer, bind mounted to the host eg mysql databases, logs etc or lost.
The big problem occupying brainy minds lately seems to be orchestration of containers and workloads in containers across multiple hosts and this is the problem Ubuntu's recently announced LXD project based on LXC, CoreOS, Docker libswarm, Mesosphere and multiple others are trying to address.
There are also efforts to run container only OSs - ie a bare minimum OS with apps, everything in isolated containers (which is where Lennart Pottering's recent blog post about using btrfs subvolumes, low cost COW snapshots, and containerized apps seems to suggest he is taking systemd to eventually). This is where CoreOS, Redhat's Project Atomic and now Ubuntu Core seem to be moving towards.
A chroot jails is barebones compared to a container. A container can use cgroups to limit memory and cpu per container and network namespaces. LXC can now use usernamespaces support in the kernel to let users run unprivileged containers by non root users.
Think of it as running separate VMs in an hypervisor, but SOME stuff can be shared. If its all in the package, yes they'd have their own mysql or postgres, but its the same thing as if you had VMs with everything included.
Nothing stopping you from having an instance for the database, and an instance for the web server that connects to the database.
It is interesting that they pick up this name, reminds me the good old days of Fedora Core :)
So that would mean then that you would need a far larger resource footprint. Say a single server with 4 domains could get by with say (just for round figures) 5 GB of RAM. From the sounds of it I would think you would require 5+ GB of RAM using containers because each container needs a minimum footprint before you add in the resources required by the application. Same for a database. I would guess that it would be significantly more than the original (significantly more than 5 GB), but I don't think it has a slope of 1. i.e. per my quite arbitrary example, I don't think you would then need 20 GBs but still more than 6 or 7 or 8. Any thoughts?
-- I ignore anonymous replies to my comments and postings.
So I have Artifactory and two versions of Glassfish on my dev box. But only one JVM. One JAVA_HOME. I also have Maven and Netbeans IDE using the same JAVA_HOME. Why would I want 4 different JVMs installed when one works just fine? And if there is a security flaw in Java and I need to upgrade, now I would need to download 1 update as opposed to 4 times that much (plus other crap like databases and other app code, assuming they want you to just download a whole new complete container whenever you upgrade). I understand the idea of encapsulation and how it can make things neater in one sense, but seems kind of crazy in another to use containers for everything on the server.
-- I ignore anonymous replies to my comments and postings.
Docker and it's like are more than just containers. Docker is more like a format and eco system around the core LXC containers that have been around for ever.
Just speaking of the container is is more in line with chroot/jail with even more isolation.
Docker the entire ecosystem is more like Amazons AWS in that there are many prebuilt containers.
And kinda like a configuration management system (chef, puppet, cfengine) in that there is a scriptible interface for building new containers.
And kind of a continuous integration environment because you can spin up instances of code for testing with minimal resources then destroy them and spin up the next iteration, rinse repeat within minutes if not seconds.
But docker's probably biggest benfit is it's repeatably. You can run the exact same object in every environment. If designed with service discovery then you can do so with exactly zero configuration changes between dev/qa and prod. Otherwise you may need to pass some environment variables in order to establish database connections and the like.
Paying taxes to buy civilization is like paying a hooker to buy love.
Comment removed based on user account deletion
They can fiddle around all they like with this new 'Ubuntu core' with containers and other rubbish. So long as they don't start messing around with normal Ubuntu. If they did, then I would bail on Ubuntu in less than half a heartbeat. As it is, Ubuntu has adopted the crazy crap from the 'lets all make everything look good on a 2 inch smartphone screen and fuck those 30 inch 'monitors' that those olde fashioned desktop users use'. And so Gnome is stripped down to nothing, nearly unsuable, and even diehard Gnome users are looking around at different desktops including the 'melted cheese' K desktop.
Why would I want 4 different JVMs installed when one works just fine?
Why don't you ask Oracle or IBM? As far as I understand, these are the people bundling their own JVMs with whatever humongous product you buy from them, not me.
Ezekiel 23:20
When did NeXTSTEP get bundles again? Around 1989? So 25 years ago. Seem like Canonical is still chasing OS X.
So about 5 years before Debian and Slackware were released, and 25 years before major distros seriously considered it. (Puppy and several others had some custom scheme to do bundles that was serviceable enough)
Docker reminds me of Qubes in some ways. https://qubes-os.org/
The containers aren't like full-fledged VMs - they're generally running only the processes they need. If you enter it & bring up top, you're quite likely to see 3 processes in there - one for top, one for your shell, and one for your actual app.
You're going to have a slightly larger footprint, yeah - I don't believe you can take advantage of shared libraries to reduce RAM usage (though... possibly in some cases. I'm not sure) - but it's much lighter than running a VM.
http://www.infoq.com/news/2014... if you want some more gory details.
Ok, so a package is like one part of the car. Many packages make the whole car. Ie, many packages make an install of something; I'll call it an instance.
A chroot jail would be like allowing the steering wheel to be swapped to a different seat in the car. It is still running on the same basic machinery, but from a different location. Ie, you run a program with a different root directory, but all the hardware is the same.
A VM is like a ferry, a giant contraption that has built in support to run multiple cars. Ie, the VM virtualizes ALL the hardware, including the cpu, and then runs that virutalized stuff on top of the real cpu.
A container is kinda like putting steering wheels in every seat of the car. Ie, it is similar to a VM in that each instance has access to a "virtualized" set of hardware, but there is no virtual machine infrastructure that handles it all. The container doesn't use virtualization to give each instance access to hardware. Multiple virtual interfaces to the hardware are created, managed and implemented by the kernel itself (and also some hardware support as well). So, you get all the virtualization of a vm with less overhead.
And just to revisit chroot jails, keep in mind that a chroot literally does nothing more than change the location of root that program a program sees. It doesn't restrict the program in any other way. I believe you could, for example, fork bomb the host within a chroot.
Only for managed code (.Net) the old COM stuff is still in the registry and as ugly as it ever was.
There are three kinds of falsehood: the first is a 'fib,' the second is a downright lie, and the third is statistics.
Artifactory, Netbeans, Maven, Glassfish, and Java are open source or close enough for me. Are you suffering from self imposed ignorance or arrogance? Either way, you sound like an uninformed dick who is trying too hard to sound programmer hip. I thought you'd appreciate the insult since from your sig you seem to be into cocks. Whatever floats your boat.
-- I ignore anonymous replies to my comments and postings.
Good lord, even the AmigaOS back in 1985 only required users of libraries to request a minimal version, (and that libraries kept backward compatibility in future versions). If you broke compatibility, then it was for significant changes and you'd have "library2".
This shouldn't be something that still needs inventing.
Now time for Ubuntu.. Lets see it will overtake Windows or not. Well all depends upon features and visibility.
Kavita
I have absolutely no idea what either your anecdotal evidence or insults are supposed to accomplish, since neither has any relevance for rational reasoning.
Ezekiel 23:20
Relax a bit bro.
Ok, so you have several reasonably well maintained open-source (or close) things running with same JVM. Good for you. In your use-case, you only need one VM.
Now try running some legacy enterprise crap from 2003 which hasn't been touched for last 10 years on JDK8. And now imagine it uses JNI.
--Coder
What's the best RSS feed for tracking progress on this? I keep using Windows because I have this peculiar notion that I'm a "serious" gamer, but I believe if you're not a gamer, you're a moron for using Windows as your main OS.
And maybe even if you're a gamer, you're a moron for using Windows as your main OS. And, yes, I realize that means I called myself a moron. Pride cometh before a fall.
I assume there is a component to this called Snappyd.
oh, no.. the .NET code is in the GAC and is just as crufty as COM. Even their best plans soon turn into old habits at Microsoft.
(if you really want to worry, take a look at the "I have no clue which assembly is actually loaded" way .NET decides what dlls to run using Probing heuristics
...Did this guy just say he brought DLL Hell to Linux? Help me to understand how he didn't just say that.
Too late -- Ruby on Rails has already brought DLL Hell to Linux. I challenge you to install a Ruby on Rails application without having the exact version of Ruby and its dependencies that was used to develop it. This is why almost everyone uses Ruby version managers such as RVM, and you will not have much luck installing Ruby packages from your OS maintainer's package repository (and I certainly hope that none of your Ruby code is pulling in something with a security vulnerability.)
I'm guessing that this solution was built by the same crowd that did Ruby on Rails.
The second line above was supposed to be quoted (I blame Slashdot Beta...)
... is systemd. You're welcome.
It sounds like YOU have no clue what you're talking about. Classically, multiple JVMs were required for such things as the Cisco PIX / ASA web interface (doesnt work with anything newer than 1.5), HP iLO (likes 1.6 / older), Blackberry Enterprise server (tends to blow up when you attempt an upgrade of the packaged JVM), and Fiery printer interfaces.
But I guess none of that stuff counts.
PaaS too, right?
By default it searches for DLLs in the working directory. If it doesn't find there, it will search in system32/system.
MS already learned its lesson from DLL hell, now it's Linux's turn.
However, docker is a great idea. Will I have enough diskspace for my program. My floppy only holds 1.44megs?
Automatic duplicate block detection and merging should have been built into the OS stack at a very low level ages and ages ago. Stop slacking kernel dudes!
I don't know about containers. It is also why I mentioned databases and other servers. I happen to know a good deal about JVMs. So just because I used Java as PART of an example don't go all aspergers and fixate on it. If you can't answer the question about containers then shut up. I responded to another fucking goof for acting like an asshole and implying I had a problem because I used Java based tools. Having worked on several projects in the last 15 years that had budgets of close to a billion dollars each, I'm willing to bet I have as much or more experience around Java and Java EE as you or anyone else around here. I asked a question about containers using simple arbitrary parameters to see if anyone could explain how they work. Instead I got a fucking idiot slagging me for using Oracle products. If you don't like my response to that, then fuck you too.
-- I ignore anonymous replies to my comments and postings.
Then don't slag people for using Oracle products. If you can't answer a question, then shut the fuck up. I don't need to listen to assholes with nothing to say spouting some smart mouth remark instead of helping or passing on the question.
-- I ignore anonymous replies to my comments and postings.
The only two questions you asked were 1) why would you want 4 different JVMs installed when one works just fine, and 2) whether I am suffering from self imposed ignorance or arrogance. To which my respective answers were 1) you should ask IBM and Oracle, not me; and 2) no.
Ezekiel 23:20
Have you ever wanted to know how to ensure that noone wants to have discussion with you? Cause I think you cracked the code.
You're right. Here's a solid overview of the type of technologies that Docker impacts which appeared on Hacker News a couple of months back http://www.brightball.com/devo...
"Don't teach a man to fish, feed yourself. He's a grown man. Fishing's not that hard." - Ron Swanson
I think of it like running multiple DOS applications under Windows 3.1. Some VM things were happening but it was lightweight enough you didn't really have to care. Except Wolf 3D (as sole running DOS application) was slower. When Windows 95 came out we'd hold F8 and boot to DOS out of habit (and still made a proper config.sys) lol.
For containers I believe and hope that at least the disk cache is one and the same for all apps (one SMARTDRV.EXE for all processes). With cgroups containers trivially get a "RAM quota" rather than old-style fixed RAM for every VM but I bet you knew that ; at least that becomes kernel's job rather than hypervisor's job.
It feels like a container vs a VM is like using a directory vs a partition or a thread vs a process, different way to do about the same thing. And most people don't need more than multitasking and directories.
To get away from 'vm' terms, we know there can be several app's that might need mysql. So this would act like an embedded mysql server for each app instead of one. Or say like Python virtualenv only different. It seems like it might allow different versions of stuff but it also occurs to me this could get confusing after a while. I guess it's something else to learn, but I can't really see what the benefit is yet. chroot still works for good security.
-- I ignore anonymous replies to my comments and postings.
I still program and run a few servers, even though it isn't my day job any more (I used to be a C Unix programmer). I'm trying to understand the benefit vs switching to a new paradigm, to try to use the word properly. It has a silver bullet smell to me. I hope this doesn't mean that Ubuntu will only be available with containers. Otherwise I'll likely have to make the switch to BSD.
-- I ignore anonymous replies to my comments and postings.
If you switch to FreeBSD then you'll have.. jails.. which are like entirely the same thing as containers.
Anyway it's the "Ubuntu Core" edition that's new, not containers, and it sure would be entirely optional. It seems to me it's a set of tools to spawn many copy-paste server instances in a gigantic "cloud" farm depending on level of activity or need to scale up. That's trendy but totally useless if you have the more usual need of caring about "that one server".
But a container is maybe like running a process as a chrooted user and not much more if you want to keep it at that. I will liken it to running Apache as non-root and using its virtual hosts features, perhaps that's similar work and benefits. So you might find some simple and boring tool if you ignore the hype and the various competing management layers. I seem to understand you can manage processes with cgroups to get the I/O, CPU, memory limits whether or not you use containers, too.
chroot I understand. :) I also did a lot of work recently with AWS. Dozens of servers. I wrote an app that when I discovered it worked almost like chef to update the servers... but that is another story. So thinking of it this way, I can perhaps see a benefit. But not so certain with tradional one physical limited with few if any VMs (traditional OS installation). We used to have configuration file generators (Perl substituting tokens in a template) for spawning new domains/servers on Web Logic instances to create test environments. So I understand the usecase. Containers seem less flexible in terms of do it our way or no way. Guess I'll have to try it to see. Kind of the same limitations as most frameworks.