Slashdot Mirror
The Ideal, Non-Proprietary Cloud
jg21 writes "As previously discussed on Slashdot, the new tendency to speak of 'The Cloud' or 'Cloud Computing' often seems to generate more heat than light, but one familiar industry fault line is becoming clear — those who believe clouds can be proprietary vs. those who believe they should be free. One CEO who sides with open clouds in order that companies can pick and choose from vendors depending on precisely what they need has written a detailed article in which he outlines how, in his opinion, Platform-as-a-Service should work. He identifies nine features of 'an ideal PaaS cloud' including the requirement that 'Developers should be able to interact with the cloud computer, to do business with it, without having to get on the phone with a sales person, or submit a help ticket.' [From the article: 'I think this means that cloud computing companies will, just like banks, begin more and more to "loan" each other infrastructure to handle our own peaks and valleys, But in order for this to happen we'd need the next requirement.']"
Am I missing something, or does the article make no mention of security?
... That cloud computing silver lining has started to tarnish already?
Ctrl-Z
What makes him so sure that interoperability will be even on the provider's list? I don't see any easy way to use EC2 with some third party solution for storage. Plus, it would be lame if I had to go via internet for every request that should ideally be local.
Microsoft: "You've got questions. We've got dancing paperclips."
Sounds like he has his head in the clouds.
http://cafepress.com/spankymm - for the Masturbating Monkey in you!
I've looked at clouds from both sides now,
From up and down, and still somehow,
It's cloud illusions I recall,
I really don't know clouds, at all.
No folly is more costly than the folly of intolerant idealism. - Winston Churchill
The guys at Red Hat have released the first version of a project called Genome genome.et.redhat.com . This looks to be an open source project that makes Fedora, Red Hat Enterprise Linux, and CentOS clouds using Xen, KVM, and commodity hardware.
Relying on third party technology is never going to provide the reliability or uptime required. The more straight forward solution is to hire some rackspace and host your own solution. 'Cloud Computing' is just the latest marketing promotion designed to move us to renting software.
davecb5620@gmail.com
The word "proprietary" is a very vague term that's usually used to connote some sort of "them", where the "us" are the good guys.
The bottom line is that wherever there is value, someone will find a way to charge for it. If this "cloud computing" really has no model under which anyone finds it valuable enough to commercialize it, then it's probably not going to be very popular anyway.
E pluribus unum
Proprietary buzzwords, what will they think of next? What will the next dynamic paradigm shift be?
In seriousness, with all of the glaring security issues and discomforts that people have with sharing their private information over a network, how will this idea ever seriously take off? Will the average home user ever consider such an idea? Personally, though it may be "inconvenient", I feel more secure having my data stored locally than working with it over a cloud.
There are mountains to cross for those that are willing.
so we'll end up with a sub-prime computing crisis?
how can you bail out companies that fail to keep sufficient computing reserves in hand to cover their potential obligations?
"If still these truths be held to be
Self evident."
-Edna St. Vincent Millay
Pretty much any technology leads to both open implementations and proprietary implementations. The central question is whether the STANDARDS for interacting with those implementations are open or proprietary. Maybe you deploy Java to a proprietary WebLogic server, or an open JBoss server... but you're dropping basically the same EAR or JAR file in either case. THAT'S one of the key factors determining whether a technology will catch on.
Before you can start developing the proprietary or open implementations, you have to develop the standards. Before you can develop the standards, you have to figure out what it is your developing in the first place. That is, can anyone actually define what this nonsense sales-and-managerspeak "cloud" buzzword even means? It seems like whenever Gartner and Gang tries to shill a new technology that doesn't widely catch on, they just wait a few years and try again with a different buzzword. Ten years ago this was called a "cluster", and few people cared. Five years ago this was called a "grid", and few people cared. Today it's called a "cloud", and I... well, you know.
I believe clouds should be free, and I'd like to do business with clouds.. not Storm clouds, however.
Today's forecaset: cloudy. This afternoon, continued cloudy with occasional periods of distributed computing.
Tonight: Dark, with periods of light toward morning.
Tomorrow: Ignorant, with occasional words coined by the ignorant used by the knowledgable. May be occasional clouds in the afternoon. In case of tornado, stay in your basement.
mcgrew's razor: Never attribute to stupidity that which can be explained by greedy self-interest
http://www.softpanorama.org/Skeptics/IT_skeptic/it_obscurantism.shtml
So the discussion comes down to: proprietary vaporware, or free vaporware.
When discussing vaporware, it seems to me that any matters beyond that point are superfluous.
In this day and age - when hardware is essentially worthless [today, for under $200, you can get what would have been a $10 million supercomputer ten years ago], and when even RDBs are essentially worthless [MySQL & PostgreSQL being free downloads], the only things which add value are:
.
Of those, at least 1), 3), and 4) are going to have to be uploaded to "The Cloud" [and 2) might have to at least interact with "The Cloud"], and unless "The Cloud" encrypts everything - both data & logic [and how do you really "encrypt" something if ultimately the registers in the CPU have to see unencrypted data, and especially unencrypted logic & algorithms?] - then you've just uploaded the crown jewels of your entire enterprise for all the world to see.
I cannot wait until Web 3.5 gets here and we can tag articles with sound clips.
So many things I would have done
But clouds got in my way
Some background:
For my employer, I've made an application, under perl and postfix, that runs an email forwarding application. The part the user interacts with mostly is a database server on a webcluster, but the smtp side is handled by (at the moment) 8 machines.
This wouldn't be so bad, but they're getting a little flooded. If I could run the software in the cloud, it could grow and shrink dynamically, which would be great.
In practice, all this would mean is that it could mark and discard spam faster, so the very small amount of legitimate email could be moved faster, rather than the several hours it currently takes to get to the front of the queue.
... and today's pet project has
Every buzzword soaked trade publication on the planet has Cloud on the cover now. When looking for a job, I'm going to put my name and contact info on my resume. Then, in place of the usual job history and qualifications I will put, in the largest font that fits, one word: CLOUD. My pay will go up 25%. Then, in 6 months, people will be saying "remember cloud computing?".
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
And in this day and age, when even medium-sized businesses can be sitting on literally terabytes of data, how are you going to upload all of that data to "The Cloud" so that "The Cloud" can analyze it for you?
Maintaining a constant 10Mbps WAN connection to "The Cloud" would be monstrously expensive, and yet, at 10Mbps = (10 / 8)MBps = 1.25MBps, that means you would need
.
just to upload a terabyte of data at WAN speeds of 10Mbps.
So "The Cloud" isn't going to have realtime interactions with your corporate database - "The Cloud" is going to BE your corporate database.
The Cloud is a made up phrase to market proprietary "solutions". Therefore the idea of a non-proprietary cloud is bull.
I am all for non-proprietary web based applications. But you can stick this Cloud into those marketing guys' proverbials.
The whole issue is a little cloudy to me.
Know your pads. One time pad: good for cryptography. Two timing pad: where to take your mistress.
Ok, I hate buzzwords as much as the next person not wearing a pale blue shirt...
But I'd like to suggest "cloudware" as a potential interchangeable word for "vapourware".
For obvious reasons...
I'll wait for Cloud 2.0
Facebook is the new AOL
"Even if the third party has way more experience and better hardware than you do?"
.. :)
I've worked for some of the 'premier' ISPs and major multinationals, one being a consultancy to the business sector. I've seen better IT infrastructure in the average tech college. As for the expertise of the consultancy, as far as I could make out it conssted of a VB macro to create unique file names for the reports, written as PPT files. Oh yea, the only other 'innovation' was splitting the research department up into teams and refering to them as ' pods '
davecb5620@gmail.com
"hiring rackspace is relying on third party technology"
..
Just for the box, a USB and an Internet connection. If you don't own your own technology, you're not a real business
davecb5620@gmail.com
"You can't get the same scaling from a physical server as you can get from "the cloud" for anywhere near the same price"
Most people don't need such scaling and I can get more per price from a box hosted in a server farm. The reason "the cloud" would be cheaper is they build and staff it at the lowest possible cost. Things happen like forgetting to test the emergency generators, or what probably really happened, skimping on routine maintenence.
davecb5620@gmail.com
abandoned after ec2 came out. EC2 does most of what it sounds like you want... except for the redundancy in providers. Personally, I'm working on building a better provisioning system for my own VPS services at http://prgmr.com/xen but the idea is that it's not that hard, even with the way ec2 is now, to take your ec2 image and run it on another xen host, or take a xen image and run it on ec2. (now getting a 'public' image off amazon ec2 and downloading it, that's hard. but if it's your image, downloading it is trivial. and amazon has all the tools you need to turn it into a plain file or tarball that any Xen provider can use)
I think having all your servers with one provider (even if you are that provider) is usually a bad idea. People make mistakes. hardware dies. natural disasters happen.
Skimming through the comments here, they seems to break down into several categories:
I don't see any posts talking about
here is what we used cloud computing for and here are the problems with the current platforms.
This tells me that whatever this technology is, it is still early and people are still testing the water. If we want some kind of standard or open implementation of clouds, we are going to need much more people using it to explore what is good and bad about the model.
The beginning of a new technology should be about trying to find the limits of it. It is stupid to worry about open or proprietary before we even know if people will want it. The proprietary people are getting first crack at it because they think it will make them money. So let them find out what the issues are and if they can make money.
After they do your R&D for you, then you can make an open and free version. That's been the model of most successful open source projects.
AlwaysOnGames Arcade
When a concept is so new that people can't even define it, now is not the time to be trying to develop an "open standard". Now is the time to be rapidly prototyping different ideas.
When we have several, stable clouds, then it might be time to talk about interoperability, or at least a compatibility layer.
Don't thank God, thank a doctor!
#1... one key feature of the dedicated model for web applications is a stable, static IP address.
No it isn't. The key feature is a stable, reliable way to connect to your apps, wherever they are -- when I type example.com, I should be routed to the right place.
This means a built-in hardware load balancer, dynamic DNS, or anything in between.
Amazon's Elastic IP, for example, can take 15 minutes to switch between instances -- something like 10-12 minutes during which requests are sent to the old instance, then 2-3 minutes during which all traffic is dropped on the floor and no instance is reachable, and then, finally, traffic is routed properly to the new instance.
The only advantage is that if you don't have to do this often, you aren't paying for a hit to your DNS servers every few minutes, just in case.
#2... The API needs to be publicly available to everyone. Provide a credit card (that works and is yours) and you should get compute, storage, and RAM on-demand.
This is important -- not only are we moving beyond "submitting a ticket", but it has to be programmatic, not just point-and-click from a web interface.
For example, one could auto-scale on demand -- kill a few instances for the night, as no one's up at 3 AM, then bring them back up when you get traffic. Or bring up an extra few dozen to gracefully handle a Slashdotting.
But there's no one-size-fits-all solution. Autoscaling is nice, but consider that we might know exactly when we need power, and when we don't. We might deliberately spawn a ton of instances to handle a large batch encoding (like YouTube should have, when switching to high-def h.264), or deliberately turn off an instance when we don't need it (only really need a staging server and SVN/Trac during the day).
#3... I propose that cloud computers should support PHP, Ruby, Python, Java and the most common frameworks, libraries, gems/plugins, and application/web servers for each of these languages.
Absolutely not. That is the wrong way to go about this.
Specifically: What happens when I want to deploy an Erlang app, which uses a Smalltalk VM for a database, and forks off ffmpeg commands to do encoding? What if I like Ruby just fine, but I want to deploy, say, Sinatra + Sequel + sqlite?
I really like what Google has done with Python. All the low-level details are handled for you -- just throw a Python app up on App Engines and you're good to go. Probably not even OS-level virtualization -- could even be a language-level jail.
Which is all really great -- if you're running Python. And it's completely useless if you're not.
Compare this to Amazon -- they'll run just about anything that will work on x86/x86_64 Linux. It's up to the individual languages/frameworks to target Amazon.
So, to answer:
Essentially, a developer should be able to move between Joyent, the Amazon Web Services, Google, Mosso, Slicehost, GoGrid, etc. by simply pointing the "deploy gun" at the cloud (having used the API mentioned above to spin up instances) and go.
This would be better accomplished by extending an existing tool like Rubber to support multiple clouds than to demand that every cloud hook into every possible framework.
After all, once we know enough about the requirements of the cloud model to actually draw up a list like this, and start working on standards, it shouldn't be too difficult to create a standard which allows Rubber (and friends, like PoolParty, Scalr, etc) to be portable without having to rewrite themselves for each cloud.
#4... Amazon is innovating with SimpleDB, Google has BigTable as solutions for the problem, but developers can't leave either cloud because neither SimpleDB nor BigTable are available anywhere else.
Again, far, far too early to create a standard.
However, we h
Don't thank God, thank a doctor!
I always thought that all those Xbox 360s and PS3s would make a great cloud when they are not being used. One could for example trade idle time for Xbox Live points to make it worthwhile. I would think Sony could cut a similar deal and render their movies on all of the idle PS3s.
Disclaimer, I'm the chief architect of a cloud vendor.
I'd say the cloud buzz started when Google's Eric Schmidt started saying that they were in the "cloud computing business" circa 2006, and with the release of Nick Carr's "The Big Switch" book in January 2008.
Here's the question: "Why is my enterprise IT so expensive, not innovative, and hard to use and my online services so affordable, innovative, and easy to use?"
The answer could be one of three things:
1. maybe the online vendors do things differently and better in some ways, let's learn from them and change IT
2. online vendors use commodity hardware in clusters, so we all should move to that too
3. maybe online vendors know more about IT than we do, let them do the dirty work.
4 they're not, go back to sleep
So, there's a few ways to look at "cloud":
- A new way to do IT management & provisioning on demand and at scale, regardless of who hosts it (e.g. Puppet + Virtualization + a Web API)
- A large cluster of computers instead of a few large computers (e.g. Hadoop)
- Outsourced infrastructure with an API for manipulating it and a "retail channel" to pay for it with credit cards (e.g. Google App Engine, Amazon EC2)
- It's a fog. Nothing to see here, move along.
Now, my opinion:
Is a cloud always a cluster? Often, yes, but not always. Sometimes it might be several independent clusters working together (e.g. database vs. web vs. integration). Sometimes one large box makes sense (e.g. for certain database apps).
Are these buzz words to rent software? Perhaps. Much enterprise software is sold and bound to physical CPU's for $10,000 to $60,000 each. That doesn't really work well when you're virtualizing stuff and growing & shrinking it regularly.
Will everyone want to outsource? Hell, no. It can be a shell game. Security is a problem (though I'll note, we do store money in a bank, not under our bed in a mattress, these days, so our computing may make some sense). Privacy is even more of a problem.
Having said that, is it not cool that you can programmatically create a small army of Linux nodes for 10 cents an hour each via Amazon EC2? Won't it be cool to be able to broaden this idea (i.e. being able to manage large numbers of services & apps on there via an API or command line or even GUI?)
Is this proprietary? In a way, yes, in a way no. Amazon EC2 is proprietary, but there already is the open source Eucalyptus project that emulates it to be able to use the EC2 API to provision Xen containers on Rocks Clusters. There are EC2 API libraries that are mostly open source. Then there's Puppet -- a couple of years old and is a great GPL project for infrastructure automation (i.e the new CFEngine) which fits into what the cloud is trying to do. Zenoss is a few years old and is a great management platform that some cloud vendors hook into. Elastra is working to open source tools for its ECML and EDML languages to build a cloud package / design / platform ecosystem.
Are online services cheaper and easier to use than the enterprise? In some cases, certainly. Some internal IT departments require a $10k+ tax on top of server purchases to cover IT installation and provisioning costs, and then take 2 weeks to 2 months to bring the server online. Using an API to bring up a server up , auto-configured as part of a MySQL cluster, in under 30 seconds, is quite a relief.
-Stu
Amazon's Elastic IP, for example, can take 15 minutes to switch between instances -- something like 10-12 minutes during which requests are sent to the old instance, then 2-3 minutes during which all traffic is dropped on the floor and no instance is reachable, and then, finally, traffic is routed properly to the new instance.
Have you experimented at all with round-robin DNS records pointing to two different elastic IPs in different availability zones? If one availability zone goes tits-up, does round-robin yield acceptable performance, routing clients to the good zone?
They don't grade fathers, but if your daughter's a stripper, you fucked up. --Chris Rock
The NY Times converted 4 terabytes / 11 million TIFF based images & articles from their archives in 24 hours using 100 EC2 instances. And continue to do it to this day. Cost? A couple hundred dollars.
-Stu