UC Berkeley Lab Examines Cloud Computing Obstacles

alphadogg writes "UC Berkeley researchers have outlined their view of cloud computing, which they say has great opportunity to exploit unprecedented IT resources if vendors can overcome a litany of obstacles. 'We argue that the construction and operation of extremely large-scale, commodity-computer data centers at low-cost locations was the key necessary enabler of Cloud Computing,' The paper outlines 10 obstacles to cloud computing [PDF]."

Comments

Whoo?

[Anonymous Coward]

Wasted on an Anon.

Vaporware Alert

[Spazztastic]

Do you trust your data being up in the "cloud"? Do you want to risk that company tanking and your work going away? I don't. I can work fine over a IPSec link to my storage server (with more redundancy) and run subversion to keep track of my work.

Re:Vaporware Alert

[klock]

What happens if it rains?

Re:Vaporware Alert

[Spazztastic]

What happens if it rains?

All of your data comes down in binary and you have to reassemble it using pen and paper.

Re:Vaporware Alert

[elashish14]

Well who knows what the security will be like? As it is, end users hardly ever update their system, so it's best to make the operating system as minimal as possible. The cloud will likely be far better maintained for that purpose.

.

Then again, I certainly wouldn't store everything on it. Unimportant files and data sure, but not things with sensitive information. People will be idiots anyways, their identities are stolen all the time, so what difference does it make?

Re:Vaporware Alert

[ocularDeathRay]

the indisputable fact that cloud computing is buzz-word bull-crap, didn't make the top ten list. I find that strange...

Re:Vaporware Alert

[postbigbang]

Even 'the cloud' needs availability procedures, backups, and though provided to the entire process.

But you didn't RTFA, did you?

Developers are one community supported in the paper, but it mostly speaks to SaaS, including observations of Ellison to Stallman.

Re:Vaporware Alert

[morgan_greywolf]

Do you trust your data being up in the "cloud"?

Why not? Is it any worse than trusting your data being in the datacenter? As long as you have good backups and a good disaster recovery plan, who cares where your data sits? As long as the company whose servers the data is sitting on have appropriate SLAs that say how the data and access to the data will be protected and to what extent, how, etc., is that any different from the SLA you have with your IT department?

Re:Vaporware Alert

[jellomizer]

Do you trust yourself to handle the company information and backups and data storage... If so is that you full time or near full time job.

Secondly if you company is going to go to cloud computer you should be smart enough to create part of the contract the ability to get your data at request even after service has ended for a period of time.

Oddly enough a piece of paper with a couple of signatures on it works better then any technical solution.

Your argument is the same as people who are afraid to fly. The fact that your fate is in someone else hands even if it is statically safer, you feel more comfortable when you are in control.

Re:Vaporware Alert

[SanityInAnarchy]

Do you want to risk that company tanking and your work going away?

Google is not likely to tank. Nor is Amazon. Nor is Rackspace, for that matter.

They are certainly not likely to tank without providing at least some warning and access to your data.

I can work fine over a IPSec link to my storage server (with more redundancy)

How is that more redundancy, exactly? Do you have your work saved across multiple machines in multiple datacenters, each with multiple generator backup, multiple backbone connections, etc?

No, you have a "server" -- implying one server. When it goes down, you're fucked.

and run subversion to keep track of my work.

And that just lost you all credibility. Were you running a DVCS, you might have a point -- I feel pretty confident about most of my data in Git, because whenever I deploy (to "the cloud", as you put it), there's a full copy of that Git repository on each server I deploy to, plus one on my laptop.

But with Subversion, when that one server goes down, you're really fucked.

Unless, of course, you're making backups, in which case your data is probably safe, but you still need to build a new server. But how is that different than me making database dumps from an EC2 instance to S3, or to my local machine?

Re:Vaporware Alert

[darkuncle]

Do you trust your data being up in the "cloud"? Do you want to risk that company tanking and your work going away? I don't. I can work fine over a IPSec link to my storage server (with more redundancy) and run subversion to keep track of my work.

if you are trusting your data to only one provider, whether it's yourself or some "cloud computing" vendor, you're doing it wrong.

Redundancy in services/vendors and backups/mirroring is just good practice, and applies whether you're computing circa 1989 or 2009. If you're using multiple vendors, the downside risk of any single one of them going AWOL for some reason is significantly mitigated.

(that said, there's always a SPOF somewhere with the current hierarchical DNS system, but even that can be made fairly robust without breaking the bank.)

Re:Vaporware Alert

[Chris Mattern]

Do you want to risk that company tanking and your work going away? I don't.

It's a serious risk. So serious that TFA listed it first in their list of 10 obstacles to cloud computing. Their suggested solution: common APIs and interfaces so that you can actually do your cloud computing on two providers with each providing full failover capability.

Re:Vaporware Alert

[psetzer]

Boy I bet they feel dumb for ignoring those issues and not making them the first three obstacles they address.

Re:Vaporware Alert

[c00p3r]

be sure, you're using compressed scp with dsa-keys auth over that ipsec link. and of course, all files must be encrypted with 2048-bit keys.

Hmm.

[fuzzyfuzzyfungus]

Where is Aristophanes when you need him?

Re:Hmm.

[Anonymous Coward]

Probably out talking to his frogs.

RTFA

[djupedal]

From the article's 'Executive Summary':

"Moreover, companies with large batch-oriented tasks can get results as quickly as their programs can scale, since using 1000 servers for one hour costs no more than using one server for 1000 hours."

Is that a given? Is it that simple? I think that statement assumes too much...

As an example, it assumes all servers are operating on the same grid, in the same environment. Last time I checked, the cloud was the epitome of vagary.

Re:RTFA

[Anonymous Coward]

And you my friend, are the epitome of spellingtonization.

Re:RTFA

[idontgno]

Is that a given? Is it that simple? I think that statement assumes too much...

Nonsense. The Cloud has spoken. Blessed be the Puffies of the Cloud.

Last time I checked, the cloud was the epitome of vagary.

Which means you have no evidence to cast doubts and aspersions. Silence, Infidel! The Cloud has spoken! Blessed be!

OK, that was over the top.

Really, this is just another cycle of "$TECHNOLOGY_I_AM_SELLING is the GAME CHANGER that will SAVE THE WORLD!!!!!111one" I've been in IT 25 years. This repetitiveness is just boring now.

Re:RTFA

[brian_tanner]

"Moreover, companies with large batch-oriented tasks can get results as quickly as their programs can scale, since using 1000 servers for one hour costs no more than using one server for 1000 hours." Is that a given? Is it that simple? I think that statement assumes too much... As an example, it assumes all servers are operating on the same grid, in the same environment. Last time I checked, the cloud was the epitome of vagary.

Nonsense! I'm starting to get tired of all this "last time I checked" bullshit. You NEVER checked. My aggression is not leveled at you necessarily, but at all of the Slashdot posters who bring up the same arguments every time there is a cloud computing story.

The truth is, Amazon's offering (for EXAMPLE), lets you start up multiple virtual machine instances, and you pay per minute per instance. Lots of (NOT ALL) academic and industrial research relies on running many independent experiments. Let's try algorithm X with parameter set [1...100]. If each parameter set takes an hour, then you can either invest 100 hours on 1 machine, or 1 hour on a 100 machines. Aside from a tiny bit of overhead to create scripts to start the virtual instances and upload the jobs, this works exactly as advertised. It is great for some of us, and it's that simple.

Re:RTFA

[SanityInAnarchy]

As an example, it assumes all servers are operating on the same grid, in the same environment.

I suspect they are thinking about things like EC2, which has a pricing model exactly like that -- per hour per server, no matter how many of each.

Re:RTFA

[djupedal]

The 'cloud' is a metaphor for the Internet - not for a controlled service from Google or Amazon. 'cloud computing' is defined as software-as-a-service over the internet which means a major factor outside a controlled environ.

Berkeley, in the exec. summary says "The datacenter hardware and software is what we will call a Cloud."

Thus 'cloud' has been hijacked and redefined for purposes of... I prefer to stick to the original definition and not ignore it. Of course, by their definition, their model and their proposal fits their conclusions nicely.

Re:RTFA

[Anonymous Coward]

9 women can't have a baby in 1 month. Not every problem works that way.

/me fears . . .

[SebaSOFT]

Would this be the beginning of SKYnet?

top-10 list: Anonymous so I'm not Karma whoring

[Anonymous Coward]

Table 1: Quick Preview of Top 10 Obstacles to and Opportunities for Growth of Cloud Computing.
Obstacle Opportunity
1 Availability of Service Use Multiple Cloud Providers; Use Elasticity to Prevent DDOS
2 Data Lock-In Standardize APIs; Compatible SW to enable Surge Computing
3 Data Confidentiality and Auditability Deploy Encryption, VLANs, Firewalls; Geographical Data Storage
4 Data Transfer Bottlenecks FedExing Disks; Data Backup/Archival; Higher BW Switches
5 Performance Unpredictability Improved VM Support; Flash Memory; Gang Schedule VMs
6 Scalable Storage Invent Scalable Store
7 Bugs in Large Distributed Systems Invent Debugger that relies on Distributed VMs
8 Scaling Quickly Invent Auto-Scaler that relies on ML; Snapshots for Conservation
9 Reputation Fate Sharing Offer reputation-guarding services like those for email
10 Software Licensing Pay-for-use licenses; Bulk use sales

ironic: my captcha is "retyping"

Re:top-10 list: Anonymous so I'm not Karma whoring

[tcopeland]

> 1 Availability of Service Use Multiple Cloud Providers; Use Elasticity to Prevent DDOS

They should add another one:

11 Use Hard Limits to Prevent Bank Account from Being Drained During Prolonged Slashdotting

Cloud Computing

[Presto Vivace]

does not require users to give up their data stores or business logic. If they hire a systems integrator to write their application, they can own it rather than license it.

What Vaporware?

[TubeSteak]

Cloud Computing is far from vaporware. Or are you suggesting Google & Amazon are fly-by-night operations that might nuke your data on their way out the door?

As long as you're willing to accept certain compromises, Cloud Computing can be the best thing ever.

Re:What Vaporware?

[Marxist Hacker 42]

Or are you suggesting Google & Amazon are fly-by-night operations that might nuke your data on their way out the door?
 
I'd suggest that Google & Amazon are fly-by-night capitalist organizations who will nuke your data with the first late payment on a bill.

Re:What Vaporware?

[digitalchinky]

You purchased the kool-aid clearly enough.

Cloud Computing = Server(s) <--> Client(s) This concept has been around for a lot of years now, you're just picking up on the new buzzword for it.

Re:What Vaporware?

[Anonymous Coward]

Oh you mean like not having to bother hacking someone's home machine? That is the only advantage to having my desktop broadcast to me from a virtual server. Then you only need my account number and ID and you can take everything out of my virtual partition without any risk of my finding out (that is until you sell and print my book, or take the device I've designed to production...yeah good luck to me in trying to establish ownership of the design after you have already made and sold a million units!!!!).

What if it's your own 'cloud'?

[Gazzonyx]

What if you run your own cloud? You'd surely trust your datas safety, then.

Along those lines...
At what point does clustered storage (RHEL has had this since mid RHEL-4) become a cloud? At Amazon's level, Google's, a single rack of RAID boxes, colo'ed RAID boxes? I'm not even sure that the idea of cloud has an exact requirement that makes it a 'cloud'.

There should be an RFC for this kind of thing. OK, I'll get off your cloud, now.

Re:What if it's your own 'cloud'?

[Tolkien]

What if you run your own cloud? You'd surely trust your datas safety, then.

OMG I think you're on to something! What if we ALL owned our own clouds? We could then host our own content on our own clouds safely! We could even connect all our clouds together in the form of a network, or "Web" if you will. That would allow us to share all our public information together whilst keeping our private data separated!

Think of the possibilities!

Note: I thought of and typed this post before reading the rest of yours :)

Only a few considerations

[ErichTheRed]

All the analysis in the world on cloud computing boils down to a simple fact -- someone else owns your infrastructure and data. If you want to go down this road, your company has to answer these questions:
- Are we comfortable with letting someone else have our data, if they promise not to let it get stolen or use it themselves? Do we really trust that promise?
- Contracts and SLAs are all important, but will getting a payment or free service from a vendor for a 5-hour outage make up for all the lost business? If not, how big does that payout need to be?
- Is the vendor really competent enough to handle the service you're outsourcing to them? Vendors have been known to hire the lowest-possible-cost staff to manage things like this...
- How easy is it to get your data back if you want to leave? Are you stuck with the vendor forever?
- If any sort of app deployment is involved, is your dev, QA and engineering organization good enough so that rolling out to production isn't a messy "oops, let's fix that by manually tweaking the system while it's running" scenario? Vendors generally don't let you do that.

I think the concept really works well for commodity stuff like mail hosting. Whether you trust core business apps to the cloud really depends on your comfort level!

Re:Only a few considerations

[bacon volcano]

I certainly agree with you when it comes to small and midsize companies.

However, where I really see a potential advantage to cloud computing is very large companies that operate multiple data centers for their own business use. If these companies were able to start using cloud computing for their internal needs, I could see a huge potential for hardware, software, and energy savings. I have seen large companies run tons of servers, all far under their capacity out of a reluctance to run multiple applications on a single box. Virtualization helps with this a lot, yes, but operating with all of the company's computing load balanced across an entire data center would certainly improve the overall utilization of resources.

I think the larger companies would certainly have more trouble getting their infrastructure to a point where this idea would be feasible though.

They forgot operational transparency

[psydeshow]

The list:

1 Availability of Service
2 Data Lock-In
3 Data Conïdentiality and Auditability
4 Data Transfer Bottlenecks
5 Performance Unpredictability
6 Scalable Storage
7 Bugs in Large Distributed Systems
8 Scaling Quickly
9 Reputation Fate Sharing
10 Software Licensing

I'm surprised they don't mention my biggest pet peeve with cloud services: lack of operational transparency. You don't know who the admins are, what their policies are, and what code they are using to operate the system.

It's a big black box and you're just supposed to trust that Amazon (or whoever) has sound policies, peer-reviewed code, and a reasonable level of accountability built-in. That's a bit like trusting your bank to only make good loans.

I actually want to know who the admins are. I want to see the code. I want to read the policies. Is that so wrong?

Re:They forgot operational transparency

[DerekLyons]

That's a bit like trusting your bank to only make good loans.

Nobody with any sense expects their bank to only make good loans - as doing so requires a fully functional crystal to determine who'll lose their job, what company will suffer a massive setback, etc.. etc.. over the life of the loan. What you do reasonably expect is for the proportion of bad loans to be minor in relation to the proportion of good loans.

Re:They forgot operational transparency

[Standard User 79]

>> Is that so wrong?

Yeah it is. A cloud service requires more trust in one provider but offers far more efficiency than requiring trust in multiple providers (e.g. my colo, cisco, netapp,redhat,etc..) A successful business has to be able to balance mitigating risk with trusting providers.

So.. while you are interviewing admins and auditing code everyone else is busy making money. Guess who is still in business in 5 years?

It seems...

[msuarezalvarez]

It seems we are soon going to need access to a big cloud of mainframes in order to read Slashdot... I have no idea what it is doing, but its Javascript freezes Firefox way too often...

when engineers write

[Sebastopol]

which they say has great opportunity to exploit unprecedented IT resources if vendors can overcome a litany of obstacles.

if the resources have never been seen, how do we know they are even there...? :-)

Group Blog and Video Discussion

[marmbrus]

We will be responding to feedback and continuing the discussion at our blog: http://abovetheclouds.cs.berkeley.edu/

Okay, it's a buzzword

[CopaceticOpus]

Every time a story mentions the "cloud", we get to enjoy many complaints about the use of this buzzword. I think it's time to accept it. It is a useful term, describing a trend which is only going to grow over time. Having a concise way to refer to it will be convenient, regardless of what you think of the trend.

When people started talking about the "web", did you complain about this buzzword too? After all, it's nothing but a system of interlinked hypertext documents accessed via the Internet. Why not just say that every time?

Re:Okay, it's a buzzword

[Anonymous Coward]

Thank You!

Re:Okay, it's a buzzword

[geekoid]

but it's not.

Cloud still has several interpretation.

Sometimes it's data, sometimes it's cpu cycles, sometimes user connections.

Short of "Some IT stuff done someplace else."

Outsourcing the hardware isn't the benefit

[lord_sarpedon]

Outsourcing the hardware isn't the benefit. Amazon and friends have that end of the game.

Within some large organization - university, corp, whatever - there are typically a huge number of workstations with lots of redundant hardware that is usually sitting idle, lots of departments with varying computing needs, and some ever changing number of servers, some crusty, some doing lebenty-jillion different jobs...

It's very handy for various departments to be able to provision servers as they need. Some pool of terminal servers can be maintained, serving VPN users as well as thin clients. Physical servers can be brought up and down as needed without even dropping net connections. Maintenance doesn't mean downtime. You're paying for bandwidth and hardware just like you always did, but its a commodity for the people that actually need to use it.

I love watching people reinvent the Mainframe!

[DutchUncle]

Once upon a time, when I started working with computers, you had a very dumb terminal connected to a remote computer. For lots of years the objective was to bring more intelligence closer to the user; editing terminals led to local storage. At the point where we could have individual computers all to ourselves, anyone acknowledging the splintering effect - the fact that you couldn't add back the cycles and memory to make one big computer - was mocked as a throwback to the outdated mainframe days. Same for anyone pointing out the problems with backup and maintenance. After all, why would you want to leave your work under the thumb of the people in the glass house when you can have your very own personal computer?

Since then we've been trying to find the best way to split the load between local and remote intelligence, distributing processing across communications, whether it's for business applications or multiplayer games. And most important, we have found that one solution does not fit all problems. Sometimes distributed knowledge addresses the problem or enables a brand new activity (like the multiplayer games); sometimes close direct access offers the most speed or best function (like Google's massive data centers).

So now, after we can put multi-gigahertz multi-core processors and gigabytes of RAM and terabytes of storage at the disposal of each individual, people are reinventing the remote mainframe. Call it a server, call it a cloud, whatever.

It is to laugh.

Those who will not learn from history are doomed to rediscover lots of things at their own cost.

Re:I love watching people reinvent the Mainframe!

[Anonymous Coward]

OK! OK! We'll get off your lawn!

Re:I love watching people reinvent the Mainframe!

[cL0h]

You seem to be operating from a very narrow definition of "the cloud".
The cloud is not a central location, it is distributed.
Service Oriented Architectures allow an application developer to cherry pick services available from disparate distributed locations knowing that the interfaces for each will be well defined (using WSDL or the like) and the transport mechanism ( e.g. SOAP over HTTP) will play nicely and not require custom translation of inputs and outputs.
There are plenty of good examples of SOA or "the cloud" other than some naff web site which badly replicates something I can do locally. An example I recently coded is an enterprise wide token based authentication which allows field engineers access to our remote equipment via a web service. So
- engineer - contacts "cloud" (i.e an RBAC webserver), gets token, attempts handshake
- kiosk - sends token to cloud gets credentials, allows access.
"Mainframe" not necessary.

Its a bad word and we wont use it

[Anonymous Coward]

forget it.
I don't like the word.
People who use it are marketting and
legal and not engineering or IT.

Tornados and Lightening come from clouds.
They picked a bad word.
Plus: there is nothing new. It doesn't mean anything. The 'web' is the network. The 'cloud' is
the lame marketting by weakminded corporate money-coin people.

Re:Its a bad word and we wont use it

[adpowers]

When you draw the internet while diagramming on a whiteboard, what do you draw? Most people draw a cloud as an abstraction for the stuff "out there". I've never seen anyone draw a spiderweb when drawing a diagram that includes the internet.

I've tried to explain the cloud to slashdotters before. If you don't like the word cloud, you don't have to call it that. A less buzzword-y and perhaps more accurate term would be "utility computing". Turn on the faucet and out comes your data.

It's good for some things, lousy for others

[zullnero]

I can see cloud computing encompassing a particular subset of applications that thrive on web space, things that absolutely require constant connectivity to function up to their intended spec, but the concept just doesn't fit a whole hell of a lot of applications.

Client/server stuff isn't going away. Thankfully. The more you have to do everything through the browser, the more you realize that you just can't make your application as seamless and clean as you can when you can configure your own client to do whatever custom thing you want it to do.

VPS and VM complexities

[jch.pgh]

After working with VPS hosting for several years, and starting to build larger infrastructure proposals with virtualization and 'cloud' resources in mind, this particular paper is going to be very useful. We have already been running into serious portability, security/compliance, and QA issues. Nevertheless, since we have a pulse, we can still see how kick-a** the price points are, and would much rather work through these issues in a community rather than bumble around with them ourselves. Just try to move an EC2 instance to Slicehost, or vice versa, and then try to get any piece of your proposal through the compliance lawyers, and you'll feel the burn.

lots of large companies have

[Trepidity]

For example, AOL nuked paying customers' websites, some of which had been hosted there for 15+ years, with a mere 2 weeks' notice, which resulted in some people losing their sites entirely if they were out of email contact for those 2 weeks and didn't have local backups.

Jason Scott has a better list

[Trepidity]

Jason Scott of textfiles.com has a nice take in Fuck the Cloud

Mod parent up.

[John Hasler]

n/t

is it really cloud computing

[DUdsen]

but is those really in the cloud computing business.

To me they seam a lot more like traditional mainframe/relocation companies i just dont see any cloud in singning a big deal with amazon stating what your entiled to and under what terms as anything different then what you did 20 years ago when different companes rulled the scene, and what exactly is the difference between what google offers and what compuserve AOL and the rest offered those 20 years ago when the internet went mainstram, your still signing in to a service and accepting the terms they offer.

The thing about is that the cloud is simply a big pile of smoke filled with otherwise completely normal businesses providing services that have existed since the dawn of computing.

If you define cloud as parallel computing themn the cloud is also old news. Everyone does that today.

Also needs fast, low-latency Internet feeds

[Brian Stretch]

for many tasks and that's a problem in much of the world, America especially (unless you live in Verizon FiOS territory, which I don't).