The Sidekick Failure and Cloud Culpability

miller60 writes "There's a vigorous debate among cloud pundits about whether the apparent loss of all Sidekick users' data is a reflection on the trustworthiness of cloud computing or simply another cautionary tale about poor backup practices. InformationWeek calls the incident 'a code red cloud disaster.' But some cloud technologists insist data center failures are not cloud failures. Is this distinction meaningful? Or does the cloud movement bear the burden of fuzzy definitions in assessing its shortcomings as well as its promise?"

Comments

Management

[FredFredrickson]

It's usually a decision on management's side to not use best practices, despite warnings from the tech dept.

tldr; There's nothing wrong with the technology, just the greedy bastards using it.

Re:Management

[sopssa]

As always, cloud computing/hosting/whatever is a vague term used like any other buzz term. I just see it as a platform where the resources should be allocated automatically and the underneath system takes care of having those available.

The same failure points are there. You're just putting the trust and management to someone else. Even if they do have backup plans and certain levels of redundancy, it can always fail. Cloud computing isn't something magical.

“Similarly datacenters fail, get disconnected, overheat, flood, burn to the ground and so on, but these events should not cause any more than a minor interruption for end users. Otherwise how are they different from ‘legacy’ web applications?”

That's because they aren't. The system is just managed by someone else, and its managed for thousands of people at the same time so its cheaper. Kind of like what Akamai has been doing for long with their content delivery network - it's cheaper for the providers because they dont have to build the infrastructure themself, and its cheaper for Akamai because they do it for so many clients.

Re:Management

[Splab]

Well there is one difference. Cloud computing and virtual servers are to computers what keychains are to keys, it enables you to lose everything at once.

Yes it is highly convienient and more effective to have everything in one place, but so much more fun when you drop your "chain" in the sewer.

Re:Management

[dkf]

Well there is one difference. Cloud computing and virtual servers are to computers what keychains are to keys, it enables you to lose everything at once.

It's not really a difference. With home-grown datacenters you still have that risk unless you do something like building multiple redundant buildings in different locales and managing some kind of replication and backup strategy. But then all of that stuff is the same with going to a Cloud provider, except you're not having to futz around with the physical facilities yourself.

There's no magic. All we're seeing is stupid people getting burned because they didn't use basic due diligence.

Re:Management

[dFaust]

But if Akamai loses a server, I don't have to repopulate the gigs of data they're hosting for me - it's not lost, it's just no longer on that particular server that died. That's exactly why I consider Akamai to be "the cloud" and why it doesn't side like Danger was. Especially with an infrastructure like Akamai or Google where things are geographically distributed, you just don't hear about servers dying, and you might not even hear about data centers dying (unless it places an unusually high burden somewhere and causes performance issues - but you don't hear about data loss as a result).

Re:Management

[Anonymous Coward]

Except the problem here is that when a large service goes down in the Cloud, millions of people can be affected.

For example, what if Google has their way with universities integrating with their system (Docs, Gmail, etc.) and Google has the sort of problems this Sidekick failure does? Now not just one university (if they own data center) has lost all of its hosted data, but any university relying on Google is out all the data hosted on the Cloud.

Re:Management

[joh]

The thing is that "the Cloud" means absolutely nothing in most cases. In almost every single case there're just remote servers storing your data as with every web app and every IMAP server since ages. The word "cloud" is just used to imply that there's something foggy you don't know anything about and to make you think that it can't fail. But of course in fact the data is stored somewhere and if there's no backup and someone wrecks the server your data is gone for good, as it always is in such cases.

I've had not a single case yet where "your data is stored somewhere where you can reach it only over the internet and you have no idea where exactly it is and how safe it is there" couldn't be used instead of "cloud". Using the "cloud" misnomer allows companies to outsource data storage to the cheapest bidder without telling you anything about it and at the same time sounding modern and innovative.

It's the same as speeding in foggy weather: Just because you can't see anything dangerous doesn't mean you're safe. It just means you feel safe as long as nothing is in your way and anything happening to you will come as a big surprise out of nothing.

A real "cloud" would mean distributed and redundant data storage in a network of servers independent of the service providers.

Re:Management

[linear a]

"There's no magic. All we're seeing is stupid people getting burned because they didn't use basic due diligence."

Due diligence becomes attenuated and more difficult to successfully apply. I'm sure the victims heard the same tales of massive backend redundancy that I'm currently hearing from cloud vendors. It's natural that a smaller portion of customers will successfully complete due diligence in a case like this versus for systems under their own control.

Note that this isn't a good excuse (based on the results in this case....).

Re:Management

[QuantumRiff]

True, but how much more money and brain power does Google have to invest in datacenter design and disaster recovery than your local college?

Seriously.. I worked at one.. All our stuff was on "next day parts" from Dell.. We had a single internet connection to the campus, single linux based sendmail email server, etc.

Granted, I had tapes up the wazoo, and could retrieve any file for the past X years, but downtime is still downtime.

Then you have Google, with multiple sites, multiple connections, replication, Load balancers, etc.

Not only do they have more to invest, but when they call up a vendor and say "we are Google, we have an outage, and we need some things from you" I bet those vendors jump a little faster than when a local school IT guy calls them up..

Re:Management

[wickerprints]

To be fair, Sidekick users didn't have a viable means to back up their personal data that was being pulled from Microsoft/Danger servers. I don't think it's reasonable to expect the users to find some hack or unofficial method to copy all their data from their devices. The only blame they could be assigned is that they bought the service being sold. Your criticism would be valid for, say, iPhone users, since the user has a backup stored on their computer. But no such functionality exists for the Sidekick, as far as I am aware.

And as to who is really being burned here.... Obviously not Microsoft/Danger. Microsoft doesn't give two shits about this, since their acquisition of Danger in 2008 was really about cannibalizing their talent for Windows Mobile 7, as the Pink project has shown. Danger is just a shell of its former self--the damage was done long before this latest failure, which I think was an inevitable consequence of the acquisition. The ones who got burned are T-Mobile (for trusting Microsoft to manage Danger, and Danger to maintain a proper backup solution), and of course, the consumers.

The real issue, of course, is that data is always at risk of being lost no matter how, where, or in what amount it is stored. The passage of time guarantees it. But people want to believe in the existence of certainties, in the notion that if something has a 99.9999% reliability, then we can effectively ignore the minuscule probability of failure. But failures happen all the time and there is no such guarantee. We need to rid ourselves of this delusion that data can somehow be made "safe," that risk can be ignored when made small. Cloud computing is just the flavor of the day.

I knew someone who worked at Danger years ago when the company was still fairly new. It was, at the time, an amazing technology. There was nothing like it. They had so much going for them, and there was a lot of good talent working there. One thing that impressed me was how they solved the problem of mobile web browsing. At the time, mobile web browsing seriously sucked ass. It was not only slow, but many sites simply would not load. Danger solved that by re-parsing the sites on their servers so that pages would look good and function properly on your mobile device. It was the best solution until mobile OSes and hardware became powerful and complex enough to support full browsing; and even then, the UI needed to be tightly integrated before browsing became efficient instead of tedious. It's sad to see such a pioneering company wither on the vine.

Re:Management

[StuartHankins]

To be fair, Sidekick users didn't have a viable means to back up their personal data that was being pulled from Microsoft/Danger servers. I don't think it's reasonable to expect the users to find some hack or unofficial method to copy all their data from their devices.

Absolutely correct. Wish I had mod points.

Re:Management

[BrokenHalo]

This all comes back to the thrust of the OP: whether the apparent loss of all Sidekick users' data is a reflection on the trustworthiness of cloud computing or simply another cautionary tale about poor backup practices.

The simple truth, of course, is that it is both. And the only solution here is the old one: if you want something done properly, you will have to do it yourself. If your data, documents or whatever are in any way important to you, you should not be relying on anyone else to keep them safe. Simple as that, and no excuses.

Re:Management

[Eskarel]

That's really a rather idiotic statement.

If your data is important then you take it's storage seriously. Sometimes that means you host it yourself, sometimes it means you get someone else to host it for you. You don't host your critical data if you can't afford the staff and infrastructure to support it, and if you've already got the staff and infrastructure you don't pay someone else to do it.

The important thing is that you take it seriously. That means contracts with your data storage provider with exactly what backup and restoration services they're promising and penalties for failing to meet those promises. It means full disaster recovery plans and proper due diligence including understanding what kind of outages you can afford and what it's going to cost you to keep outages under that value.

There's nothing inherently more safe about storing your own data or inherently unsafe about having someone else do it. In most cases the person who stores the data and the person who actually needs it are different anyway. What is unsafe is trusting someone else to look after your data without checking up that they actually are, be they internal or external.

Re:Management

[sexconker]

I don't know about you, but I keep all my keys in a safe.

When I go out, I open up the safe (which has it's own key), and then take only the keys I'll need for my outing.

I keep them loose, in my various pockets.

When I return, I return the keys.

One time I locked the key to the safe inside the safe itself.

What a day that was.

Re:Management

[pz]

There's no magic. All we're seeing is stupid people getting burned because they didn't use basic due diligence.

Yes, and, no. The people getting burned here are customers, by the many thousands. You can't expect the end-user to know what the DRP / BCP is for a subcontractor of the provider of their wireless communicator data plan. I wouldn't call the end-users stupid, and they are the ones most significantly affected in this case.

Re:Management

[Anonymous Coward]

One big difference is when using cloud computing you ultimately have to trust that they are both doing what they say they are doing to protect your data, and that their implementation isn't technically flawed for some reason.

When you do it yourself, you know what is being done, and you can test things yourself.

How much due diligence can you reasonably expect to do when dealing with Microsoft or Google or another large provider. They neither give you access to their facilities nor to detailed testing results/methodologies.

Re:Management

[lukas84]

Yep, but consider this:

Google doesn't care about your university.

You did. Or at least should've ;)

If you have local infrastructure, and you run into an issue, there's at least something you can do. For example, if your internet connection fails, it might be possible to hack something together to get email working again, for example using an UMTS Router.

Or if your Exchange server fails, you'll be able to get SOME backup restored, even if you can only take weekly backups because management said tapes are expensive.

If Google has a total system crash and loses 98% of it's data, but would be able to restore the remaining 2% at very high expenses, do you think they'd do it? Probably not.

Disclaimer: I use Google Apps Premier for my private domain.

Re:Management

[steelfood]

As always, the correct response to this is that it depends.

Using Akami as an example, they have mirrors in dozens and dozens of locations around the world. So if one or two of Akami's physical data centers goes up in flames, well, there are the other fifty data centers that're still around with your data. In this case, there is certainly redundance in the cloud.

However, if say, Google kept all of their apps data in one data center, and didn't mirror it, then if that data center gets swallowed by the earth, so does your data.

The difference between the "cloud" and your own implementation is that the cloud isn't under your control. You toss your data into the air, and the cloud picks it up. What happens with your data is beyond you now.

Some people like to have control of their data. Some people like to just put it somewhere and forget about it. The cloud isn't for the former type, but it's perfect for the latter. There are plusses and minuses to both sides, and tradeoffs to be made using one over the other. But FWIW, the layman probably likes the cloud because he doesn't have to deal with storing and securing his data. On the other hand, the geeks and experts probably like to have control, since we're fully capable of protecting our data.

Re:Management

[Anonymous Coward]

Although I agree to some extent with your argument, the fact remains that it sure didn't work out in this case!

Sometimes a company will so thoroughly fall in love with the savings of server consolidation, they fail to implement (and especially TEST) their nifty backup and failover infrastructure. You might think that a company that SELLS cloud-style services would be at the forefront of robust testing. Evidently not in this case. Competent datacenter sysadmins are an endangered species.

Sidekick users would have been better off with self-service USB backup to a laptop. Even if some of them neglected or screwed up their backups, the would have (at worst) the same scenario they have now. There is something to be said for being in charge of your own destiny.

This is the problem I have with outsourced cloud-style services. Because so much risk has been consolidated into a single service provider, they either run a spectactular operation or they fake it until disaster strikes. In the end, there is no guarantee that what they do will perform any better than a homebuilt Linux box.

Re:Management

[Anonymous Coward]

Perhaps it is time for two things:

1) An inspection / certification system to ensure data centers are following best practices in the protection of customers data. (probably expensive but good PR)

2) An option to email customer data to the customer on demand or on a customer selected schedule. (feasability would depend on nature of data)

Certainly the iPhone model where the customers data resides on his / her Mac or PC. (it just works, does depend on having an app on customers computer)

Re:Management

[DadLeopard]

One hundred percent right! Though you can also blame T-Mobile for locking down the Sidekick in such a way that there is no other viable backup solution! If they had left you a way to export your data to your personal computer it wouldn't be such a big issue!

Re:Management

[jacksonj04]

Google doesn't have to care specifically about your University, because the same backend runs everything. If Google Apps dies it's not just your University which has gone down, it's thousands of companies around the globe (Including Google themselves).

Cloud computing benefit - the one system is a single point of failure, but it's also a single point of 'we must keep this working at all costs'.

Re:Management

[jacksonj04]

So, that'll be you going through every line of code which powers your services?

You always need to have some element of trust in somebody else to provide a service fit for purpose. I'm pretty sure you trust your hardware and software vendors - cloud computing means instead of your vendors all you need to trust is the company providing the service.

Re:Management

[dkf]

If your data is important then you take it's storage seriously. Sometimes that means you host it yourself, sometimes it means you get someone else to host it for you.

Sometimes it also means getting multiple someones to host it for you and checking that they're not actually sharing the same hardware on the backend.

How much effort you put into this depends on how much you really care. If you're not willing to at least put some money in, frankly you're not going to get a good service and shouldn't be too surprised if things go "Poof!" at an awkward time. All a data cloud does is allow you to avoid having to have one provisioned datacenter yourself (a nice benefit for many people).

Re:Management

[dkf]

Yep, but consider this:

Google doesn't care about your university.

You did. Or at least should've ;)

Google care to about the extent that you're paying them (modulo the contract terms, which are usually not very good for cheap contracts TBH) and to about the extent that giving you crappy service would result in bad publicity.

In other words, normal for an external contractor.

You can have cloud infrastructure providers who provide stronger guarantees of service. They'll be more expensive because you are paying for those guarantees. A lot of people seem to prefer cheaper contracts and to either bear the risk themselves (or potentially by insurance) or to use multiple cheap providers and hope that the risks are independent. (For example, if a mysterious alien race destroyed all power generation capacity in the US, you'd be screwed even with multiple distributed providers, but the probability of that actually happening is low.)

Re:Management

[Anonymous Coward]

So you're saying someone could accidentally lock the cloud in their car?

Re:Management

[coaxial]

There's nothing wrong with the technology, just the greedy bastards using it.

Well that's a ringing endorsement of cloud computing if I've ever heard one.

Re:Management

[jimicus]

Google have been very open about their hardware from the off - they don't use servers with however much redundancy and 4-hour response times, they write applications which allow them to make the individual servers disposable. If a part fails, who cares?

It's a bit of a shame that so few applications allow you to do this yourself.

Re:Management

[dkf]

Though you can also blame T-Mobile for locking down the Sidekick in such a way that there is no other viable backup solution!

That's definitely the real issue, and such crappiness also used to happen before there were cloud storage providers. Lay the blame where it belongs: with the people who were selling the overall service that the end-user was getting without taking adequate measures to ensure its reliability.

Re:Management

[suomynonAyletamitlU]

I think that the point of the Cloud as being anything more than a buzzword comes down to one idea and one idea only, and that is specialization of labor.

I don't grown my own wheat, tomatoes, or cows, nor process any of those components, but I like Pizza. We have the culture we have in America because there are people who not only make their living DOING these things, but also figuring out how BEST to do them. They are regulated to make sure that your bulk-grown beef doesn't have diseases or other obvious dangers in it, and they are regulated to some extent so you know what kind of crap they're using to make them grow big, or at least what they AREN'T. All of this means that it's reasonably safe to go down to the market, grab a slab of beef off the rack, and cook it up at home. (Paying for it on the way out is recommended to avoid criminal charges as well.)

The idea that net-borne applications and data storage can work on a similar model ain't new, and anyone who suggests otherwise is blowing out their butthole. Server farms are aptly named; people are just finally figuring out exactly what their business model does, can, and should imply. Frankly, I'd rather lose all the cloud data in the world than all the bulk-grown food in the world, and that's not to say I have my own "backup" garden for self-sufficiency. Rather, it's just pointing out that this sort of thing has been done, and it's just because it's now "cool" that we're all ooh-ing and aah-ing and buzz-wording and anti-buzz-wording.

Re:Management

[Pascal Sartoretti]

To be fair, Sidekick users didn't have a viable means to back up their personal data that was being pulled from Microsoft/Danger servers.

At this point, they should have asked themselves if it was safe to depend from a single point of failure, and look for an other solution.

Re:Management

[Vo1t]

you probably mean economy of scale... a veeeery old concept indeed.

Re:Management

[mcrbids]

Don't confuse downtime (EG server powered down) with a catastrophic failure like this one. (total, irretrievable data loss)

Your school was a far better place (apparently) than MS Danger. Although downtime was more likely with your single sendmail server, you would still expect about 99.9% to 99.99% uptime year on year. that equates to about 4 hours per year on average. That's definitely down in the 'minor inconvenience' range for a school.

And your risk of catastrophic failure with all the (verified?) tapes is near zero. Sounds like a good solution to me!

Re:Management

[Trogre]

I think you mean light, fluffy keychains.

Look, it's very simple. If you put your data on someone elses server then no matter what they promise you, that data is no longer yours. Period.

Re:Management

[Anonymous Coward]

But then all of that stuff is the same with going to a Cloud provider

The biggest difference is that you are ceding control, and responsibility, to someone else.

You are also leaving your data backups in the alley at night and hoping they don't get stolen.

Re:Management

[mike260]

Cloud computing benefit - the one system is a single point of failure, but it's also a single point of 'we must keep this working at all costs'.

This argument, while very reasonable-sounding, probably needs to be reexamined in the light of Microsoft accidentally flushing half a billion dollars worth of their own data down the toilet.

Re:Management

[shentino]

While that is true in principle, there is also something to be said for hiring the best help and not trying to do everything yourself.

"Doing everything yourself" doesn't always work, particularly in the case of "a man who defends himself has a fool for a client".

However, the minute you want to have someone more competent than yourself handling the details, you are forced to trust them in one way or another.

When it comes down to it, simply accept that Murphy's law is an inevitable part of the world we live in, and it is certainly going to be aggravated by human nature being what it is.

Bottom line: There is NO way to guarantee that your data will survive no matter what. If lady luck decides to hold a grudge against you, you are screwed no matter how well you cover yourself.

Re:Management

[shentino]

And even if the end user was entirely competent, there simply aren't enough resources for one person to take care of all his needs by himself.

At some point, if you want to expand past a certain point you have to start delegating and giving up control. And that, unfortunately, is where you are forced to implicitly trust that whoever you delegate to isn't going to stab you in the back, either by accident or on purpose.

Ideally, you should be able to trust people not to let you down. Unfortunately, in the real world it doesn't work that day, and you need to give Mr. Murphy a place to sit so he doesn't plop down on your lap and squash you like a bug.

Re:Management

[Timothy Brownawell]

And the only solution here is the old one: if you want something done properly, you will have to do it yourself. If your data, documents or whatever are in any way important to you, you should not be relying on anyone else to keep them safe. Simple as that, and no excuses.

So do you also make your own condoms, or replace the airbags in your car with ones you made yourself?

Re:Management

[Mista2]

The Issue with the cloud is if it is just personal or worth money to you.
My Calendar and contacts are very important to me. If I am on call I need access to this information in a hurry. If I had lost them due to a failure like this then I would be severly hamperd in my ability to do my job.
To counter this, I store critical stuff in multiple places and I make sure my phone is backed up every time it syncs at home.

I use the cloud, but I dont trust it 110% just yet 8)

Re:Management

[conureman]

Was that an African or a European Keychain?

Re:Management

[AmiMoJo]

Furthermore it seems reasonable to expect Microsoft/Danger to have a backup system in place and to use it before attempting risky actions like a SAN upgrade. It seems like even just having a single redundant SAN or backup would have saved them here.

That doesn't sound like too much for a paying customer to expect. After all, you can bet they have your billing details backed up multiple times.

Re:Management

[Eskarel]

True, but that comes under the banner of taking it seriously.

There's this huge push on Slashdot that you have to do everything yourself and that all forms of outsourcing are bad. Neither of these things is true. I didn't build my house, and I don't do my own surgery. I have someone else do those things because they can do it better, I do manage my own computer because I can do it better and cheaper than I could hire someone for.

The same is true with outsourcing, not every organization has a full complement of IT staff and their own fully kitted out data center, nor should they. If they did there would be a whole lot of really bored staff, bankrupt businesses, and every spare inch would be filled up with gigantic DCs.

Replacing your existing staff with outside people you can't manage properly and aren't paying enough for is stupid. Hiring a company to provide you with a service you don't have the internal expertise for and which isn't a core part of your business(and while the data may be core, the storage probably isn't) is sensible. I shouldn't build a house and most bricklayers probably shouldn't architect a computer system, other people can do it better.

Re:Management

[elnyka]

Yep, but consider this:

Google doesn't care about your university.

You did. Or at least should've ;)

If you have local infrastructure, and you run into an issue, there's at least something you can do. For example, if your internet connection fails, it might be possible to hack something together to get email working again, for example using an UMTS Router.

Or if your Exchange server fails, you'll be able to get SOME backup restored, even if you can only take weekly backups because management said tapes are expensive.

If Google has a total system crash and loses 98% of it's data, but would be able to restore the remaining 2% at very high expenses, do you think they'd do it? Probably not.

Disclaimer: I use Google Apps Premier for my private domain.

Google (or any cloud provider) doesn't care about your organization the same way you care about it.

They do care, however, about SLAs. And that's all you need. I wouldn't care much if a 3rd party doesn't care about my business the same way I do. I care that he cares about SLAs between they and us.

Re:Management

[lukas84]

They do care, however, about SLAs. And that's all you need. I wouldn't care much if a 3rd party doesn't care about my business the same way I do. I care that he cares about SLAs between they and us.

Yes - and now look how extremely low the SLA is that comes with Google Apps. Specifically, they have no guarantee against data loss. You have to do your own backups - which kinda negates the whole "managed service" aspect.

This isn't really an issue if you know what you're doing - but many people simply think that services like these won't ever fail.

Re:Management

[Eunuchswear]

The important thing is that you take it seriously. That means contracts with your data storage provider with exactly what backup and restoration services they're promising and penalties for failing to meet those promises.

Which do you want, your data, or whatever crumbs the penalty will get you? In the real world penalties are only useful for CYA. "But boss, they will pay us $100 for every day our $1000/min server isn't up!"

Never put your eggs in one basket. Never. If you want to use a cloud, use two.

Re:Management

[Anonymous Coward]

I would at least call them foolish for storing any needed data on a system with no means of making secondary backups....

Cloud computing may be convenient. But I would never rely on their backup procedures.

Re:Management

[BrokenHalo]

That's really a rather idiotic statement.

No, actually it's not. I have worked in sysadmin jobs since the '70s, and there's one thing I learned very early, and that is to keep multiple backups in more than one place, and to verify those backups. In the 30-something years since, I have fucked up once (fortunately not catastrophically) by not having verified a backup when two others were destroyed by fire, and I make no excuses.

Backups are now easier than ever before to manage, so there really is no excuse to not do them properly. It won't wash to tell your customers (or your boss) that your hosting service should have maintained a better backup policy. That'll go down really well when you've managed to lose everything.

Re:Management

[Moofie]

Do you have any idea how many single points of failure there are in, say, a car?

Re:Management

[Eskarel]

Yes, but best practice for backups has nothing to do with who is following those best practices. Your backup provider doesn't have to be internal, they just have to be doing the right thing.

A reason why cloud computing might be hated on SD

[Anonymous Coward]

With Cloud Computing, those who modify FOSS software do not have to redistribute the code, because they are only providing a service and not a functional program.

This is an unforeseen hole in the bulletproof Gandhi mechanism, so I foresee a quick "GPL V3.1" to close this. And then all is well.

The problems with outsourcing

[davidwr]

If you can't trust your outsourcing partner, replace them or bring the work in-house.

Re:The problems with outsourcing

[postbigbang]

Trust? All that data's gone without much chance of it being recovered, as in bye-bye.

Do you think that perhaps T-Mobile, or their "trusted partners" might have had a full backup (an IT 101 sort of plan), a mirror or highly available machine (an IT 201 sort of plan), a disaster plan (IT 301), or maybe just an encrypted torrent out there somewhere?

No.

Heads oughta roll. Cloud computing is only as good as you make it; it only represents a server outside of your office's NOC or physical boundaries. Nothing else is guaranteed. In this case, it was a service running on somebody else's host (and not properly done) and so it's not a matter of doubting the cloud, it's a matter of firing an incompetent vendor, then getting ready for the barrage of litigation and shame. Stupid stupid stupid. Put a bell on these guy's necks. I don't want them around me.

Re:The problems with outsourcing

[vertinox]

If you can't trust your outsourcing partner, replace them or bring the work in-house.

Trust? I don't think the upper management trusts local IT either.

Really, I don't think it matters who runs the servers or what they call them as long as it is run well.

Just because its outsourced or inhouse or its gold big iron or cloud computinhg doesn't make it good or bad because either way can be run poorly with the wrong administration.

Personally I think things should be done in house merely for moral issues bit business speaking an incompetent admin is going to mess up things whether he works for the outsource company or directly for you.

Re:The problems with outsourcing

[jeffasselin]

How CAN you trust them? Any big corporation offering those services are after only one thing: profit. And to get it, they WILL cut corners. Doesn't everyone? But in this case you'll have no idea where they cut short, and no idea where you're unsafe, and how much downtime you might have if something goes wrong.

I'm sorry, but the main issues with Cloud Computing aren't technological, they're issues of trust and reliability of the human, financial and legal factors at work. And when it's you vs Big Corp, you'll lose every single time.

Re:The problems with outsourcing

[sjames]

Or do something sensible like store the data on the phone AND on a server for a best of both worlds solution with added reliability?

Re:The problems with outsourcing

[postbigbang]

You would want that to work.

Maybe use something like rseven.com.

How many offsetting offsets to the offsets does one need to ensure that simple data is backed up??

In the real world, too few people backup anything, then cry lots of tears. Here are people that actually went thru the process to get the service. Now their backups are missing-in-action. To the hosts of this service: off with your heads. Not excusable.

Re:The problems with outsourcing

[sjames]

As the number of backups is increased, the chances of total data loss approach (but never reach) zero.

The designers of the phone passed on a great opportunity to greatly improve the reliability and the availability of the data storage, presumably to lock the users in. It is, indeed, inexcusable.

The moral: Nobody values your data more than you do, insist on keeping a local copy.

Re:The problems with outsourcing

[postbigbang]

Amen.

Re:The problems with outsourcing

[SBrach]

But then people who don't want to pay for a data plan can use their phone!!

Re:The problems with outsourcing

[sjames]

I'm guessing that had everything to do with it. Once again, customers come last.

Re:The problems with outsourcing

[symbolset]

Daniel Dilger on Roughly Drafted is speculating that it's an inside job.

For the love of God the company is called "Danger"

[syntap]

Didn't that throw up any red flags for ANYONE?

Cloud Failure

[commodore64_love]

I know my songs, videos, and other important files are backed-up across triple drives. I don't know if the same is true if I stored them online, and this major failure of Sidekick demonstrates I'm right not to trust them.

Re:Cloud Failure

[commodore64_love]

Also with ISPs like Comcast imposing 250 gig limits, why on earth would I want to offload my information across the net? It makes more sense to *minimize* the data transfer to avoid overage fees, not increase it.

Re:Cloud Failure

[sopssa]

I know my songs, videos, and other important files are backed-up across triple drives.

Dude. Now close your torrent client and go out.

Re:Cloud Failure

[MBGMorden]

While true, I'd say that if you're unhappy with a 250gb cap, do your darndest to at least come close to it each month (though in reality if you're unhappy with it you're likely coming close to or breaking it each month anyways). The only thing that will raise that number (or cause them to upgrade infrastructure allowing them to raise it) is if they see their average monthly consumption getting to close to that number. If their average monthly consumption is 2.5GB (I know people whose monthly bandwidth stays at that level or less), then they'll never increase a 250GB limit. If it's 227GB, you can bet they'll scramble to up it.

Do your part to drag that number higher :).

FWIW though, my ISP is a smaller outfit (Spirit Telecom) and has yet to impose any bandwidth restrictions that I'm aware of, though checking my router's logs I personally only come in at 80 to 125GB per month.

Re:Cloud Failure

[commodore64_love]

No it's cold. Besides how am I going to watch these latest episodes of Stargate and Eureka if I'm outside playing with the squirrels and birds?

Re:Cloud Failure

[slim]

I know my songs, videos, and other important files are backed-up across triple drives. I don't know if the same is true if I stored them online, and this major failure of Sidekick demonstrates I'm right not to trust them.

That depends entirely on the online storage service you use. If your contract says the files are backed up across triple drives, then you've a right to expect that they are. If your contract doesn't say that, then you shouldn't expect it. Simple.

Now, I'd argue that any cloud service worthy of the name ought to have very robust mirrored storage. But since there's no legal definition of the word, you'd better read the contract.

Re:Cloud Failure

[LVSlushdat]

According to my Tomato router (Linksys WRT54GL) I'm using well over 100GB/mo, and all I do online is watch videos (youtube etc), a pretty throttled back torrent client for a few linux distros/month, then leave them up to help out, plus wife has a pc she does email/IM and some watching youtube.. I'm on Cox and until very recently I was always kind of holding my breath waiting for a note from them telling me I was over their 40GB cap.. Now I understand they've raised them..

Re:Cloud Failure

[Anonymous Coward]

all I do online is watch videos (youtube etc), a pretty throttled back torrent client for a few linux distros/month, then leave them up to help out

whiskey tango foxtrot. Can't imagine how you use 100 gb/month.

Re:Cloud Failure

[Anonymous Coward]

That sounds nice. In the real world however consistently reaching your 250GB cap with Comcast results in a telemarketer calling you every few hours trying to "upgrade" you to the business service, because for $150/month more you can get that cap removed. Been there, calls only stopped once I moved out of their area. Now I'm with Optimum, no cap, but God have mercy on you if you ever have to call them up because your modem decided to shut down again and it needs to be remotely reset...

Re:Cloud Failure

[sexconker]

Uh....
This is the telco industry you're talking about.

If a lot of people sit at 225 GB (out of 250 GB), they simply drop the cap to 200 GB.

Re:Cloud Failure

[sexconker]

Dude.
DUDE.

Get a team of squirrels, and dress them up in SGC uniforms, and create your own episodes.

None of this Atlantis shit.
No fucking Universe crap.

Original SG-1 squirrel reenactment GO!

http://www.sugarbushsquirrel.com/642346.html

Re:Cloud Failure

[JaegerTCat]

It's $89.99/month here in Chicago. Only $50 more than regular. I get 16Mb DL, constant (I download stock data all day every day), bursts to 50Mb, no cap, and excellent service.

Re:Cloud Failure

[vadim_t]

Contracts aren't a guarantee. They may sign a contract and not follow the terms. They may go bankrupt. A massive failure may make the total amount of compensation to be paid larger than the money the company has. If you're a small customer, they may be able to screw you over and ignore it, because suing isn't worthwile.

Even if a service ended up paying me for a data loss there's still information that can't be replaced.

I can't take another photo of my dead cat.

A company may not be able to rebuild a database regardless of the amount of compensation it gets -- the data may not exist anywhere else. The loss of trust by the clients may never be replaced.

If you have something really important, back it up yourself, and ensure it's being done right.

Re:Cloud Failure

[JStegmaier]

The only thing that will raise that number (or cause them to upgrade infrastructure allowing them to raise it) is if they see their average monthly consumption getting to close to that number.

I think you're confusing Comcast with a reasonable company. Increasing bandwidth caps would be a reasonable response to your scenario. With Comcast, it's more likely that they'll just burn down his house.

Re:Cloud Failure

[commodore64_love]

>>>It's $89.99/month here in Chicago. Only $50 more than regular.

So about $140/month.

Holy shit (picture in your mind a priest blessing a pile of crap). You have a strange definition of "only". For me that word should come before $15 as in, "I pay only $15/month for my internet" not in front of $140.

Re:Cloud Failure

[commodore64_love]

>>>I can't take another photo of my dead cat

You could but it would probably be unpleasant.

Re:Cloud Failure

[JaegerTCat]

No, it's $89.99/month, all in. Only $50/month more than the basic service.

Re:Cloud Failure

[slim]

Contracts aren't a guarantee. They may sign a contract and not follow the terms. They may go bankrupt. A massive failure may make the total amount of compensation to be paid larger than the money the company has.

By that logic, you'd never outsource anything at all. Never hire in an external company to do your office catering: they may sign a contract and not follow the terms!

Wrong story

[davidwr]

This belongs in the BSA story. At least there it might be modded insightful or funny.

Who do you think will get fired over this?

[TheMaTrIxBEL]

The managers responsible for not implementing the backups or the techs maintaining the infrastructure? My bet is on the little people.

Re:Who do you think will get fired over this?

[Anonymous Coward]

Why do you think it wasn't deliberate? Blowing up Sidekick data is a small price to pay if it scares people away from cloud computing and gets them back into the Microsoft ecosystem.

Re:Who do you think will get fired over this?

[harlows_monkeys]

Danger is a Unix, Oracle, Java shop, so if you are going to go down the "they did it on purpose" path, you might as well theorize that it isn't just to scare people away from cloud computing. It could be to scare them away from Unix, and Oracle, and Java! Conspiracy nuts might as well be bold.

Re:Who do you think will get fired over this?

[Anonymous Coward]

Why do you think it wasn't deliberate?

Well, because Microsoft is launching their home-grown cloud computing initiative in just a few weeks, and the fact that another one that was under their care was so poorly managed that it unrecoverably, catastrophically failed will probably make people rather hesitant to put their trust in Microsoft's.

Re:Who do you think will get fired over this?

[TheMaTrIxBEL]

Microsoft is one of the big forces pushing for Cloud computing ... Cloud computing is Microsoft's ecosystem. They are kinda betting their future on it. As for 106428, yeah, I did think about that, but then I said, who's to say the SAN they used was Microsoft based. It could well have been Linux based. A good SAN or NAS doesn't discriminate between the platforms to start with. Not to mention a really good SAN is completely hardware with no Linux or MS OS logic behind it other then the servers connecting to it for storage over hardware links and universal standards like iSCSI.

Cloud Computing

[b3x]

No matter where you go, there it is.

Re:Cloud Computing

[sopssa]

I've experienced cloud computing myself. It wasn't as cool as I thought tho. A week ago I was traveling and the airplane was landing but I couldn't see out - but there I was, happily computing in the cloud (until the stewardess came to tell me shutdown the freaking laptop during landing)

OT: Your sig line

[davidwr]

According to two separate studies, 2500 song downloads == just 1 album lost sale

According to multiple separate anecdotes, lousy music = multiple separate lost sales.

Re:For the love of God the company is called "Dang

[Sockatume]

I thought the same thing about "Microsoft".

Okay guys, that joke's done, let's get on with our lives.

AGPL

[Koohoolinn]

This is an unforeseen hole in the bulletproof Gandhi mechanism, so I foresee a quick "GPL V3.1" to close this.

It already exists. It is called AGPL: http://en.wikipedia.org/wiki/AGPL/

Semantics

[jayhawk88]

In the end, it doesn't really matter if it's a data center failure or a "cloud" failure. It matters who the user blames. And if you trumpet yourself as "in the cloud", and then that cloud rains on your consumer, whomever is at fault, ultimately it's you, the provider, who has a problem.

Has there never been a non-cloud data loss?

[iamacat]

Just like people lose their stuff on personal hard drives when not backed up, they will lose cloud data when not backed up. Both kinds of computing have merits, and long term persistence of data is not automatic with either. Most people do not place THAT hard a value on backups of their cell phones. They typically sync with a PC anyway. But any business that doesn't have weekly reliable offsite backups of their fundamental assets should be sued by shareholders/customers for irresponsibility weather they use cloud or not.

Re:Has there never been a non-cloud data loss?

[slimjim8094]

We take these sorts of catastrophic failures more seriously. If my data is on my computer, I know that I'm responsible for my own damn data - if I don't back it up, it's my own fault. But more importantly, when I lose everything, you don't care. If I'm holding millions of peoples' data, you suddenly care a lot more - especially if you're one of the people.

My point is, with all that data they damn well better be backing up properly, because it's out of my control. If I don't back up properly, I had the option and (hopefully) know the trade-offs.

Re:Has there never been a non-cloud data loss?

[alen]

the hype about "cloud computing" is that there are never any failures and all your data is always going to be safe. at least that's the way the tech rags hype it.

Re:Has there never been a non-cloud data loss?

[vertinox]

Just like people lose their stuff on personal hard drives when not backed up, they will lose cloud data when not backed up. Both kinds of computing have merits, and long term persistence of data is not automatic with either.

Neither RAID or Cloud Computing is a backup solution. Its merely a way for more uptime and availability of data.

If by chance the user overwrites or deletes the data on a RAID or an online storage service... Then you've lost your data just the same as if the server crashed.

Re:Has there never been a non-cloud data loss?

[iamacat]

So the basic problem is that people don't understand what cloud computing is. If I have cash under my mattress, I presumably understand the urgency of keeping my home secure. If I keep money in the bank, I better understand what measures are in place to give me back my money in case of robbery, wire fraud or bank failure. This does mean keeping money in bank is bad. Neither FDIC negates the usefulness of keeping some money under the mattress or at least in a hard asset like gold or real estate. It just means that one has to keep track of modern financial and computing developments to avoid serious trouble.

Re:Has there never been a non-cloud data loss?

[John Whitley]

Heck, I know folks who've lost entire well-known (hobbyist) web-portals some years back due to provider server failures. It was a harsh lesson for those involved. So much for the provider's backup policies. The real solution is to have multiple copies of the data, ideally in different formats. For example, when I was in grad school the University had (for the time) a huge email installation, basically full email hosting for the entire institution. The server and storage spec was excellent -- a big SAN-like dual storage array that could handle failures at multiple levels, including one entire half of the storage system. Turns out they got hit by a nasty filesystem corruption bug, which nuked the whole array. Oops. Their bacon was saved because they also had regular verified tape backups (IIRC, it took many, many weeks to fully restore archived mail to the cluster).

These problems really have little to do with the computing models involved. There's a misperception that the "cloud" provides some sort of data robustness beyond what mere mortals can accomplish, but the reality is that valuable data just needs more copies. Perhaps their backup strategies are layered and awesome, but you never really know where the weak links are. One remote service provider really only ever counts as one copy. And so it's useful to consider a service like GitHub. The fundamental model of the service is to encourage folks to share and copy their data around, because that's a prime goal of the supporting software: git. If a git-based service goes down, there should be many copies of the repository data, and the various users will regroup, republish, and move on. No single user has to be overly conscious of maintaining lots of backups, because copying is the basic working model.

There's a lesson there for those of us working in software: design for subversive backup, where critical data is backed up/synced/secured as a normal part of day-to-day workflow. Make sure that failure in any one point doesn't induce the others to similarly fail or become corrupt. Think through and verify the recovery schemes. Imagine that it's your data going down the tubes...

Re:Has there never been a non-cloud data loss?

[Anonymous McCartneyf]

Problem for T-Mobile Sidekick users -- the computer they synched with was the one that crashed and took everything with it. Sidekicks weren't designed to sync with the user's computer.

Need more info

[LS1 Brains]

Can't form a complete picture without an inside look at what really happened. Danger/Microsoft obviously isn't going to just come out and tell us the who or how, they have enough egg on their face as it is.

We can throw hunches around all day long, but it all boils down to human error somewhere - or more likely, a series of errors. Perhaps backups weren't properly taken. Perhaps they were performing a platform shift to .NET and something went awry. Perhaps a dev was tapping out a query and forgot part of his where clause, irreversibly damaging an entire table. Perhaps the cleaning crew poured milk in the disk cluster. These are all quite valid possibilities, which singly probably wouldn't be an issue.

I don't think there's any argument for instability or reliability issues with a "cloud" platform, any more than one could form an argument for a traditional arrangement. If the system as a whole isn't managed and maintained, you are at a very high risk for disaster. The only universal truth is things WILL fail, and you have to plan for them.

Re:For the love of God the company is called "Dang

[garcia]

Didn't that throw up any red flags for ANYONE?

I was a Sidekick user from 4/2004 until 10/2008. There had been only one 'catastrophic' failure in that time that left Sidekick users without data service for an extended period. Danger produced one of the best mobile devices, which in many ways is still better than anything out there even though the OS and devices that utilize it (the various Sidekick models that exist these days) is quite a bit outdated compared to devices like the iPhone.

I miss my Sidekick immensely. I loved true multitasking, a fully capable QWERTY keyboard, and incredible battery life. Unfortunately it didn't sync well with calendaring software, didn't keep up with music playing, and is now partially controlled by Microsoft. There have been immense trade offs with moving to the iPhone but based on my main reason for owning an iPhone (I ride the bus and enjoy the music/video player and screen size) it was the right choice for me.

That said, "cloud computing" is something which usually works (and did, in the case of the Sidekick since 2002). I don't think that this is a proven warning sign that "cloud computing" isn't as reliable as everyone believes, I just think it's proof that companies need to do a much better job of ensuring data integrity than they could have ever imagined before.

Will I stop using Flickr, Google products, and other future "cloud" devices/software because of this? No. I am smart enough, as a computer savvy end-user, to keep my own backups of my data but I do believe people need to become better educated in what can and will happen as we move to the model we have slowly done in the last 10 years.

What IS cloud computing?

[dFaust]

Personally, I always interpreted cloud computing as software that's running on a number of boxes of which the number can fluctuate without being meaningful (obviously there are performance implications depending on the overall load and number of boxes, but one box going down doesn't inherently bring down the system). One nice thing is these boxes can be geographically distributed as well - so when one data center gets nuked, the others are safe. Now, I realize geographic distribution isn't a requirement but even still, the press release says the data loss is due to a "server failure." Not a data center failure, but the apparent failure of a single server.

So is this really even "the cloud"? Does that mean that Geocities was "the cloud" or that every web host out there is "the cloud" because they've got my data running on a single machine? I certainly never interpreted it that way, but I'm no expert on the matter. It seems like if this data was in "the cloud" that it could have all been retrieved off of another machine somewhere. Perhaps for some customers those other machines might not yet be completely synced with very recent updates, but that would affect a small amount of data for a subset of customers.

Re:What IS cloud computing?

[Anonymous Coward]

The way most tech journalists use "the cloud" these days is effectively a synonym for "teh internets". It's something remote that they don't really understand.

Re:What IS cloud computing?

[Anonymous Coward]

Cloud computing is where your data goes up in smoke.

AC for a reason.

Re:What IS cloud computing?

[plague3106]

Cloud computing is just what it sounds like; something meant to obscure whats really going on. Nobody seems to know if it was a single server or the data center. And thats the risk of the cloud... you won't really know whats going on at the other end.

So while they'll say "let us manage it" they'll really do that same things you did anyway; keep only as many servers as you need, and they may more may not be backing up your data. Oh, and your data doesn't even need to be lost, just unavailable because the internet isn't as reliable as everyone would like you to think.

Re:What IS cloud computing?

[mick88]

I think more people should be asking your question: "what is cloud computing?". Because, in my opinion, it's easy to hide behind the name "cloud" - hell the name itself implies obfuscation and mystery.

But the real answer is that the "cloud" just is an internet-facing datacenter housing services or data. The trustworthyness (is that a word?) of the cloud is really dependent on the provider of the cloud. Some clouds are more redundant, resilient, and secure than others. That's important to consider when you're evaluating a move to the cloud. You _need_ to know where the data lives & how it's being backed up / secured. The term "cloud" implies it, but doesn't ensure it.

The cloud is like the internet - you could think of it as one giant nebulous entity, but in reality it's a bunch of independently owned & run services. just like AOL != the internet, geocities != the cloud. But there is a relationship there.

To me, this story about the "cloud failure" is like having someone's local ISP have an outage, then cry about how the Internet isn't reliable.

Re:What IS cloud computing?

[rwa2]

We'll, I was hoping to just google cloud vs. grid vs. distributed vs. cluster vs. etc. computing, but there doesn't seem to be much official-sounding distinction out there. Which means if we start our own thread here it might become definitive!

"cloud" computing: fluffy term used by people who really don't know anything other than that they run their applications from a web page and their data appears to be stored on the web because they can access it from more than one web browser.

"hosted" / "server farm" computing: buying server resources from someone who has a real datacenter who tries to take care of your hardware. You access all of your data over the network "cloud". Redundancy & support varies based on pricing & services.

"grid" / "utility" computing: computing infrastructure where you should be able to simply scale up CPU, data, etc. resources for your operation simply by throwing money at turning on more boxes. You don't necessarily need to share it with others, though.

"cluster" computing: a computing system made up of more or less independent, generally homogeneous nodes, where problems can be partitioned out. Generally has some form of redundancy so you don't lose work when a single node dies, but probably won't survive a data center failure.

"distributed" computing: special applications that can be farmed out to the net to break parts of computing or storage across a heterogeneous network of computers distributed over many locations. Ideally it's written to be highly redundant and tolerate faults such as nodes joining / leaving the cluster.

As far as reliability goes, the TIA data center tiers seems to be the only common way of talking about maintaining "business continuity". I've read through it briefly, and can somewhat paraphrase the intent (mildly inaccurately, mostly because the standard itself is kinda loose and not defined in too much detail with regards to servers) as:

Tier 1 "basic" : You have a room for servers with a door to keep random people from tripping over the plugs. Maybe you have a UPS on your server so it can do a graceful shutdown without data loss when the power or AC goes out.

Tier 2 : You have your stuff in racks with a raised floor for air conditioning and some wire racks hanging from the ceiling for cable management.

Tier 3 : You have redundant UPS's and RAIDs, CRACs, network links, and stuff, so you can make repairs when common things break without turning off the system (typically anything with moving parts or high currents, like power supplies, fans, disks, batteries needs to be hot-swappable). Which means you should also have some sort of monitoring and alert system so you know when that stuff actually fails so you can replace it before the redundant components also fail. This is intended to reach 24x7 availability with high uptimes... , maybe 3-5 nines.

Tier 4 : Like Tier 3, but certified for mission-critical / life-critical use, like in hospitals and maybe for airplanes and stuff. It should survive prolonged power outages (so you have a diesel generator with a day or two worth of fuel.)

Unfortunately, it just covers build specs for individual data centers, so it doesn't really cover other business continuity things like maintaining offsite backups so you can somewhat easily rebuild from scratch if a natural disaster takes out one of your data centers or something. But it's kind of different worlds of IT between designing facilities and architecting "cloud" services, which unfortunately don't seem to communicate or collaborate as much as they should to reach the kinds of "distributed grid of redundant load-sharing data centers" configurations we'd expect.

Re:What IS cloud computing?

[KDingo]

Around when I first heard the term, I thought the internet was the cloud. For the simple fact that I represent the internet as a cloud whenever drawing network diagrams.

I saw a video from datacenterknowledge.com asking this very same question, though. It's fourteen minutes, but it shows that there is not a single definitive answer.

Meta-cloud, anyone?

[Fjodor42]

To my mind, this failure just goes to show that what people call clouds are merely the mainframes of yesterdecades... For the cloud to become "THE" cloud, the providers need to cooperate to replicate data across their different implementations, such that when one provider suffers an unforeseen crash of unforeseen magnitudes, the data is til there in the "real" (in this definition) cloud.

Sure, it would take no small amount of convincing to get the management drones to accept this, but I should think that a cost/benefit analysis that includes catastrophic failure would be somewhat persuasive...

Re:Meta-cloud, anyone?

[FlyingBishop]

If someone tells you that they can cheaply prevent catastrophic failure, expect a catastrophic failure. Nothing can correct something like this, which involved an error propagating to the backups.

Re:Meta-cloud, anyone?

[Fjodor42]

Incidentally, that is my point exactly! Distributing the data across multiple providers would be a no-no to the providers involved, but if they realised that no one provider can prevent catastrophic failures might persuade them to at least think about it...

Re:Meta-cloud, anyone?

[Anonymous Coward]

Nothing can correct something like this, which involved an error propagating to the backups.

If you can instantly corrupt your "backups", they're not backups. If you're doing anything serious and you don't have offline copies at a remote location going back at least a few months, you're doing something very wrong.

Re:Meta-cloud, anyone?

[linear a]

Not strictly true. Couple of alternatives that could completely or partially remediate after the fact. Partial - static archive copies (e.g., tape) Complete - those data tracking systems that note all changes to blocks and let you restore back to a point in time. Not 100%, since nothing it, but you can get as close to 100% as you care to pay for.

Re:Meta-cloud, anyone?

[FlyingBishop]

Well, I think where I differ is that I don't think duplicating providers offers an increased level of security. It's really just giving you twice as many cheap providers who aren't watching out for the sort of problem that can propagate everywhere.

With the money you spend on those extra providers, you should be shoring up your investment in your primary provider (including some more redundancy) and hiring more people to make sure that you've covered every contingency.

Re:Meta-cloud, anyone?

[Anonymous Coward]

Well, I think that we are talking past each other...

Money spent on security at one provider *is* well spent, but their whole infrastructure is still theirs, and, to my mind, presents a single point of failure, should all go horribly wrong.

By cooperating with your competitors with regards to data security doesn't make sense in the frame of mind that you are helping them, but they would be helping you as well to an equal degree. In that frame, you could say that no one loses or gains anything, but *the consumer* would gain a significant amount of extra security, and that might be a selling point for the corporations that cooperated with each other...

Re:Meta-cloud, anyone?

[afidel]

A backup system which can be corrupted is no backup system at all, which is why I still like tape damnit no matter who says tape is dead. The reality is that with tape you might have an extended RTO and you might not make your RPO but unless you haven't tested your restore procedure at all you WILL be able to restore eventually. The fact is that if Danger was able to tell their users they would be getting everything back as of 1 weeks ago within 48 hours they wouldn't be in the world of hurt they are certainly going to be in due to this complete farkup.

Re:A reason why cloud computing might be hated on

[vagabond_gr]

It's called Affero GPL

Not a cloud, so why the fuss?

[mangastudent]

A single data center apparently without even a geographically distinct failover site is about as far as I can imagine from being a "cloud". Old fashioned best practices in the form of having two or more sites each capable of handling the entire load would have prevented this particular mess, let alone classic cloud approaches like that of the Google File System (GFS) which keeps at least three copies of a file's contents.

(Granted, if you're storing vital stuff in GFS or Amazon S3 you still have a logical single point of failure (e.g. a mistaken delete command) and therefore you aren't freed from the duty of doing your own backups, but that's a separate issue.)

Or we could just say that trusting Microsoft for anything is relatively unwise compared to other "higher tier" companies. Or that if you're depending on a service provider that's massively laying off staff you need to take action before something seriously ugly happens, because it likely will.

Your data is your responsibility.

[zerofoo]

As a wise auditor once told me:

You can outsource the work, but you can not outsource the responsibility.

If your data is important to you - you must back it up, and you must test your backups.

The end.

-ted

Re:Your data is your responsibility.

[Slashdot Parent]

If your data is important to you - you must back it up, and you must test your backups.

What is your suggestion for people who don't know how to back up their own data?

Re:Your data is your responsibility.

[coaxial]

There's nothing wrong with the technology, just the greedy bastards using it.

That's interesting, because all you need to show to defend yourself in a lawsuit is due diligence. "We backed up our stuff off site with Microsoft/Google/Amazon, and they dropped the ball," is a perfectly defensible position.

I know a prominent researcher in storage security, including hard drive forensics. (FYI: That whole "wipe your data 7 times with ones and zeros," has been pointless for at least a decade. With today's servos and magnetic domain sizes, one wipe and it's gone.) He pointed out that many times companies that suffer some hard drive crash and want the whole clean room recovery, actually don't want the recovery to succeed. They just want to be able to say, that they tried. (They law says, you don't have to succeed, you just have to try.) Same thing with backups from seven years ago. The only reason why you have them is in case of a lawsuit, and anyone who has been sued knows, the less records you have the better. If you have the backups, but they've become unreadable in the years hence, it's not destruction of evidence. It's just a loss.

Re:Your data is your responsibility.

[dkf]

What is your suggestion for people who don't know how to back up their own data?

Learn to do it yourself or hire someone who does know how.

Re:Your data is your responsibility.

[Slashdot Parent]

What is your suggestion for people who don't know how to back up their own data?

Learn to do it yourself or hire someone who does know how.

How would I know--before it's too late--that I hired someone capable of safeguarding my data? I ask, because I imagine there are an awful lot of Sidekick owners who thought they DID "hire someone who does know how".

In hindsight, they were wrong. But the Microsoft name connotes expertise to a lot of people (although not a lot of people around here--for good reason).

Re:Your data is your responsibility.

[symes]

You can outsource the work, but you can not outsource the responsibility

Spot on. And that is so true in so many walks of life. But I think the deal with the Sidekick was that there was no easy way for users to backup their data...

Re:Your data is your responsibility.

[Kumiorava]

If I understood this issue correctly all the data on the server was safe and local only data was lost. Therefore users did have an easy way to backup their data, should have just synced with the server. The data would have been also lost if phone was stolen, broken, or well, reprogrammed.

I don't know how Sidekick works, but something made the system think local data was older than server data. Or the server just sent out massive reprogram command to all devices devices asking reboot sync. To me this sounds like a dumb mistake on Danger side that could have been avoided easily.

Re:Your data is your responsibility.

[Kumiorava]

Nevermind, it seems that Sidekick doesn't have any data stored locally during boot...

Assumptions

[eagl]

Just because you're paying someone to store your data doesn't mean they care about that data as much as you do... That's one of the two big problems with cloud computing that can't be solved by technology. First, nobody cares about your data as much as you do. Second, nobody will protect your data (ie. control it's distribution and prevent unauthorized changes) to the level you find appropriate.

It's usually a good idea to avoid using broad generalities (like I just did), but it seems like in general it would be a bad idea to let someone else be the sole keeper of anything even remotely important or sensitive. There are exceptions, but those seem to be internal to a company (ie. the company runs it's own cloud and has all employees use it). Or military/government applications where centralized security and backup can keep user errors from becoming a real danger to the organization beyond "help I lost my email!".

It's the backup stupid

[trybywrench]

I think the key here is was it only T-Mobile's data that was lost or was every customer of the "cloud" affected. If it was only T-Mobile's data than the issue is T-Mobile's backup policy, if it was "cloud"-wide than it's an issue with the "cloud" provider. In either case, I don't think you can paint the entire "cloud" concept as unstable. Cloud computing is really just a dynamic datacenter with all the usual weak links and issues present in a traditional metal datacenter.

Not a cloud disaster, not a "data center" disaster

[TheLoneGundam]

Leaving aside the fact that a "data center" could consist of two servers under Mabel's desk, this is not a "data center" disaster, nor is it a cloud catastrophe.

This a contract and contract management failure: the contract with the outsource was probably written without specifying that they must do the backups, AND no one established any sort of audit (formal or informal) test to ensure that there _were_ backups being taken and that the outsourcer was performing according to the contract.

Too often, the MBA doing the contract thinks "there, that's handled" once they've gotten all the signatures on the dotted line. "There, backups are handled now" he thinks, because many business folk (not ALL, I don't think it's fair to generalize that far) see these kinds of things as milestones, rather than ongoing processes to be managed.

The Cloud is Just a Big Mainframe

[tjstork]

When you cut through the "cloud", if you look into the center of things, you see that the so-called modern "cloud" computing environment is a giant computer(s), surrounded by high powered priestly geeks, doling out resources to everyone, completely centralized. The priests have some new tricks to entertain the masses with, but there's nothing fundamentally different between cloud computing and IBM's vision of computing in the 1960s.

Re:The Cloud is Just a Big Mainframe

[Attila Dimedici]

I wish I had mod points because that is the best summation of "cloud" computing I have read yet. Every few years some technological development causes this computing paradigm to be brought up as the "new thing" in computing. Every time this happens there are all these people talking about how it is the "wave of the future" and that all computing will go that way. After a few years, people realize that it has the same limitations that caused it to be rejected except for those limited industries and applications where it is a good idea. Of course, most of those were already using the old way of doing this and only move to this where the new implementation is an improvement over the old way.
There are always companies who massively push this "new" approach because it is a great way to guarantee a steady income stream

Re:The Cloud is Just a Big Mainframe

[natoochtoniket]

There is one difference.

In previous decades, for the most part, the company that operated the computing center considered the data to be valuable, and took great care to prevent data loss. They knew that the hardware could fail, and so they made multiple copies of each data file. They did backups, and they checked and tested the backups. Most even stored some copies off-site to hedge against the possibility of catastrophic loss of the entire data center.

At present time, many young people have never seen data loss. Many people do not realize that hardware failure is even possible. If they make backups, they rarely check or test the fidelity or reliability of those backups. Those same people are administering the data center operations. Managing the disk farm, replacing failed mirrors, and making backups of customers data, are all activities that are part of the service. As far as many of the MBA types are concerned, all of those are just costs to be minimized.

A single disk might have a MTBF of 30 years. But a system that uses ten thousand disks will have a MTBF of about a day. (On average, a disk will fail somewhere in the system, every day.) RAID systems do not eliminate the issue, because simultaneous disk failure is possible. And a power-supply failure, fire, explosion, software failure, or employee can kill a whole bunch of disks all at once.

In my own organization, I want to know where my data is. I want mirrored disks to minimize the operational effects of common hardware failure, and off-line/off-site backups so we can stay in business after an uncommon failure. I want to review the backup schedule. I want regular verification of backup status. I want periodic audits of the backups, to be sure they really exist and that they can really be read. And, when the data is vital to the continuance of my business, that verification and auditing must not be outsourced.

Whenever your MBAs want to cut the cost of doing backups, you really should check with the underwriter of your business-continuation insurance.

Re:The Cloud is Just a Big Mainframe

[MosesJones]

Straight from the big blue school of "please buy our cloud". Some pretty significant differences using Amazon as an example

1) Self-provisioning - ever tried to get an LPAR commissioned on a a mainframe yourself?
2) Multi-domain failover - want an EU and a US instance? Want to manage failover between them? Yup you can
3) Speed of expansion - The number of LPARs and MIPS isn't fixed you don't have to fight to the death for another 1 hour slice
4) Separation of Storage and compute - You've got you on-compute disk and your off compute disk (S3) and you can scale them independently and without requiring IT support
5) Low lock-in. Yup the AMI is an Amazon thing but Linux and Windows aren't. You don't get locked into a single vendor like with big blue

I could go on. I speak as someone who has had to work in Mainframe environments and while there are similarities there are massive differences. Its like the difference between Green Screen and VDI, yup they are both "remote terminals" served from a central infrastructure but the operational differences are huge.

Now the question is whether a website that backs up your data (which is all Danger/Sidekick was) counts as a cloud. Of course it isn't, its just a website that does backups like we've had for years. What sort of redundancy was built in ? (answer: nothing) what sort of distribution and federation was built in? (answer: nothing)

The problem is that like lots of things people are just slapping a "cloud" sticker on a website just because it allows lots of people to do something. Danger/Sidekick/Microsoft had a WEBSITE which had a big old SINGLE disk with NO backups. People could sync their data with this SINGLE disk. The disk failed, it all failed.

Danger/Sidekick was as much of a cloud as a single Mainframe connected to a single SAN. Only in marketing world would it count.

Re:The Cloud is Just a Big Mainframe

[ducomputergeek]

In other works, Timeshare 2.0. Funny is I remember the old timers talking about that when I started in IT 15 years ago and exactly how well it worked then....and I've come to the conclusion it work just about as well now. The Cloud sounds like a great idea. I have friends who work at a company that switched to a thin-client/cloud system. It's great, until the network switch or router for a floor goes wonky. Then all of a sudden you have 200 employees dead in the water unable to do anything productive. At least before when the nets went down, they had local installs of the applications and usually could do SOMETHING productive.

Re:The Cloud is Just a Big Mainframe

[lennier]

"but there's nothing fundamentally different between cloud computing and IBM's vision of computing in the 1960s."

Yeah, there is.

The Cloud(tm) is now built by the lowest bidder instead of a single big company known for its boring reliability. You still can't see inside, you still can't get your data out, and it's still expensive, but now we've done away with all that needless safety and redundancy.

That's progress!

Re:The Cloud is Just a Big Mainframe

[Salamander]

Yeah, except for the part where you get to run your own OS. And the part where it often costs *less* than running your own servers. And the part where it's composed of multiple machines and can be scaled incrementally. And the part where you can integrate functionality between multiple clouds including those you run privately. And the part where you and everyone else access it through a network that hadn't even been imagined in the old mainframe days. Except for just about everything, you're right. Very insightful.

Re:The Cloud is Just a Big Mainframe

[drinkypoo]

Every time this comes around, though, it gains a little more ground because technological limitations have been eliminated, or at least pushed back. Arguably Web Hosting is much the same model, except that instead of moving your applications onto a mainframe, you're moving them to a hosting provider. To you, the hosting service looks much like a mainframe. Using a website is like using a mainframe in many ways, especially that webpages are request-based. There will always be an ongoing demand for both local and remote applications. Sometimes, communications just aren't practical. Today that's true if you're just in a remote location someplace; tomorrow it might be the case that you're just too far away from the server, out someplace in the asteroid belt where the speed of light becomes a practical consideration. By the same token, there will always be demand for client-server applications, when it's just not practical to access all the necessary data locally.

Re:The Cloud is Just a Big Mainframe

[Attila Dimedici]

Every time this comes around, though, it gains a little more ground because technological limitations have been eliminated, or at least pushed back.

I actually believe that the share of the computing market that this type of system applies to remains pretty much the same. However, every time the next wave of this comes around, some segment of the market discovers that they should be using it. So in a way it does increase each time, but only because industries/uses that advances in technology have only recently gotten computerized discover that this is the model they should be using.
However, if you are of toward the libertarian end of the political spectrum (as opposed to the statist end) you should be resistant to this paradigm each time it comes around because it is about centralized control rather than individualized control.
In my opinion, the true political spectrum runs from those who believe that the individual should have complete control over their actions and complete responsibility for the consequences of those actions and those who believe that their should be complete centralized control over people's actions and centralized mitigation of the consequences of those actions. Most "conservatives" on this board tend toward the individualist end of this spectrum, most Democrats on this board tend toward the centralist end of this spectrum. The "liberals' on this board vary as to where they fall largely because they have been trained to think of the political spectrum as being communism on one end and fascism on the other end, there is also a group opposite the "liberals" with a similar mistake (although I think that group is underrepresented on slashdot...not a bad thing).

Re:The Cloud is Just a Big Mainframe

[drinkypoo]

It's important to remember that the majority of people who self-identify as liberals are no such thing. They would, in fact, like to define morality for the rest of us; because they disagree with less of what the average person does, they think that they are liberal. In fact, however, they are overwhelmingly populist in their ideals, just as are the self-declared conservatives. They simply disagree on where the line is drawn. Libertarians, however, are those who want police protection from their slaves — they would like the right to take advantage of anyone. They are of course not anarchists; they [nearly/statistically] all believe in some government intervention in commerce, although many of them will deny this. The Socratic method is generally sufficient to point out their logical failings.

In any case, this is all a digression since politics is a response to logistics. A government is the best way we currently know to make the majority of people happy. And a client-server architecture is the only way we have to permit someone with a small computer to work on large data sets. If your data is encrypted, the only problem with storing it in "the cloud" is accessing it on your schedule.

I am mostly liberal, which means (in this case) that I believe in government intervention in commerce, but that personal choices should be left to people until they affect others. I also have a lot of radical ideas about how society should be structured for maximum fairness, but they are not really germane to this conversation.

Re:The Cloud is Just a Big Mainframe

[Attila Dimedici]

I am mostly liberal, which means (in this case) that I believe in government intervention in commerce, but that personal choices should be left to people until they affect others. I also have a lot of radical ideas about how society should be structured for maximum fairness, but they are not really germane to this conversation.

The problem with identifying yourself as a "liberal" is that it does not really say anything. However, from some of your other comments, you appear to be on the centralist side of the political spectrum. I am on the individualist side of the political spectrum.
Client/server architecture (which we seem to agree that cloud computing is a variation on) is about centralized control of computing (which you appear to disagree with). There has long been a conflict between centralized control of computing resources and individual control of computing resources. I believe we share some common ground on how we understand "cloud" computing and even the political spectrum although we are at different places as to what we believe is the correct balance between the individual and central control (and the importance of that method of organizing ideas).
Please keep this discussion in mind when we exchange ideas on other threads and I will try to remember your user id in mind so that we can continue to make constructive exchanges of ideas and build on this exchange (I know that is heresy here on slashdot).

Re:The Cloud is Just a Big Mainframe

[drinkypoo]

Yeah well, communications are either balanced or imbalanced, client-server or peer-to-peer. Sure, there's overlap, but it's clear that the cloud is simply a client-server model that abstracts computation and not merely storage. What's new is the way it is structured, and being able to submit arbitrary computation tasks to the server for return. That's where the real danger is; you can safely store encrypted data on the cloud, but there's no way to safely perform computing tasks there, where your data must exist (however briefly) in encrypted form before it can be manipulated.

It all depends on the meaning of 'failure'?

[Anonymous Coward]

Seems like we've heard that sort of thing before and look how that turned out

Re:For the love of God the company is called "Dang

[commodore64_love]

I never ever heard of Microsoft prior to 1992 (first time I used Windows 3). Prior to that the world revolved around IBM, Apple, and Commodore. Funny how fast things can change, and a small company can leverage itself to the top of the heap.

Redundancy scales

[rpp3po]

What has this got to do with the "cloud"? If your data is critical enough, do it in house or mirror/slave/backup across two or more vendors. The probability of chain failure at one vendor's site alone is much higher than when you use several. The required isolation and separation of your components will also benefit your overall architecture.

No true scotsman

[vadim_t]

This is awfully convenient. Something that at least to my eyes looks a lot like a cloud crashes. Cloud pundits announce:

"if it loses your data - it's not a cloud".

So if Amazon's S3 ever fails horribly and loses everybody's data, then it wasn't a cloud either.

Re:No true scotsman

[Anonymous Coward]

At this point, for S3, complete data loss would require the physical destruction or seizure of the contents of buildings in 3 distinct states.

Or, you can encrypt your data inside S3 and misplace the keys...

Re:No true scotsman

[Anonymous Coward]

That depends on how big a failure was required to lose everyone's data. If failure in a single geographic location loses any more than an hour or so (frequency of geographic duplication can be a contract-specified item, of course), then it's not a cloud. It's an organization of data centers all under the same billing company.

Re:No true scotsman

[lennier]

""if it loses your data - it's not a cloud"."

Which is quite a good way of putting it. We could say much the same about markets.

"If it loses your money - it's not a financial market." :)

There is a HUGE difference

[sribe]

That difference being that when you're doing things in your own data center, your own people can evaluate what's actually being done. With cloud computing you cannot do that. In both cases you have similar tensions between thoroughness and cost, but in the the one your company gets to make the decisions and verify that they're carried out, in the other you do not.

Is the distinction meaningful?

[FauxReal]

But some cloud technologists insist data center failures are not cloud failures. Is this distinction meaningful?

Do you think the customer will want to argue semantics with you after you've lose their data?

Re:Is the distinction meaningful?

[NickFortune]

Do you think the customer will want to argue semantics with you after you've lose their data?

If it was me, I could be tempted. Let's have a go at it...

To start with, I would have said that, technically, the cloud can't meaningfully fail at all. I suppose it could be down when you needed to access your data, but the connectivity part of the deal (which is the actual cloud) is going to come back up.

The trouble is that no one buys the "cloud". They buy services delivered over the cloud. So the question isn't about whether the cloud is reliable as such, so much as it is about whether the datacenters the cloud connects to are reliable.

So I'd say the distinction is meaningful, but not helpful, unless you're specifically discussion downtime which we aren't in this case.

Furthermore, nothing about the distinction makes cloud computing sound any more attractive to me than it did before I encountered it. Clearly some marketing type thinks they can put a bit of spin on this, and make it sound less like an indictment of the industries latest hot buzzword. If so, I don't think it's working.

Re:A reason why cloud computing might be hated on

[Anonymous Coward]

I don't think that has anything to do with it; at least not for me. My main concern with cloud computing is trust. Do I trust someone other than myself to not fuck up and lose all my data? For critical data, the answer is no. If somebody is going to fuck up and lose all my data, it's going to be me. I don't know if all the data on a Sidekick would qualify as critical, but it would certainly be annoying as fuck to lose it all.

An epic fail, and missed lessons (so far)

[rickb928]

I'm a TMO subscriber, and I love them, so this is painful. And my sister-in-law is a longtime Sidekick user, so she's in a special agony.

But T-Mobile is in a potentially no-win situation. They obviously have to believe Danger/Microsoft that they have good processes to avoid and recover from such failures. They didn't, and now TMO is probably going to take the hit. On one hand, they should - if the service is important, take responsibility and ensure management. On the other hand, they have good assurances, so hey, how much is enough?

BlackBerry users, you should take note. Rim differs only in scale. Ahd, you hope, depth of resilience. Not that RIM hasn't had outages, though not total failure yet.

TMO may have to tell their Sidekick users to be prepared for the inevitable restore, and of course, work with Danger/Microsoft to re-establish service (even though they don't provide service, D/M does), and of course some money compensation no matter how inadequate.

And maybe offer them shiny new myTouch3Gs to give the disillusioned Sidekick users an option with a marginally better track record.

No, wait, that isn't right. I've had to wipe my G1 every update, and some apps don't have a way to save data. They just don't.

I'm glad I never got on the Sidekick train, but I have no hope that this won't some day hit me. Do you suppose the next major Sidekick update will include data backup? :)

Re:An epic fail, and missed lessons (so far)

[R2.0]

"But T-Mobile is in a potentially no-win situation. They obviously have to believe Danger/Microsoft that they have good processes to avoid and recover from such failures. They didn't, and now TMO is probably going to take the hit. On one hand, they should - if the service is important, take responsibility and ensure management. On the other hand, they have good assurances, so hey, how much is enough?"

3 words: "Trust, but verify"

Re:An epic fail, and missed lessons (so far)

[Anonymous Coward]

Blackberry users would be unaffected by data loss since all Blackberry devices can have their data backed up by the end user.

Re:An epic fail, and missed lessons (so far)

[Anonymous Coward]

I have had a G1 since February 2009, and I have NEVER had to wipe my G1. I've been through 2 or 3 updates.

Re:An epic fail, and missed lessons (so far)

[rickb928]

After 1.6 (Donut) all Search activities caused an aCore force close - even searching within browser, the widget, or an app that supported global search. The TMO rep I spoke with confirmed it also affected his device.

After 1.5 (Cupcake) Google Maps force closed, the udpated Maps also did. Power Manager Free force closed as did many other apps, and because of these apps even uninstalling them did not resolve the issues. After a week of deathly slow performance and losing >35MB of space that could only be reclaimed by uninstall/reinstall of several apps DAILY, a wipe fixed everything. Three TMO reps said wipes should be expected. My version of akNotepad could not save to the Sd card, and I lost all notes. And I lost all SMS messages and call logs.

RC33 also caused me a wipe, Power Manager had to be reinstalled but still force closed.

You may have escaped, but I'm just sharing the reality of my experience.

Re:An epic fail, and missed lessons (so far)

[Chuckstar]

Blackberry data flows through RIM servers, but does not reside there.

Unauthorized backup

[thijsh]

Didn't Paris Hilton already find a backup solution for this?

Re:Unauthorized backup

[Megane]

I'm pretty sure I heard her say that she got the idea from "that guy with the blanket in the Charlie Brown comics".

Cloud failure? Microsoft failure.

[RR]

This is a service run by Microsoft. Microsoft is a bit hostile to consumers. It would be ironic and sad if Microsoft's failure to maintain the Sidekick service gets blamed on the faceless "Cloud" and it hurts Microsoft's competitors.

Control and failures

[Culture20]

People don't like (potential) failures that are out of their control. According to statistics, my data on my personal HDD or phone is more likely to fail than data on an average "cloud" storage array, but I can keep my HDDs and phone from harm and monitor and test backups. Same idea with automobiles versus airplanes: cars are supposedly more dangerous, but we're each our own pilot. Airplane crashes are scary partly because of the size, but also because in an emergency, we know there's nothing we can do.

Vaporous Program or Data?

[Atraxen]

It seems to me that the issue lies in whether the data pieces are on the cloud, or if just the programs are. If I lose the ability to edit a Word document from Office-For-Cloud but I have the file stored locally, I grumble that 'the idiots who run the thing' broke the program, and wait for the 'smart guy white knights' to come fix it for them. But in this case I'm holding those bits (exclusively, or a copy) so I know the data are safe. Nuke the server from orbit, for all I care - I'm annoyed that I lost the ability to continue working, but I've only lost time (bad enough, I know...) Downtime length and frequency becomes the only factors to my unhappiness

If the whole thing is on the cloud without a user-held copy, my SuperImportantLifeWork.doc can turn into vapor if the worst case happens. Now, we add a new factor - what files I lost, and what's involved in regenerating them. This is the predominate factor in my user unhappiness - phone numbers are hard enough to pull together again for many of us, but when we expand that to everything else on the phones (or extrapolate to what may eventually be on-cloud - pictures, documents, schedules, patient data, etc.) these losses become more catastrophic.

In the end, we usually hear about the same set of factors being important for 'good' backups - different physical hardware, offsite, different power system, geographically-separate, etc., in something like that order (depending on data, usage, etc.) These companies really need to make sure that the user has the opportunity to implement these factors by maintaining a complete (or optionally partial) copy of the data local-to-user.

Re:For the love of God the company is called "Dang

[commodore64_love]

>>>I miss my Sidekick immensely.

For some reason this sentence suddenly reminded me of this youtube video of two sisters arguing over a Sidekick (fastforward to 1:40) - http://www.youtube.com/watch?v=kcFUDvTFokg

This video is also fun to watch http://www.youtube.com/watch?v=STtL5EJFUgE

predictably doomed

[jipn4]

Danger held your data hostage from the start and didn't provide backup. Then, when Microsoft took them over, it was clear that they were going to mess with the service and servers. No backup + Microsoft mucking with the servers = kiss your data goodbye.

But that's no more an indictment of hosted services or "cloud computing" than a Windows BSOD is an indictment of desktop computing. Microsoft screwed up, and quite predictably, too.

"if it loses your data - it's not a cloud".

[John Hasler]

Just define away your problems. ROFL.

Re:"if it loses your data - it's not a cloud".

[TubeSteak]

Just define away your problems. ROFL.

No true cloud service would lose data.
http://en.wikipedia.org/wiki/No_true_Scotsman

Maybe we need a Cloud (R)(TM) brand to separate between "cloud" and "no really it's the Cloud" services.

If You Want Something Done Right!

[Prototerm]

Why on Earth would you trust your valuable data (and if it wasn't valuable to you, why keep it in the first place?) to someone else, someone who doesn't answer to the same people you do? I have always thought that "the cloud" is an epic fail waiting to happen. As a concept, it makes no sense. It's a scheme worthy of Professor Harold Hill himself.

You want your data safe? You want it backed up properly? Don't want to lose it? Then put it on your own hardware and take care of it yourself. Don't leave it to someone else to save your bacon when something goes wrong. Because, in the end, they don't care about you. You're just a monthly fee to them, and the agreement/contract/whatever you signed with them absolves them of all responsibility.

Re:If You Want Something Done Right!

[david_thornley]

Your ideas are excellent for computer geeks and medium-sized companies on up. For most people, they fail.

Most people don't know how to back up data, or design a bridge, or build a television. This is a civilization. We have specialization of functions, because there's so many different things to do that one person can't possibly know how to do them all.

The only solution for most people to accomplish most things is to rely on an expert. I'm the person with the most at stake if my car malfunctions, and I take it to mechanics for maintenance and repair. I'm the person most affected by my health, and I go to a clinic to find out what to do in certain situations. I can handle my own data backup, because I'm a computer geek, but the typical doctor or mechanic isn't, and can't.

What we need is some sort of certification for online services. I'm not saying certification should be required to operate (doctors need certification, mechanics don't), but that it should be possible for a cloud provider to get an audit by a reputable organization and advertise that. If a hosting company could guarantee the customers' data, presumably they could get business. This could involve verification that they do frequent backups, that they have offsite backups, that they can restore from backups, and that the data is available at some sort of escrow service, among other practices.

Re:If You Want Something Done Right!

[elnyka]

Why on Earth would you trust your valuable data (and if it wasn't valuable to you, why keep it in the first place?) to someone else, someone who doesn't answer to the same people you do? I have always thought that "the cloud" is an epic fail waiting to happen. As a concept, it makes no sense. It's a scheme worthy of Professor Harold Hill himself.

You want your data safe? You want it backed up properly? Don't want to lose it? Then put it on your own hardware and take care of it yourself. Don't leave it to someone else to save your bacon when something goes wrong. Because, in the end, they don't care about you. You're just a monthly fee to them, and the agreement/contract/whatever you signed with them absolves them of all responsibility.

You have never worked in a large scale enterprise that stores terabytes (or even petabytes), moving 100's of gigabytes (or even terabytes) every single day, have you?

Every large organization (and many midsize ones) maintain most of their computing muscle and data on externally managed data centers (or proprietary data centers managed by 3rd party staff.) You can't just do it on your own, and if you can, it is awfully expensive.

This doesn't count the inevitable politics and rivalries that arise when you take a DIY approach. You end up with data teams, network teams, IT ops teams, all of them competing for resources (and sometimes prestige). Unloading most of that wild west crap out to a 3rd party that is only interested in sticking to SLAs to the letter frees a lot of time and $$$ to take care of actual business/business-related technology needs. Disasters like these are actually extremely rare, and it seems to me it was more about the execution of the idea than on the idea itself (which has been done for years, decades without major problems.)

To the customer, it doesn't matter

[s.d.]

The people running the cloud and the data center can bicker till the cows come home, but to the customer, someone says, "trust me, I can let you run your apps and store your data better than if you did it yourself," and then *poof*, it's all gone. Since the customer only interfaces with the company managing the cloud services, the customer sees it as a cloud services failure.

If the cloud company wants to tell all their customers, "It wasn't our fuck-up, it was this other company that we pay to store your data," that's kind of a cop-out move in my book, but ok. However, since the customer will still see you the data center people as working for the cloud company (since it likely approached them and sold them the services as itself, not as "a team of companies X, Y, and Z working together, each doing specific tasks as defined below,"), the cloud company still screwed up and the customer is going to take their business somewhere else next time.

Definition of terms

[Anonymous Coward]

AC'ing due to rumor and innuendo, and completely unconfirmed insider info:

There's 3 terms being interchangably thrown around as "Cloud" here, so let's back up and make sure we're all talking about the same thing.

1. Managed hosting - Traditional "the hosting provider owns the box and runs your code on it" outsourcing.
2. Cloud hosting - A large cluster of virtual machines running on a platform that 100% abstracts hardware, such as vmotion, combined with per-minute billing and web-based provisioning. A marketing term coined by Amazon for their hosting service. This is insanely lucrative, by the way.
3. Offsite storage managed for you by a service provider, typically built on resold managed hosting or cloud hosting.

This is clearly a failure of cloud definition 3. So the question here is: Should you ever trust your data to a single outside provider? Of course not. Putting all your eggs in one basket has been a bad idea re: storage for as long as we've had computers. The first rule is always MAKE BACKUPS. You don't trust your disk, you don't trust your backup disk, you don't trust your live data, you don't trust someone else to back up your live data. The pitch for cloud has never been "We'll keep your data safe." It's been "We'll make your data available."

I'm going to come down on the side of two bad practices: First, T-Mobile made it very, very difficult to get your personal data off of a SK. It was a conscious business decision, designed to keep the barrier to migration onto other platforms / carriers high enough that the average celebrity SK owner wouldn't bother. Second, scuttlebutt is that T-Mob/Danger/MS lost all of this data because they brought in an outside consultant to upgrade the microcode on a SAN controller, which went wrong, leadingto a cascade failure.

If true, this means that a national carrier with hundreds of thousands of users' worth of data, if not millions, did not have a DR site available. If all the information was on a single storage array, then they didn't even have segregated databases on physically independent storage hardware.

That's a failure of architecture, a failure of engineering, and a failure of management. There are known best practices here when dealing with customer data, and a failure of this scale indicates that T-Mobile/Danger followed none of them. I simply can't think of a single reason as to why they're unable to restore from an offsite backup, unless those backups doesn't exist.

Sort of

[Kirby]

Well, any time you're storing data in a central place, you have a greater consequence of failure. That's a downside of "cloud computing", or any web application that stores data in a database too.

The alternative approach is everyone to have a local version of their data, which will be lost by individuals all the time but not by everyone all at once.

Obviously, if you have a server that's a single point of failure for your company, and you botch a maintenance, something went very wrong. And not having a backup - it seems strange for a company that's been around the block a few times and has big resources behind it. You have to write this off as more of a specific failure and not a failure of the concept of storing data on a remote server.

I do have a good friend that works for Danger - I really don't envy the week he must be having.

Re:Sort of

[elnyka]

Well, any time you're storing data in a central place, you have a greater consequence of failure. That's a downside of "cloud computing", or any web application that stores data in a database too.

Nope. That's the downside of any type of supposedly distributed system (including cloud/web apps) that stores data (or run systems) as single point of failures.

A cloud should be redundant, up to the database - a true cloud would even have redundant data centers (think google/ebay/yahoo infrastructures).

Moreover, and independently of whether it is a cloud or not, it should have a clear disaster recovery process. I've worked in systems were production runs in hardware in a data center in one state, and an identical/near identical hardware (usually running as UAT) in another data center in another state. Exercises were conducted routinely to demonstrate the ability to migrate a *failed* production environment to the 2nd environment, or even to rebuild it from scratch from tape backup (with tests on backup procedures done on a regular basis as well.)

This would include migrating app servers, http servers, database clusters, verifying firewalls let things go in and out as necessary (and no more than that), changing DNS entries to point to the new environments, etc, etc.

The only problem with a cloud environment is that the customer might not have visibility to see these things - ergo, the need to have the ability to back up your stuff off the cloud service. Obviously that was not what happened here.

This was a major foobar for all parties involved - you don't put your life on a cloud (or whatever) if you don't have a way to pull it back out with ease for the purpose of making backups.

Re:Sort of

[natoochtoniket]

There are two conceptual problems here:
One is storing all of that data in a single server, which could possibly fail and destroy all of the data;
The second is storing all of that data under the management control of a single MBA, who could possibly make a very dumb decision and destroy all of the data.

It appears that the second of those actually occurred.

Re:For the love of God the company is called "Dang

[sopssa]

Tip: If you want to link to specific part in youtube video, you can add #t=1m3s etc on it, ie http://www.youtube.com/watch?v=kcFUDvTFokg#t=1m40s

Also adding &hd=1 gives hq/hd version.

Re:For the love of God the company is called "Dang

[NoYob]

Didn't that throw up any red flags for ANYONE?

The company was probably named after someone's middle name. Like Austin _Danger_ Powers.

I'm sure it's no reflection on the software.

Sidekicks lack non-volatile storage

[James McP]

That's the real killer. Even if you had all your data loaded on the phone, lose power and poof! With no mechanism to make local backups, you're utterly at the mercy of the cloud.

I've got a Pre, which is a cloud device, but if my battery dies the same time as the remote servers my data's safe for quite a while. Once the battery recharges I can get one of the sync apps to offload my data. If I were more paranoid I'd get one now but I try to make my own archives straight from Google, webmail, et al.

Re:Sidekicks lack non-volatile storage

[John Hasler]

And yet people evidently stored important data on it.

Idiots.

Clouds are unaccontable, and that's their weakness

[Anonymous Coward]

"No, I'm saying that cloud storage services are engineered to tolerate failure."

Oh, really? Take a completely new, mostly proprietary set of code managing a huge datacenter, and I'm supposed to -assume- these clouds are engineered to tolerate failure?

The lesson of the Sidekick failure isn't a failure of "cloud computing", or of "bad backups", or of "old datacenters". As usual, everybody misses the real problem here. It's a disturbing reminder that reliability is completely dependent on people that you are hoping are running these networks correctly.

That's where the Cloud services fail. The difference between a cloud and running my own server is that I know when my own server is being run correctly, because I can check everything on it and physically inspect and audit the datacenter it's placed in. The Cloud services promise that they do the same, but all I can do is trust them, because their process is completely transparent to me.

People need to start demanding proof that these "Clouds" are being run correctly, and that's the hallmark difference between good engineers that know how servers work and fat nerds that jump on the hype bandwagon, becoming apologists for big companies that I hope are receiving bribes for their blind and unquestioning loyalty.

As for your comment on "occasional blackouts", we run millions of dollars through our company. Our servers should have NO blackouts, at all. With a good server cluster and a real datacenter with generators and redundant internet connections, this is a very achievable goal.

Cloud Computing

[snspdaarf]

The best part of TFA is the comment below from their version of an A/C:

Cloud architecture shards data

In this case it certainly did.

TOS

[ei4anb]

The TOS probably made the users aware that "your data is in Danger" so they can't complain now :-)

Re:TOS

[symbolset]

They may want to change this page then. This looks like a promise:

Data is always synchronized and backed up

Danger-powered devices are always connected to the Danger service. All user data is automatically and securely backed up over-the-air, and emails, photos, and organzier data are automatically synchronized with a Web-based application. All changes that are made on the device are instantly and automatically reflected on the user's computer, and vice versa.

And if you're wondering, yes, archive.org does have a backup copy of that page.

Bottom line, for me?

[FlyByPC]

Cloud computing is trusting Someone Else to take care of your data. While there are good, trustworthy organizations out there, for me, it comes down to the old adage of "if you want to ensure something is done right, do it yourself."

Networks are great for communication, collaboration, and sharing information not available locally (Wikipedia, online scholarly journals, etc) -- but for me, putting word processors online doesn't pass the laugh test. No matter how reliable your network is, if you already have a local computer (and a local computer capable of word processing is trivial these days), why would you introduce another possible point-of-failure by making everything go over the network?


And also -- why name a computing company "Danger"?? That's like naming a cruise line Titanic Cruises, or naming an airline after the Tenerife disaster!

Re:Bottom line, for me?

[DragonWriter]

Cloud computing is trusting Someone Else to take care of your data.

No, its not.

Paying someone for hosting your data is trusting someone else to take care of your data, which you can do with or without cloud computing (hosting services were popular well before cloud computing.)

Cloud computing is abstracting away the physical servers in doing logical server partitioning. You can do it completely on hardware you control in your own datacenter (Ubuntu Server, from 9.04, comes with a stack of software for cloud computing, and there is other software available, as well.) You may choose a hosted cloud, just as you can choose a hosted traditional server, that someone else owns and manages, but that isn't essential to cloud computing any more than it is to traditional server computing.

if you don't control your destiny, you will fail.

[swschrad]

not just stuffy history book stuff or national security, IMPHO it fully applies to "the cloud."

if Microsoft can't even build a robust cloud environment, that experiment is done.

"danger," indeed.

Tinfoil Hat Time

[nick357]

This seems fishy to me.

Microsoft wants you to use your own copies of Word, Excel, Exchange server and the rest, so that you do not have to trust in cloud computing. They want to sell you (or rent to you) software that allows you control your own data. They may be hedging their bets with some of their latest cloud offerings... but really - they'd prefer to rent you software that allows you to save your data on your own network and your own hard disk.

"Before finalizing your decision to move away from MS Office to an online service, perhaps you should review some of the hazards of trusting your data to others. There has been some recent events that might cause you to want to hold onto your own data"

Hehehe... just kidding - I am sure there is nothing underhanded about the whole thing.

Lightning bolt: Microsoft's gutting of Danger

[Locutus]

Microsoft gutted Danger and left it on life support but all the while they lead their customers( T-Mobile and users ) to believe Danger was thriving and doing fine. Wow, doesn't that sound like Paulson in early 2007 having stated that the banking system was just fine? The difference, Paulson really was clueless while Microsoft knew darn well they'd pulled most of Dangers developers over to their project Pink.

This is what should be up in lights with flares and fireworks and not anything about how bad/good cloud computing is. But once again, there is Microsoft at the wheel and yet the press is saying "pay no attention to that man behind the curtain".

And this interesting in tying this to cloud computing sounds eerily familiar since I just read how Steve Ballmer was bashing IBM for not running their business correctly. Basically, paying too much attention to software and cloud computing and he's all amped about this right when yet another Microsoft failure proves how bad they are at this. Could be spin control so watch for more of the same if it is.

LoB

Re:A reason why cloud computing might be hated on

[Jezza]

Mod the parent up!

There are two sides to this (at least). If you're moving your data "to the cloud" you'd expect that "the cloud" is one hell of a lot more reliable than you are. Let's face it, they should be - the economics of scale mean it's a lot cheaper for them to host your data and lots of other's data, than it is for you alone.

But that isn't what's happened in this case, here Microsoft (!) haven't even covered the basics. This is stunning.

So does this call into question "cloud computing" or just Microsoft's "cloud computing"? This is a difficult question to answer, without being able to see for yourself your cloud partner's infrastructure and procedures you can't really be sure... But would anyone make such a foolish mistake? Microsoft have proven that the answer is "yes, if it's Microsoft", the real question is should that be just: "yes"?

I think most of us now want a more hybrid approach, "in the cloud" is nice, but I also want a "local copy".

Then you have to think about the other kind of "lose" where others gain access to data they shouldn't see...

The cloud is irrelevant to this problem.

[DragonWriter]

But some cloud technologists insist data center failures are not cloud failures. Is this distinction meaningful?

Of course its meaningful. If you have a local server that you own, and you choose not to back it up, and it fails with a complete loss of data, that isn't primarily a problem with owning a local server, or with the particular server operating system (though there may be factors associated with either of those that contribute to the crash), its a problem with you choosing not to back up data.

If you have a single traditional server that you pay for access to, but is owned and managed by someone else with your data, and the same thing happens because you didn't assure (e.g., via contract) that the server would be backed up, again, the problem is with your failure.

If you have a cloud using one or more local physical servers that you manjage (e.g., using the cloud software included with Ubuntu Server), and the same thing happens, its not a problem with either cloud technology in general or the particular cloud technology you used, again, its a problem with your choice not to back it up.

If you instead pay for the use of someone else's cloud to host your virtual server instances than either you should be backing them up (if you manage the virtual servers, even though the vendor will be managing the physical servers and the cloud software) or you should assure that the vendor managing the virtual servers is backing them up (if a vendor, either the cloud vendor or someone else, is doing that for you.) If the servers aren't backed up, its your fault, and not the (general or specific) cloud technologies fault.

Or does the cloud movement bear the burden of fuzzy definitions in assessing its shortcomings as well as its promise?

There's no fuzzy definition involved here. The problem is quite simply one of failing to plan for recovery in the case of failure. This is a need that is independent of whether your logical servers are identical to your physical servers or whether they are decoupled as is the case in cloud technology, and likewise independent of whether your physical servers are owned by you and located in your data center or owned by someoneone else and located in their data center (or any mix and match of ownership and location), and further independent of whether you manage your logical servers personally (or with your regular employees) or contract out for the management of them.

This is not a failure of cloud technology, it is a failure of the particular parties managing this particular implementation to do something to which the the use or not of cloud technologies is completely irrelevant.

Dan

[Anonymous Coward]

This brings up an interesting question on Cloud Computing. How does the Cloud know what to backup? And how does the system check the distributed backups are OK? Probably a thorny problem depending upon the Application Service.

Re:A reason why cloud computing might be hated on

[Just Some Guy]

This is an unforeseen hole in the bulletproof Gandhi mechanism, so I foresee a quick "GPL V3.1" to close this. And then all is well.

How is it a hole when people who don't redistribute code aren't required to redistribute the source that created it? If you maintain a local branch of my code and use it to process your data, more power to you. It'd be nice if you did give back your changes, but that wasn't the offer I made to you and I don't have any right to expect it of you. End-user licenses like the AGPL are dangerous hacks that'll get more bad press than they'll make up for with the minor community good they do.

Larry Ellison on cloud computing

[Anonymous Coward]

As always, cloud computing/hosting/whatever is a vague term used like any other buzz term. I just see it as a platform where the resources should be allocated automatically and the underneath system takes care of having those available.

I like Larry Ellison's (of Oracle) rant on the matter:

http://www.youtube.com/watch?v=8UYa6gQC14o

Void Based Computing

[j33px0r]

Maybe I'm a bit old school but "the cloud" was a term for the inconsistent path that data would follow as it traveled from one point on the net to another. We already have terms such as remote, off-site, WAN, out sourced, etc. to describe such an environment, each of which is more accurate for any particular situation. When I hear someone state "we are implementing a cloud based solution" I inwardly laugh because I'm applying the old definition. What we have is a lame advertising buzzword that has gone mainstream. I patiently await for the term cloud to follow the word bling into the...well, I can't say cloud, into the void. I vote we call it the void instead of the cloud.

Re:Not a cloud disaster, not a "data center" disas

[R2.0]

"This a contract and contract management failure: the contract with the outsource was probably written without specifying that they must do the backups, AND no one established any sort of audit (formal or informal) test to ensure that there _were_ backups being taken and that the outsourcer was performing according to the contract.

Too often, the MBA doing the contract thinks "there, that's handled" once they've gotten all the signatures on the dotted line. "There, backups are handled now" he thinks, because many business folk (not ALL, I don't think it's fair to generalize that far) see these kinds of things as milestones, rather than ongoing processes to be managed."

That goes for a lot more than outsourcing. Construction contracts are a good example.

- In the perfect world, all the subcontractors do their work in accordance with the contract, and project management consists of making sure the procedures are followed.

- In the almost perfect world, sometimes subcontractors don't do their work in accordance with the contract, so you write provisions in on addressing changes to the work and non-performance. The PM is still making sure procedures are followed, but now there are more of them and are more complex.

- In the real world, the PM monitors subcontractor performance and addresses problems as they come up. Why? Because 2 weeks before a building opening is too late to discover a contractor is 2 months behind.

causes of the meltdown

[viralMeme]

"According to some reports, the failure was due to a SAN (Storage Area Network) gone wrong at Microsoft's end. It is claimed that Microsoft does not have a working backup of some of the data that has gone missing from customers devices. The SAN upgrade is rumoured to have been outsourced to Hitachi to complete"

"Microsoft, possibly trying to compensate for lost and / or laid-off Danger employees, outsources an upgrade of its Sidekick SAN to Hitachi, which -- for reasons unknown -- fails to make a backup before starting"

Re:causes of the meltdown

[Anonymous Coward]

Like, when m$ did buy this dangerous thing, did they have intentions of keeping two mobile platforms in the house? One of their platforms has now been flushed down the toilet, another still remains. Blaming hw failure or outsourcing is just blatant..

Re:causes of the meltdown

[MrLogic17]

Outsourcing had nothing to do with it.
Regardless of who did the upgrade- Microsoft owned the servers, and didn't have usable backups in place. It's the owner of the server that's responsible for backups.

Would you give your PC to Best Buy's service dept without a backup?

Too cloudy

[Quiet_Desperation]

Don't know why there's no sun up in the sky
Stormy weather
Since my data and I ain't together,
Keeps deletin' all the time

Re:A reason why cloud computing might be hated on

[John Hasler]

> How is it a hole when people who don't redistribute code aren't required to
> redistribute the source that created it?

The argument is that allowing people to use the software by accessing servers over the Web while denying them access to the source is functionally equivalent to selling them binaries while denying them source. I think it is functionally equivalent to allowing them to use the software on your timesharing system via Telnet (something the FSF never objected to) but it is a point on which people can honestly disagree.

Google Gears, is it a solution?

[InterGuru]

I had installed Google Gears as a precaution against Google losing my email. Does this work. Gears does have a copy on your local computer. Is this sufficient?

New MS Datacenter in Chicago

[Metapsyborg]

Just today in the Chicago Tribune there is an article about a new Microsoft "cloud computing" datacenter in the suburbs. It goes on and on about how great cloud computing is and how visionary Microsoft is for their work in this field (*snicker*). They briefly mention some other companies, I think one called "google" and yahoo or whatever, that are following in Microsoft's footsteps into this brave new world of internet-based applications.

Given that, I doubt MS planned the Danger/Sidekick fiasco in order to discredit cloud computing. In fact I found it very amusing to read about the new data center and then just under it another article about an MS data center losing all of it's user data.

Of Course this is not a "Cloud" problem...

[Vitriol Angst]

A Datacenter is the backbone of a Cloud. Cloud Computing is 100% reliable -- until of course, when it fails, and then Marketing/Tech Support will tell you this was a datacenter problem and ask you if you saved a backup.

Re:For the love of God the company is called "Dang

[gnick]

You'd never heard of MS before 1992? MS-DOS was a pretty darned popular home OS through most of the 80's. Wikipedia tells me that it was the most popular DOS variant and the most popular personal computer OS at the time (which is what I remember too, but I was only 4 when it came out). You might not have known what MS stood for, but MS-DOS was everywhere. Where were you hiding?

Re:Management - We Saved Big $$$!

[Anonymous Coward]

It's symptomatic of the times, I think. "Git 'R Dun!" means provide the superficial essentials as fast and as cheap as possible and don't sweat the details. Once it's running, forget about filling in the missing stuff - all it does is make the product more expensive; and besides, "nothing like that's ever going to actually go wrong". Pat yourself on the back because you're so "productive". Rinse, repeat on next product.

Microsoft renders Sidekick data completely secure

[David Gerard]

Microsoft today implemented its 100% Data Confidentiality package for T-Mobile Sidekick, comprehensively protecting users' contacts, email and messages from any possible attacker.

"Our data security is impenetrable," said Steve Ballmer, "and will reassure everyone of the data integrity of our Windows Azure Screen Of Death cloud computing and Windows Mobile initiatives."

Microsoft plans to leverage the new confidentiality mechanism to finally purge the horror of Vista from the face of the earth, in the same manner as firing all the contractors who knew how to build Windows 2000 and having to reconstruct Windows XP from bits of NT 4.

Microsoft Sharepoint users looked forward to a similar denouement as the only safe way to scour their hopelessly incompetent organisations from the world in a manner that would not infect successor organisations.

Microsoft is putting together an outsourcing proposal to the UK government for data protection.

Illustration: Secure Windows data storage mechanism.

Re:A reason why cloud computing might be hated on

[sexconker]

You must be the GoTo Guy at your office.
Something went wrong? Blame AC.

Cloud had little to do with it.

[MrCrassic]

As I'm sure somebody pointed out here, the Sidekick data loss fiasco occurred largely because nobody had off-site tape backups on hand and nobody wanted to do a backup BEFORE performing their big upgrades.

It's that simple.

Re:For the love of God the company is called "Dang

[sexconker]

Unfortunately, project director Robinson ignored all the warnings. Had the company been named "Danger Danger", ol' Will would have paid attention.

Re:Not a cloud disaster, not a "data center" disas

[benjamindees]

Tech job outsourced to "cloud" provider == job done, bonus paid

Mortgage signed, packaged and re-sold to quasi-government agency == job done, commission paid

I am consistently amazed at the wholesale failure of the US capital markets to recognize and reward actual productive work. It's been going on for far too long now. I would say it simply is a case of the people who have money being complete idiots, but you'd think that would be a self-correcting problem. It probably has something to do with the people printing money and handing it out to idiots.

Re:For the love of God the company is called "Dang

[sexconker]

Adding &fmt=8 gives you the raw version as uploaded to Youtube.

Useful for when the damned server just isn't loading your video all the way - you can force your way through to a different server (since the original isn't propagated to all their distribution servers).

Re:For the love of God the company is called "Dang

[sexconker]

For some reason that sentence reminded me of the issue where Robin left the Bat Cave (both actually and metaphorically) to become Nightwing.

Re:A reason why cloud computing might be hated on

[Just Some Guy]

I think it is functionally equivalent to allowing them to use the software on your timesharing system via Telnet (something the FSF never objected to) but it is a point on which people can honestly disagree.

That's precisely my position. The AGPL proponents' ideas are ones that I hadn't heard before: I have to share code that never physically leaves my control? And that a I can run an AGPLed GUI application internally without obligation, but if someone access it via Citrix or VNC or Remote Desktop then I have to release the changes? That's pretty bizarre to me.

I'm officially lost.

[MrCrassic]

What exactly is the difference between today's "cloud computing" and yesterday's "internet-based services?"
 
  I'm sure this question is often asked, considering that every single web site is a file stored on a remote computer which by way of internet services is displayed on computer screens everywhere. Additionally, people have been uploading data to remote storage services since the late 90's with XDrive and its precedessors, but these were never known as "cloud computing" then...

Re:I'm officially lost.

[James McP]

"Real" cloud computing is supposed to be based on a mesh of geographically diverse, redundant servers each carrying various subsets of the data. Think RAID5 for servers, with each partition located in a different part of the world and on different networks.

Which means it is nothing more than an internet based service with five 9s of reliability and availability.

However it is an *expensive* internet based service so it needs a new moniker. But without a "Cloud Computing Consortium" with ownership of the trademark "cloud computing" to enforce correct usage, there's nothing to prevent everyone and their dog from using the term incorrectly for any "always connected" application.

The problem with all this is that its almost impossible for an end user to know for sure if someone really has a proper cloud application until something fails. If there is a total failure of a site and no one notices, you've got a working cloud. If people lose data or functionality, you don't.

Re:I'm officially lost.

[DragonWriter]

What exactly is the difference between today's "cloud computing" and yesterday's "internet-based services?"

"Cloud computing" is a catch-all name for technologies for building systems where the underlying physical servers are abstracted and compute resources can be dynamically provisioned.

One common application of this technology is providing hosting for internet-based services.

The relation between the two is about the same as the relation between, say, "internal combustion engines" and "mail delivery services".

Exactly

[Lysol]

This is headline trolling. The 'cloud' is just a term used to describe what's already been around for over a decade. This has nothing to do with 'the cloud' and everything to do with bad infrastructure policy and incompetent IT staff.

Re:if you don't control your destiny, you will fai

[benjamindees]

if Microsoft can't even build a robust cloud environment, that experiment is done.

The "experiment" that is Microsoft, you mean?

Re:A reason why cloud computing might be hated on

[MightyMartian]

And that's the side of the fence I sit on. You really can personally guarantee nothing about something sitting on someone else's server, particularly if you're using their apps to store and alter your data. Maybe they're backing up the data, maybe they're not, maybe the data is secure, maybe it isn't. Sure they can hand out guarantees, but so what?

It strikes me that there is a long list of cloud companies who are waging a serious PR campaign to undermine some of the core rules of IT. We see it a lot, with articles by industry shills declaring IT is dead, that no one needs much more than a dumb terminal capable of rendering web pages, that the age of the PC is dead, that corporations can fire most of their IT departments because Google/Microsoft/who-the-fuck-ever will look after their needs.

There are in some cases solid reasons for going with an online service rather than trying to run your own, and I won't deny that. But before anyone embarks down that path, they need to understand the risks, and pick services that allow them to dump their data in an open and usable way (ie. GMail allowing POP3 and IMAP access).

#1 sign your data is going to vanish ...

[NotBornYesterday]

You notice the word "Microsoft", followed by the word "Danger".

Palm Pre FTW :-)

[Slashdot Parent]

The Pre syncs contact/calendar/etc. data with several popular online services (and Microsoft Exchange). It also performs an automatic daily backup (turned on by default) to Palm's servers. All of your important data is stored in 3 places. On the device, on the service provider, and on Palm's servers. (As an added precaution, I like to backup my Google contacts directly from Google on a monthly basis).

All this makes is extremely unlikely that a Palm Pre user will ever suffer an unintentional data loss.

Hope your sister gets her data back, but if she doesn't, she should upgrade to a Pre. :)

Re:For the love of God the company is called "Dang

[Anonymous Coward]

Mustn't have ever used a C-128 :)
http://home.datacomm.ch/fmeyer/c64/pictures/c128_scr.gif

Re:For the love of God the company is called "Dang

[commodore64_love]

Well first off, my focus was on the world of Commodores and Amigas, not IBMs.

Second I never used either DOS or Windows prior to 1992, and when people who owned IBM PCs discussed their machines it was usually in context of what "uber hardware" they owned, like SoundBlaster or Intel 486, rather than the software. I knew they had DOS but I had no idea who made it - I just figured it was IBM's DOS since it was running on an IBM PC. Electronic Arts and Activision made more of an impact on my 80s-era life than microsoft.

Microsoft FAIL, not Cloud

[Anonymous Coward]

You outsource to a company A.

Then company A gets bought by company B. Company B has historically opposed the existence of company A's entire industry. Company B wants the technological transition to that industry to occur as slowly as possible. Company B made the purchase for patents and personnel, not customers. Company B doesn't extend its own reputation to cover company A. Company B's concern for customers is... not a pillar of it's business model.

What part of company A having a high profile failure comes as a surprise?

Reliability is HARD. The legals say "don't blame us when we slip up". This need not be "let's smear cloud by having a big fail". Merely "we don't really care, some benefit, some cost, so we'll try somewhat less hard then we might if we cared". Delinquency, rather than evil. Though the border isn't clear cut.

Re:Not a cloud disaster, not a "data center" disas

[R2.0]

It IS a self correcting problem - the bubble burst was the correction. We just don't want to live with the consequences.

Re:Palm Pre FTW :-)

[rickb928]

"The Pre syncs contact/calendar/etc. data with several popular online services (and Microsoft Exchange)."

- Assuming you set those up...

"It also performs an automatic daily backup (turned on by default) to Palm's servers."

- We now know to trust the vendor's servers as far as we can throw them. History is no guide.

"All of your important data is stored in 3 places. On the device, on the service provider, and on Palm's servers."

Uh, 'on the device' is not a backup. That's the 'data'. Make that 2 places, more if you use multiple online services/Exchange.

"(As an added precaution, I like to backup my Google contacts directly from Google on a monthly basis)."

I save my POP mail to Google and Yahoo! daily or more often. Contacts ditto. I actually have 4 email accounts with copies of 3 spread around. I also export from my Outlook client at home, andthen distribute the CD ISOs around to three different places, all encrypted with the name of the file as the hint. I copy my SMS into Gmail, and from there it goes to 2 other servers, and is part of the Outlook export.

I have email so old I cannot really read it in anything but a text editor. Elm barfs on some of the old MBox stuff, which is probably just corrupted, and the old Eudora stuff is pretty much done for I think. I lost some AOL mail, but that was mostly noise and spam anyways.

Oh, and up to 2005 I saved spam. I have a great collection. Stuff you would have to pay for nowadays :)

Re:A reason why cloud computing might be hated on

[DragonWriter]

With Cloud Computing, those who modify FOSS software do not have to redistribute the code, because they are only providing a service and not a functional program.

There is no general requirement to redistribute code with modified FOSS software anyway. That's a feature specific to particular licenses (particularly, the GPL.)

Haven't you heard?

[Anonymous Coward]

Granted, I had tapes up the wazoo

Haven't you heard? It's spiders, now.

Re:Haven't you heard?

[DinDaddy]

I award you +1 wish-I-had-a-modpoint

Re:Palm Pre FTW :-)

[Slashdot Parent]

"The Pre syncs contact/calendar/etc. data with several popular online services (and Microsoft Exchange)."

- Assuming you set those up...

My usual yardstick by which I measure whether or not an average user might set something up is my estimation on whether or not my technologically-disinclined, AARP-card-carrying mother would set it up.

My mother syncs her Palm Pre contacts and calendar with google, so I am forced to conclude that the average user would do this.

"It also performs an automatic daily backup (turned on by default) to Palm's servers."

- We now know to trust the vendor's servers as far as we can throw them. History is no guide.

I guess I didn't make my point clearly enough. I don't give one tenth of one hoot if Palm's servers crash because my data is safely on my device and safely on google and safely on my personal server and safely backed up as part of my offsite backup procedure.

"All of your important data is stored in 3 places. On the device, on the service provider, and on Palm's servers."

Uh, 'on the device' is not a backup. That's the 'data'. Make that 2 places, more if you use multiple online services/Exchange.

I'm not sure your point, because I never claimed that the on-device data was the backup. At any rate, the overwhelming majority of Palm Pre users will have their data replicated to three different locations, and managed by 2+ different vendors. A data loss for your average user is extremely unlikely.

"(As an added precaution, I like to backup my Google contacts directly from Google on a monthly basis)."

I save my POP mail to Google and Yahoo! daily or more often. Contacts ditto. I actually have 4 email accounts with copies of 3 spread around. I also export from my Outlook client at home, andthen distribute the CD ISOs around to three different places, all encrypted with the name of the file as the hint. I copy my SMS into Gmail, and from there it goes to 2 other servers, and is part of the Outlook export.

I have email so old I cannot really read it in anything but a text editor. Elm barfs on some of the old MBox stuff, which is probably just corrupted, and the old Eudora stuff is pretty much done for I think. I lost some AOL mail, but that was mostly noise and spam anyways.

Oh, and up to 2005 I saved spam. I have a great collection. Stuff you would have to pay for nowadays :)

I'm glad that you have all that data, but do you find it useful?

I converted all of my old email to Maildir format, and my IMAP server maintains a fulltext index so I can actually find something when I need it.

And your sister should still get a Palm Pre. ;)

Five 9's?

[mbone]

But some cloud technologists insist data center failures are not cloud failures.

I have heard this before. Back when the phone company touted four or five nines reliability, and my office phone service died, I was always told that the reason for the failure meant it didn't counr against 5 nines. (For example, if you can call out, but no one can call in, it doesn't count, apparently. You can, after all, still reach 911.)

So, I always take claims of near perfection with a large grain of salt.

Not the "Cloud...."

[Hasai]

....Though I think it is a ridiculously over-hyped term for something that has been around since 60s-70s (think "time-sharing").

The problem was the consumers' assumption they could safely consign their info to a data-store operated by Microsoft.

data lost never reaches zero

[davidwr]

Assuming your backups are stored planetside, the next time the Vogons want to build a bypass, you can kiss all your backups goodbye.

On the plus side, you probably won't care.

Re:For the love of God the company is called "Dang

[Pascal Sartoretti]

I was a Sidekick user from 4/2004 until 10/2008. There had been only one 'catastrophic' failure in that time that left Sidekick users without data service for an extended period.

In other words, they already had one opportunity to question their reliance on this service. Lesson learned ?

Paying the stupidity tax

[gestalt_n_pepper]

Bottom line: Anyone's "cloud" is inherently insecure and it's reliability is outside your control.

WTF ?

[smoker2]

Or does the cloud movement bear the burden of fuzzy definitions in assessing its shortcomings as well as its promise?"

FUCK OFF !

Anybody with a brain does not use the cloud for anything important. This necessarily leads to the phrase, "don't trust the cloud". Which historically refers to "Don't leave your eggs in one basket", which means " look after your own". You can make the question as "fuzzy" as you like but the answer is still NO !

Re:Not a cloud disaster, not a "data center" disas

[innocent_white_lamb]

- In the real world, the PM monitors subcontractor performance and addresses problems as they come up. Why? Because 2 weeks before a building opening is too late to discover a contractor is 2 months behind.
 
In the real world, small businesses come up against larger hurdles (relative to their operation) than larger enterprises.
 
An independent clothing store currently rents a storefront across the street from me. In the spring of last year they purchased the vacant lot beside my building with the intention of having a new building constructed and being open in the new location by last Christmas.
 
Christmas came and went and now it's almost Christmas again and the building is nowhere near completed; only the outside (unfinished) shell has been built.
 
There's no comeback on the contractor who's doing the job; if the owner had insisted on writing some kind of specific performance penalties into the contract the guy simply would have refused the job -- there's lots of other construction work around here.
 
So the store owner is paying rent on his existing location, plus the materials, labour and ongoing costs (electricity, property taxes) for a new building that's nowhere near ready for him to occupy a year after it was supposed to be completed.
 
Somehow I don't think this situation would have occurred if the clothing store was [insert big name company here].
 
Same thing with computer services and small companies. A small business owner has very little leverage.

Billing

[symes]

If you can't trust your outsourcing partner, replace them or bring the work in-house

At this point it would be informative to know how T-Mobile backup their billing data...

sidekicks have been around BEFORE the cloud.

[recharged95]

Hence, they likely never followed the practices of cloud companies and elsson learned when the cloud started.

That's why the sidekick ecosystem failed. Bad design from the beginning (put users' data on the network with no real management/backup, outsource the heck out of it since they can't compete with cloud services and of course, do it the Microsoft way: servers and standalone networked computers), and piss poor management to modernize the system as new tech (i.e. cloud services like S3, GDrive, etc..) was available.

Conclusion: Management WTL! (not T-mobile as they we're the customer!)

Re:Not a cloud disaster, not a "data center" disas

[lennier]

"many business folk (not ALL, I don't think it's fair to generalize that far) see these kinds of things as milestones, rather than ongoing processes to be managed."

This is something which subtly nags at me.

We have this assumption in Western technological society that ultimately, we can break stuff down into linear-composable processes: things like milestones. We tick one box (acquire a piece of technology, install a system, research a subject, , that's done, we go on to tick another, and we just tacitly assume that ticking box 2 does not untick box 1. Our row of boxes grows in a straight line. Simple and manageable. That's what Management Science is founded on.

However, more and more, the realisation is growing on me that Life Does Not Work Like That At All, and that Reality(tm) has signed no contract which requires it to be linear. And in fact it's never been like that except in some very extreme special cases - which we've unfortunately seized on, because they're so nice to think about, as being The Way Things Are.

What actually happens, I suspect, is that each piece of technology we acquire, each piece of knowledge, doesn't simply ADD to the rest but at the very least MULTIPLIES - possibly even worse. So as we keep adding entities to our organisations or programs or social lives, we get an exponential explosion of potential interactions. And each of these interactions is a process which can potentially feed back into every other one. It's all a big messy chaotic graph which only (deceptively) LOOKS linear in places. And then suddenly, wham, an earthquake or a data loss event reminds us that actually, what we thought was a solved problem was in fact just a temporary straight line in a very kinky process. And we don't ultimately understand the systems we're creating - let alone the systems we didn't create, like the biosphere and our bodies and minds, which we inhabit and depend on for life.

I don't think this realisation has yet sunk in, even now. I don't know what will happen to our psyche when it does. Robert Persig in Zen and the Art of Motorcycle Maintenance points to the 'explosion of hypotheses' problem in science, where Popper falsifiability doesn't guarantee convergence towards 'truth' or that scientific revolutions will always be about smaller or smaller disagreements, but that we might get a divergent cascade of increasingly contradictory and not-yet-falsified ideas. This might be arguable, but I think we're now seeing a similar problem in systems administration. The Internet is increasing mutual connectivity between otherwise partitioned social and computational systems; we fundamentally do not know and can't predict how these systems will fail.

We can't even be sure that currently shipping software is free from major security holes; in fact, about all we can predict is that all our software doe have deeply dangerous flaws. Not just small, linear, flaws. Big nonlinear ones which can hit us like a data crash, or a financial crash, or getting pwned by a botnet, on a global scale. And yet we use it, because it's there. We think we can trust our systems because other people are. But that's not true at all.

A big rethink for management techniques is ahead of us.

A little off topic

[Anonymous Coward]

I interviewed with Danger for a senior storage admin position about 2 weeks before the announcement.
Fortunately for me I had one of those major fail interviews. We have all had them, when asked your name you even get that wrong.
Anyway, this is clearly a case of poor planning and I would have to say poor management.
At the end of the day that (more often than not idiot) person with CIO in their title should have directed the staff away from a disaster like this.

This could have and should have been avoided.

Ah good...

[Builder]

I'm _not_ the only one to have horrific experiences with Hitachi SAN rigs then. Not that this helps me sleep better, because they'll still probably blow up the next scheduled work again.

ED Handbags,Armani jacket for sale cheap now go

[Anonymous Coward]

            Welcome TO Our Website: Http://www.tntshoes.com

        we are a prefession online store, you can see more photos and price in our website which is show in the photos.
          pls find the more photos and the price for our product in our website,hellow see our website in the photos attached, on line shipping sotre, selling all kinds of brand new shoes,clothing, handbag,sunglasses,hats etc, if interested please email me by we are selling all brand new handbag, we take paypal as payment, . shoes Nike jordan1-23 $23-$28 free shiping.

  OUR WEBSITE:
                                                        YAHOO:shoppertrade@yahoo.com.cn

                                                                MSN:shoppertrade@hotmail.com

                                                                    Http://www.tntshoes.com