Slashdot Mirror
MS Says All Sidekick Data Recovered, But Damage Done
nandemoari writes "T-Mobile is taking a huge financial hit in the fallout over the Sidekick data loss. But Microsoft, which bears at least part of the responsibility for the mistake, is paying the price with its reputation. As reported earlier this week, the phone network had to admit that some users' data had been permanently lost due to a problem with a server run by Microsoft-owned company Danger. The handset works by storing data such as contacts and appointments on a remote computer rather than on the phone itself. BBC news reports today that Microsoft has in fact recovered all data, but a minority are still affected (out of 1 million subscribers). Amidst this, Microsoft appears not to have suffered any financial damage. However, it seems certain that its relationship with T-Mobile will have taken a major knock. The software giant is also the target of some very bad publicity as critics question how on earth it failed to put in place adequate back-ups of the data. That could seriously damage the potential success of the firm's other 'cloud computing' plans, such as web-only editions of Office."
Not just buzz, it's the future bro.
here the damage to T-Mobile is compounded by their tone deafness on customer support.
Best Slashdot Co
Well, to be fair, whoever said 'All data is lost' to the press should have been dragged out back and shot. They should have said 'We're looking in to how long it will take to restore data, and to see if there will be any problems' and left it at that for a few days.
Vintage computer games and RPG books available. Email me if you're interested.
It is hard for me to blame T-Mobile for the MS/Danger server / backups failure. Danger both makes the phones and runs the service, where as T-Mobile appear to be little more than common carriers and the customer service department. It is a bit unreasonable to suggest that T-Mobile could have prevented the outage. I mean it not like they could host the data somewhere else right? Sure they could have done a much better job handling the failure after it happened, much much better, but I just don't think they could have prevented it.
Hey, at least this fiasco took the heat off their crappy network for a while.
SJW: Someone who has run out of real oppression, and has to fake it.
Wow, this is a terrible blow for Microsoft. This might make people think that they produce unreliable products!
Worth repeating every time. Nobody cares if you back up your data. Take a blank server; take whatever it is that you store offsite. If you can turn the blank server into your production system then you are fine. If you can't then your strategy is failing. If you never try it then you are an amateur.
This incompetence is something far beyond serious for MS. T-mobile is a much bigger customer than almost anyone short of vodafone can ever hope to be. MS have been moving strategically into hosting servers such as exchange for many customers. If you're a CEO you should be calling your CIO in and asking him when he plans to be free of MS services. If you are a CIO you want to be able to answer "there's nothing business critical relying on MS services" by the time that meeting comes.
=~ s,(.*),<sarcasm>$1</sarcasm>,g if any_point_you_wish();
"That could seriously damage the potential success of the firm's other 'cloud computing' plans, such as web-only editions of Office."
I can't tell whether this is spin put on the summary by the submitter or some other third-party (because we all know submitters are, absent any editorial constraints on /., free to post what they want without attribution). That said, it's highly unlikely Microsoft will suffer from this. Wisely, they offloaded all responsibility the moment they created this entity known as Danger. They've effectively washed their hands of the entire affair, because it wasn't really a Microsoft problem in the end, but a problem with an affiliated company.
It is simply wishful thinking on the part of the submitter (or whomever) that Microsoft will be tainted by this deal. In all likelihood, Microsoft will simply walk away from their relationship with Danger, and it will be business again as usual.
The worrisome part about cloud computing is putting your trust in someone else's hands. But keeping your backup process internal to the company is no panacea either. Bad management practice is what led to the cloud screwing up, just like bad management practice led to in-house data losses at other companies.
How many of you guys generate your own power 24x7? C'mon, you're really going to place the face of your business in the hands of people running off the wire? Wire power. Feh! That wire could be going anywhere. Real men run their own generators!
Sounds silly, right? Of course, that's only because we're used to power companies running like utilities, government-regulated monopolies allowed to exclusively service the public with a healthy, dependable profit in return for low rates and universal service. In such an environment having your own generators for anything other than emergencies is paranoia. But wow, you start deregulating things and let the businessmen go nuts and it almost seems like you'd have to.
The real question with cloud computing is whether the companies are going to operate in a fashion that brings to mind steady, sober, dependable service like a local utility, like a giant rapacious corporation uncaring of human concerns, or like a fly-by-night dotcom. My personal opinion is that I don't trust these fuckers. Current company's situation is that we have a major software product we run our business on and the publisher got gobbled up by a bigger company and that company got gobbled up by a bigger one. The big company has decided to discontinue the product and have been slowly dismantling the team that supports it. We know we're going to have to make a jump eventually but the conglomerate could pull the plug tomorrow and we'd still be in operation. If it was a cloud app, we could be dead in the water.
Kwisatz Haderach
Sell the spice to CHOAM
This Mahdi took Shaddam's Throne
What? That's not so bad. I mean if you really wanted a conspiracy theory you could surmise that MS bought Danger with some knowledge of how in bed they were with T-Mobile. AND seeing that one of the major carriers that Google's Android is T-Mobile, MS purposefully destroyed data to strike out against T-Mobile for partnering with their sworn enemy. Right now, Ballmer is sitting in his evil lair over an active volcano, cackling fitfully while stroking a white cat. Now THAT is a conspiracy theory.
Now that I said it, it doesn't seem unpossible. I better call Hollywood.
What's up with all the editorializing in the summary? Danger was bought by MS only 18 months ago. What the heck has this got to with Office and cloud computing except wishful thinking by the submitter?
So... in a year and a half they shouldn't have toured their new acquisition and checked for basic things like:
1) Updated server software
2) Firewalls
3) Backups
And other "yer an idjit if you don't do this" kinda stuff?
For *any* kind of hosted service, having backups measures just slightly below "is it turned on" in terms of importance. And for a year and a half, NONE WERE DONE? Further, they did a major update to a SAN and didn't backup first?
This isn't about bashing Microsoft - highly successful businesses have had to close shop forever due to glaring, horrid oversights like this. This is gross incompetence.
I have no problem with your religion until you decide it's reason to deprive others of the truth.
Given how much of our internet access is being spied on by the government, how could ANYBODY want to trust their critical data to a cloud service? Sounds like Microsoft has Cumulonimbus clouds.
Er... because it is a form of cloud computing which failed? When a failure like this occurs, it rightfully raises doubt as to the reliability of other cloud computing services, one of which happens to involve office.
Sigs are too short to say anything truly profound so read the above post instead.
Dear Sir or Madam,
The responsible Anti-Microsoft Troll that should have replied to this post by now is on sick leave and was unable to prepare a custom flaming reply to this particular post. In lieu of that, attached is our generic template which we use to write all our flaming responses.
1. Make a general anti-Microsoft jab
2. Blame Microsoft for it's stance against Free Software (and also for lack of network neutrality, the current state of patent laws, the Iraq war, and the extinction of the dinosaurs)
3. Accuse the poster who wrote something positive about Microsoft of being either a fanboy or a Microsoft employee. If the poster in question made a comment about Microsoft's actual support of Free Software in a particular instance, accuse the poster of being an oblivious idiot unable to see through their Embrace-Extend-Extinguish approach
4. State that the Linux revolution is inevitable
5. Finish off with another outpour of flames
We hope you will be able to infer the potential content of the post that should have been done by the respective Troll. Please accept our apologies.
Sincerely,
Assistant Secretary,
Anti-Microsoft Trolling Association, Ltd.
Years of BSODS.
Years of viruses.
Years of trojans.
Yet THIS "damages Microsoft's reputation"?!?!?!
Well, it's a bit of a non sequitur, to be sure. But the whole incident spells out in stark detail the dangers of "cloud computing", or as us folks who actually have worked with computers for more than than ten minutes call it; the client-server model. When explained as what it really is, it's a matter of ensuring adequate and timely backups. When described in some pathetic marketing term, it sounds like some magical new way of computing, no longer constrained by those old-fashioned good practices.
Quite frankly, I would never ever ever put any mission critical data or apps on a system that I couldn't back end the data on my own out of. If I can't move my data out of the app, then my data never gets there in the first place.
The world's burning. Moped Jesus spotted on I50. Details at 11.
You really don't see the connection?
Yesterday, you put all your cell phone contacts and calendar data up in the "cloud".
Today, your data is lost.
Tomorrow, the same companies responsible for losing your cell phone data now want to take over all your Office documents.
Well, since this is /., you take your car in for a routine oil change. The mechanic botches the job.
Are you going to go back to the same mechanic for a transmission rebuild?
Cloud computing and remote storage are not necessarily the same.
What we see here is a small device storing it's data remotely and I wonder why.
Considering how cheap a couple of GB of memory are and how precious wireless bandwidth is this can mean only one thing, having and thus exploiting that data is worth more than the cost of the bandwidth.
"The likes of Facebook and WhatsApp are free to those whose privacy is of zero value."
If T-Mobile plasters their name on the contract, the device, and the service, then the buck stops there. Period. Internally, T-Mobile can choose to blame the Easter Bunny if they like, but ultimately, it was T-Mobile's responsibility to ensure that their customer's data was properly protected. This absolutely could have been prevented by audits of Microsofts/Danger's operations, checks of backup integrity, tighter contracts, etc. T-Mobile can go try and sue MS to get their damages back, but in the meantime, customers can, and should, be blaming (and suing) T-Mobile.
SirWired
A company called Danger? Responsible for data and servers? Yowsa! Red alert time!
What's up with all the editorializing in the summary?
You must be new here.
Danger was bought by MS only 18 months ago.
A year and a half later and they don't have a handle on it? Someone's getting paid WAY too much.
What the heck has this got to with Office and cloud computing
Nothing to do with office (unless they're using Access, which would explain the data loss), but "cloud computing" is what a couple here have more logically and less buzzwordily renamed "OPS" -- Other People's Servers. This is EXACTLY what "cloud computing" is.
Free Martian Whores!
I don't see this as having a big effect on Microsoft. T-Mobile on the other hand....
I don't believe that customers care if your services providers have problems. They have an agreement with you, not your providers.
Why, without your clothes, you're naked, Miss Dudley!
They have not had this problem in their first 8 years. Then, 18 months after Microsoft acquires them, they have a critical failure. You think that's all coincidence?
I suppose it's possible for one company to buy another and leave the company alone, but Microsoft certainly didn't do this. They moved most of the developers to Project Pink (and most of them have left MS entirely by now). I think it's pretty clear that the new MS was responsible. They managed the company. The data was stored at Microsoft's data centers.
Meanwhile, Microsoft is trying to sell people on the idea that their data should be hosted at Microsoft data centers. Am I not supposed to be skeptical about this now?
-- Don't Tase me, bro!
Last I checked, Hotmail still ran on FreeBSD
Which was what? 8 years ago?
In fact, yes, people have lost data from Google. That isn't even the only example one can find.
BBC news reports today that Microsoft has in fact recovered all data
So here's what confuses me... "BBC news reports today that Microsoft has in fact recovered all data, but a minority are still affected." If all the data has been recovered, wouldn't NO ONE still be affected? I mean... being affected by this means your data was lost in such a way that it couldn't be recovered. So...
At this point, the name Microsoft is pretty much a synonym for danger.
But the damage is not limited to Microsoft's reputation, the damage extends to the concept behind 'cloud computing', whatever that is. I think it is safe to say that Microsoft will recover from this incident, after all, it's record is already pretty suspect, but cloud computing will have this example hanging over it from now on.
I doubt that people will take this as a lesson that Microsoft is not to be trusted or believed since they are the public face of computing, but that computing generally, and 'cloud computing' is what's untrustworthy. Microsoft can abandon this particular project, coin a new term to replace 'cloud computing', and move on.
This is an opening for Google or other competitors. Will they step up and displace Microsoft as the public face of computing? We can be rid of monolithic operating systems if someone can make a system that boots a minimal browser/front-end that connects to the internet. A combination of BIOS and replaceable flash drive. Sell flash drives with the kernel and the drivers for the display/keyboard and network interface.
Best regards.
It's kind of like having a reference customer. It's all very well showing that they are incompetent in theory. It's good to be able to set up the production servers and run load tests. Here we have a real life demo that MS can really damage loads of customer's data. There are always cynics who say "yes, but they won't be able to do it in production". Now nobody will be able to claim that MS can't do an up to date full scale cloud screw up.
=~ s,(.*),<sarcasm>$1</sarcasm>,g if any_point_you_wish();
Who decides that a server farm called "Danger" is a safe place to store backups?
If you really want a conspiracy theory, toss in that another factor Microsoft considered was that Danger uses Unix servers, Oracle RAC, Java apps, and Hitachi SAN software. No sign of any significant Windows technology. So, they purposefully destroy the data. That not only hits T-Mobile, per your proposed conspiracy theory, but also hits Oracle and Unix and Java, and it shakes confidence in the whose Cloud idea.
Google and Amazon are ahead of MS right now in Cloud stuff, so if Microsoft can throw a delay into that sector, it hurts Google and Amazon more than it hurts Microsoft. By the time people get over the fright and are ready to jump back in, Microsoft will have its cloud offering out, AND they can point out that all major cloud failures have been on Unix or Linux, and with non-MS databases and app servers--and argue that if you want to get back into the cloud, go with MS on Windows servers, MSSQL databases, and .NET apps.
The problem with proposing fun conspiracy theories like you and I are doing, though, is that the real conspiracy theorists are already there. I've already seen several of the tin-foil hat crowd saying it was on purpose.
Hey, it was the last time he checked :)
[John]
Shit better not happen!
except word out on the street is that Microsoft moved over the vast majority of the Danger employees who stayed over to Microsoft's Ping Project and left the Danger division seriously under staffed. It also is going around that Microsoft had been telling T-Mobile that everything at Danger was fine and they were putting much effort into improving the software. In other words, they were lying to T-Mobile to keep T-Mobile selling the products and paying lots of money when Microsoft was really just putting the division on life-support and biding their time in hopes that Project Pink would produce something Microsoft could move Danger customers over to.
So is there NOT a reason to blame Microsoft for any of this? I guess you also don't remember all the talk about Microsoft trying to get the Danger product moved onto a Windows platform instead of it's BSD and Java platform. Microsoft is well known for either buying a competitor and shutting them down or buying them and dictating the product be ported to Windows. They bashed the engineers at SoftImage for a few years on dropping the UNIX versions of their software even though they did get a Windows version running. Customers and engineers didn't want Windows and wanted to keep the UNIX versions. Microsoft finally sold the company and walked away with its tail between its legs and you can see by what the film industry uses that Windows was not welcome much in that environment. BSODs really piss off people who spend hours crunching data and don't see BSODs or the like on nix boxes. IMO
LoB
"Anyone who stands out in the middle of a road looks like roadkill to me." --Linus
Dear Sir or Madam, The responsible Anti-Linux Troll that should have replied to this post by now is on sick leave and was unable to prepare a custom flaming reply to this particular post. In lieu of that, attached is our generic template which we use to write all our flaming responses.
1. Assert that the poster is wrong because you think so (that ought to convince everyone, but if it fails go to 2)
2. Call the poster a blind zealot (after all, religion and philosophy are the same thing and equally useless, right?)
3. Explain how unable you are to survive as a programmer in a world where you can't sit on your code for infinity to make money by charging for that which can be copied for free (as programmers can't possibly do anything else in the IT-arena with their skills), and generally explain how your responsibility to your kids is somehow bigger than your responsibility to your kids + the rest of the world.
4. Developers developers developers developers!
5. Give a detailed anecdote about how you manage to run a Windows-program succesfully.
6. Explain about how Microsoft is the dominant player on the market purely out of technical merit (nothing else of course).
7. Finish up trying defuse the effect of future responses by "anticipating" their arrival. Sentences like "But this is Slashdot so I'm expecting to get modded down" are quite popular.
We hope you will be able to infer the potential content of the post that should have been done by the respective Troll. Please accept our apologies. Sincerely, Assistant Secretary, Anti-Linux Trolling Association, Ltd.
If you quote this signature there'll be 72 copies of Windows ME waiting for you in Heaven.
I seem to remember Microsoft buying Hotmail back in the 90s, and royally screwing up its operations in much less than 18 months. They tried to move to Windows servers very quickly, and it was a disaster, and they were forced to go back to their FreeBSD infrastructure for a while.
Maybe something similar happened here.
I bet that most companies, when buying other companies, don't check a lot of basic things before buying them. As for "mickey-mouse outfit", in my experience, most corporations fit that definition well. The people running them really aren't that smart, and make all kinds of dumb mistakes.
Microsoft has been employing and run by Americans since it started, and they've produced nothing but buggy crap.
Meanwhile, the Mars rovers have been a tremendous success, built by American engineers using American-made software I believe (I'm pretty sure they use vxWorks). This is the epitome of software reliability I think.
I don't think nationality has much to do with this one.
This is much like when MS bought Hotmail and promptly screwed it up in their attempt to move it to Windows servers.
"But Microsoft, which bears at least part of the responsibility for the mistake, is paying the price with its reputation."
Microsoft bears ALL THE RESPONSIBILITY FOR THE MISTAKE!
They own Danger and they run the data center that stores the data!
It was their fault 100%.
See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
You really don't see the connection?
Yesterday, you put all your cell phone contacts and calendar data up in the "cloud".
Today, your data is lost.
Tomorrow, the same companies responsible for losing your cell phone data now want to take over all your Office documents.
The phrasing of this sounds chilling until one realizes that the main point here is that you still want to keep your own local copy. The T-Mobile phones should have done that. You should do that when creating documents on-line.
This is such a silly reason to vilify 'the cloud'.
From where I sit, the problem started when some guy wearing a tie said "and the phones use the server exclusively to house the data!" Dumb. The 'cloud' shouldn't even be part of this discussion.
"I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)
I have a Sidekick.
I still, a week later, can't get e-mail on it. My contacts were never lost, but the damn thing still doesn't work! I'm getting tired of waiting.
My contract is up in August and I'm going to find a phone that stores everything locally AND a new provider. I have learned my lesson.
> "The outage was caused by a system failure that created data loss in the core database and the back up,"
> [Microsoft Corporate Vice President Roz Ho] wrote in an open letter to customers.
It sounds like their "backup" was a replica on another connected server.
No actual offline backups at all.
When JournalSpace was destroyed, one SlashDot thread was "Why Mirroring Is Not a Backup Solution".
My favorite comment was by JoelKatz:
>> The whole point of a backup is that it is *stable*. Neither copy is stable, so there is no ... if the active copy of the data is corrupted, there is no backup.
>> "backup on the hardware level". There are two active systems.
>>
>> If you cannot restore an accidentally-deleted file from it, it's not a backup.
>>
Well, since this is /., you take your car in for a routine oil change. The mechanic botches the job.
Yeah, that sounds about right. That's why I try to do my own oil changes when I can.
The worst, though, is the state inspection. Without fail, something always seems to fail after one of those for me. Next time I'm going to demand they let me watch the work being done.
Can you be Even More Awesome?!
"But Microsoft, which bears at least part of the responsibility for the mistake, is paying the price with its reputation"
Just out of curiosity, what reputation might that be? :-)
Insert
I think this is more like 1984 scandal of Amazon Kindle, it will have long time impact on cloud computing and the general direction of things to come.
Even if you invent a system about e-ink/store tomorrow which has NOTHING to do with Amazon Kindle, you will still be asked "but will you delete my books remotely?". Just like some dead tech acquired by MS and not managed well will cost even IBM Mainframe dept. sales.
If one is a hopeless conspiracy theorist, he can easily suggest MS did it on purpose to lower general public trust to cloud which they have almost nothing. Cloud is all open source empire right now, Apache Hadoop etc. are being talked about, not some MS enterprise server or technology.