Slashdot Mirror
What Happened To Hotmail?
Blastercbi1 asks: "I have an e-mail account with Hotmail and use it for all my personal e-mails and some business e-mails. The last time I was able to Log in was last Thursday. I waited two days (got used to the frequent short outages that Hotmail had) and still nothing. Finally I decided to contact Hotmail support. Well... I just couldn't. They just had one phone line which was down. And a bunch of automated e-mail support. I just would like to know if someone else had this problem and how they dealt with it. The only information I could find on this topic was at:
news.cnet.com . And from what I read, I have a feeling I'm in for a bad surprise!
(First thing I'll do when and if I get my account back is backup all my stuff and get an account with Yahoo!!)" Sounds like a good idea. Anyone have any more news about recent Hotmail outages?
here.
I like the guy in the Wired article bitching about all the business e-mail he lost. Dude, you're relying on a _free_ account sponsored by some faceless company? Maybe you can ask for your money back, I'm sure they'll be happy to pay it in full.
--
jojo
If there was a recent outage, I've missed it, and I check two accounts on Hotmail almost daily. They have had their screwups, but when they did wipe out a couple dozen messages in my box, they were restored within a couple of days. No critical stuff, but at least I got it back.
Jack
...and then slowly began going down the toilet.
As the guy who ran YahooMail ops from our humble origins as RocketMail until a couple of weeks ago when I retired to play with more interesting things (after the first 50 million users it just gets boring :) it seems to me that you are somewhat "reality challenged" about how to scale a system to handle the userbase we are talking about here.
For starters, the first rule I learned (the hard way) was to have as few failure points as possible. This means _no_ centralized databases. EVER. A centralized database is your worst nightmare waiting to happen because when it does fail, and it happens more than you would think even if you are running overpriced "fault-tolerant" servers and database software (just ask EBay..), then the entire system is down. Having a component failure take out 5% of your users it a lot better than having it take out 100% of your users.
The key concept to grasp here is to design a system so that it degrades gracefully when it begins to fail. That means that instead of just locking up when a key box dies your system will continue working (albeit a little slower) and you have time to investigate the problem rather than having to hit a panic button. Remember that we are talking about mail here, so if you lose your account database for even a few hours you are going to be unable to accept incoming mail during this downtime and when you come back up you will get hammered by queued messages. Swarms of little boxes are the only way to do something like this.
The swarm scales up very well on a $/user basis and is much more tolerant of failures than any centralized system. The swarm gives you much more flexibility in dealing with the daily traffic flux because it is trivial too add a few more cheap boxes or to re-task a box with a simple reboot instead of having to spend a lot of energy fixing the one box that everything else is depending on.
The other problem with your approach is that there are not many systems that can handle the sort of load and volume we are talking about. Those that are available are extremely expensive. A service that only gets revenue from ad banners does not stay in business unless it figures out how to get the best performance for the least amount of money.
As for Hotmail's troubles I am guessing that earlier posters are correct: Hotmail probably had a catastrophic disk failure on a key system and no backups, once that happens you start throwing the disks into other boxes and trying to do a sector-by-sector repair or recovery (a very slow and unpleasant process.) This is the sort of nightmare scenario that caused me to fear every 3am phone call or alarm page; eventually one of the major services was going to step on this particular landmine and I am sooooooo happy that something like that did not happen on my watch...
jim
> Maybe you shouldnt be complaining about Microsoft if you cant get _your_ math right.
Whoops. My pseudo-dislexia presented it to me as "1%". Not like it makes any difference - "BFD, only a third of a million people can't get their mail." That really shows a lot more concern for customers.
> And that's not many when you consider that Napster removed that many accounts from a _much_ smaller userbase.
Which has precisely nothing to do with it. Napster was reacting to a legal challenge; Hotmail is (not) reacting to a technical challenge that could easily be fixed by a management that cared enough. (As I mentioned previously, at worst it would require spending a bit of money.)
As an aside, it might not be fair to criticise Microsoft in any case. Does anyone know whether Hotmail is run by MS employees? Or is it just a separate company that they 0wn but do not directly operate? I notice that the media are taking questions to MS and getting their answers there, but that might not reflect the way Hotmail is actually operated.
--
Sheesh, evil *and* a jerk. -- Jade
> My Hotmail account works just fine, no missing emails, address book, or whatnot.
It's really bizarre. Some of the (few) news articles on the subject seem to imply that your account is associated with a particular server, and that when that server is down you're screwed. No rollover. (Perhaps the stories left the wrong impression; it's perfectly reasonable that an account would be associated with a single disks, in which case a drive failure would put you out of the ring.)
What's really bizzare is that whatever the problem is, it shouldn't run for a reported 10 days (and counting?). If it's a bad drive, you replace it, restore the backups, and you're back in business a few hours later. If it's a bad server, you fail the accounts over to another one and you're back in business a few hours later. Or even buy another fsckin' server, if it's something with as high a profile as Hotmail.
Truly, it's beyond comprehension. Unless it was a simple drive failure, discovery that there were no backups, and an unwillingness to admit it. Unfortunately, this not only would explain the otherwise bizarre circumstances, but is also true to form for Microsoft's manner of operations.
Other things that show Microsoft's {cluelessness, lack-of-concern} are the outrageous claim that "less than one-half of 1 percent of its 67 million users" are affected (as if up to 670,000 customers didn't matter), and the text of the automated response from the help desk, which says [quoting from memory] that "the temporary inconvenience is so we can enhance the system to serve you more efficiently" (how much efficiency will it take to make up for 10 days' outage and a loss of data?).
Also, for better or worse, that last [pseudo-]quote seems to imply that the problem is nothing so simple (and understandable, though inexcusable) as a simple loss of a disk drive.
I can't help wonder whether they were trying to "upgrade" to W63K.
--
Sheesh, evil *and* a jerk. -- Jade
A common attitude I've seen is "it's free, what do you expect?"
The reality is that Hotmail is not free. You view banner ads, you give up demographics information, and you put up with having numerous partner services pitched at you.
I work at a company that also does web based e-mail -- cobranded to your site's e-mail domain and look&feel... This service is free for end users, and free for web sites.
Our attitude is that it is unacceptable for even 1 e-mail to be lost. Downtime is unacceptable -- although admittedly we've had our fair share.
We strive for excellence in reliability, and availability. Why? Because our users (several million), are entrusting us with the most important means of communication on the Internet.
It is my perspective that downtimes like this are unacceptable: you *are* paying us -- with information, in many ways a more valuable commodity than money.
Demand more for your demographics and your time. You deserve it.
-JF
MrJoy.com -- Because coding is FUN!
Sigh...I submitted this Wired story several days ago. Word as a backup format?
"less than one-half of 1 percent of its 67 million users" are affected (as if up to 670,000 customers didn't matter)
Maybe you shouldnt be complaining about Microsoft if you cant get _your_ math right. One percent of 67 million is 670 thousand, one half of one percent of 67 million is 335 thousand. And that's not many when you consider that Napster removed that many accounts from a _much_ smaller userbase.
Mark Duell
Why trust Microsoft with your mail, (Is hotmail still useing FreeBSD??). I have never prblems with yahoo, I would trust it more to people with a funny name before I would trust it to microsoft.
Where are we going and why are we in a handbasket?
And, do you mean to tell me you haven't backed that stuff up anyway? What, you actually trust Microsoft to do something right? Sheesh. Newbie. ;-)
Anyway, I like Hotmail; I like being able to check my email from anywhere I can find an internet connection. The longest downtime I've had was 12-14 hours once. They've lost and restored my messages (all of them) twice. They are notorious for locking people out of accounts that are too large. They obviously have some problems with spam. If they ever try to move their backend servers to NT again, I will probably be screwed.
That said, I like Hotmail. They pioneered free webmail, and they remain one of the better free webmail services out there. That doesn't mean you should trust them. It's fricking Microsoft we're talking about here.
Free music from Jack Merlot.
ouch. Since when is the last time you trusted the PR group?
--
Free Software: Like love, it grows best when given away.
The article brings up a good point: Hotmail is a free service run by Microsoft. Those two words put togethere don't exactly spell 'reliability'... I don't exactly have important data on my Hotmail account (just some attachments), and if Hotmail goes down for a week, I wouldn't be too concerned, because I don't get most of my mail there, nor do I get any important mail, for that matter. I mostly use it as dumping grounds for webpages I need to register for (not Slashdot, of course).
I would read The Register's article. It brings up good points I don't necessarily need to repeat.
Small potatoes make the steak look bigger.
He also thinks it's amazing because he can check his email from anywhere. I said to him, have you even heard about/tried any of the other services? No. So how do you know Hotmail is so good? Well, it's M$ he said. At which point I decided it might be easier to just not say anything at all. He also casually mentioned that he has over 100 spam addresses blocked at Hotmail. On my 5 non-hotmail addresses, I'm lucky to get 1 spam message a week.
And you know what the worst thing is ? He was asking me yesterday about what I knew about MCSE - he's thinking about doing it. Sigh...
Hate to laugh at you, but this is just the sort of think that gets me started. I'll be laughing about this for weeks. =P
Jsut the sort of thing you'd expect from Microsoft tech support, eh?
When someone is suggesting getting a free web-based email address, its always "Get a Hotmail account or something.." They don't really mean specifcally a Hotmail account.. it's just easier to say Hotmail than "free web-based email".
its happened three or four times, not fun I can tell u.
M$'s exuse, the server holding my account details was down for some reason, this went on for a period of 24 hours at the longest incident. Strangly enough my other account hasn't had any problems.
Ohh well.....
Da Cr33p
No, I've no idea if it will be or not, but the recent reporting of outages at Hotmail play right into MS' hands. We've all read the statements 'if NT is so good, why is hotmail not running on NT?', etc. Well, they may be planning this. First, instill some panic that the system is starting to not handle things well. We will probably see some responses from MS at some point stating that "BSD is OK, but we knew it could never scale, etc", and that they've silently been planning a migration to W2k, and it'll be rolling out "soon". Could it happen? Of course it could. They've got the money to make it happen, even it if meant thousands of servers. They're doing that multi-terabyte geomap thing to prove how scaleable SQL server is, this makes perfect sense. I just don't see them keeping Hotmail on non Windows servers much past this year (hope I'm wrong) - it hurts their credibility too much when trying to convince large customers to use their software.
creation science book
Mine was out for about 7/8 days. I'm surprised this wasn't mentioned in the media too. I suppose they have bigger 'MS story' fishes to fry.
Besides uReach, Yahoo!Mail is the only one I use for "real" mail because I'm hooked by the rest of Yahoo's personalized services, esp. Yahoo!Companion. The rest are just "throwaway's", good for pseudononymous transactions. If you're placing a lot of trust in Hotmail not to lose your data, or keep it secure, or always be available...I think you're being foolish.
"I do not avoid women, Mandrake. But I do deny them my essence"
I have lost everything in my hotmail account and am now looking at what other free service I will use. I lost over 100 email addresses alone, some of them I have no idea how to recover. Although I don't subscribe to any of the conspiracy theories around why this data has been lost, I do find myself wondering why Microsoft can't even be bothered to apologise or to tell the truth behind this disaster. I also don't believe their figures, I personally know several people in New Zealand alone who have totally lost their accounts. Microsoft don't do themselves any favours when it comes to customer services do they.