Slashdot Mirror
Blackberry Network is Down
Brian writes "According to CNET and others, "A system failure at Research In Motion has affected BlackBerry users in the Western Hemisphere, a news channel reported on its Web site late on Tuesday. The infrastructure failed on Tuesday night, and e-mails were not being delivered to the handheld devices.""
Common, this was an easy one!
"Blackberry Blackout"
MABASPLOOM!
My thumbs are twitching!
Now I have to go in to work and explain that I dont control the Blackberry network........Cmon RIM!!
If you ignore the other uses of a tool, does that make the tool less useful, or you less useful?
Not to mention the service interruption, but imagine the backlog once everyone realizes their crackberries are working again... Frankly, I'm scared.
.... In the following locations:
M .20070418.wblackberry18/BNStory/Business/home
o utage-update-its-kinda-up-253214.php
http://www.theglobeandmail.com/servlet/story/RTGA
http://www.wnbc.com/news/12339359/detail.html
But I can verify that their network is up (sort of) and Engaget.com confirms this:
http://gizmodo.com/gadgets/cellphones/blackberry-
So YMMV.
This is my opinion. To make sure you don't steal it, it's covered by the DMCA.
It is expected to be back up by 11:00 EST.
Because millions of voices really ARE crying out in pain. Man, the cooler we make things, and the more that entire business cultures get built around this stuff, the more fragile it is. Just think of the war stories we're going to hear about people who've come to utterly depends on their Crackberries having lost a deal, not heard that a critical server was down, not realized that a surgery had been rescheduled, and so on. I wouldn't make a living if people didn't depend on fancy networked technologies, but it sure does feel like a house of cards, some days.
Don't disappoint your bird dog. Go to the range.
For what its worth, my employer (in Washington D.C.) has their own Blackberry Enterprise Server (an on-site server that interfaces directly with corporate mail systems), and it appears to be unaffected by this outage.
A single point of failure can bring down the entire network? Not very reassuring, especially considering Blackberry is predominately a business tool.
Dan East
Better known as 318230.
More interesting will be the addicts reaction. Some people really hang on the devices and get addicted to their Crackberries. I wonder how they will adjust (most people will do just fine) and what lawsuits will result. Or if the plantiffs are too worred about simply having their service cancelled!
Although I did notice I couldn't browse some web sites last night and this morning. My email and phone calls are working fine!
I already beat you to it.
If you ignore the other uses of a tool, does that make the tool less useful, or you less useful?
Can't be true... I would have got an email telling me if it was......
Cruise TT
I have to wonder where is the redundancy in a system if a failure such as this can occur. I shudder to think of the amount of money that will be lost by companies who have become dependent on the device. I wonder if there will be a lawsuit in the States from companies claiming lost revenue?
This wasn't really Tuesday night methinks. My boss who is Deaf and uses a blackberry like the dickens for contacting outside the office said she couldn't send from it all Tuesday day, starting in the morning.
Please allow me to hate the creator of the 120-character limit: *HATES*. Thank you.
... and tell everyone to stay home.
I can predict at least two possible reactions from end-users.
First: jumping out of a building due to the terror and pain of sudden withdrawal. It happens to heroin addicts, it can happen here too, folks.
Second: people wandering the streets of major cities bright-eyed and staring in open wonder, as though they were waking up from a long dream. Joining hand in hand, they frolic in the parks or whatever greenspace they can find chanting "Free at Last, Free at Last..." The clouds part, and an auspicious rainbow graces the sky. Oh, and I suppose there are other reactions: incoherent rage at no one in particular (ever chat with a cold-turkey smoking quitter?), unjustified rage at corporate IT for letting this happen, curling up in a fetal position in the corner, uncontrollable thumb twitching (almost like phantom limb pain).
Then there's another reaction: simply shrugging and going back to computer-based email and cellphones.
It's fine here in the UK, and the rest of Europe. As far as I can tell it's the US server, which 'only' serves the USA and Canada. As mentioned by others though, one point of failure destroying such a large portion of the network is bad planning at the very least!
"Officials with RIM said they are trying to reset the system and told NewsChannel4 that they are concerned that the backlog of data, which will rush through when it comes back on line, could cause a bigger problem"
When in doubt, reboot!
I got in trouble with my wife last night for not reading her emails to me, and I could not figure out why synchronization wasn't working. Thank you, Slashdot! You saved my marriage!!!
I'm proud of my Northern Tibetian Heritage
Did anyone read Western hemisphere and think WTF?
And I don't want to carry one, first I heard is when a user asked me about it. Sure enough, my boss is asking me why their blackberries aren't working and I'm in a meeting trying to show them how the system works... CURSE YOU RIM!
They could at least tell a time they plan on having it working again...
Ask not what you can do for your country. Ask what your country did to you
To (mis)quote someone(s) I don't remember:
Society is a millipede - the millipede has a thousand Achilles heels.
1 in 4 Maine children in struggle with hunger.
There were so many double-quotes in that write up that I thought it was a """paean to VMS""". It's nice to see that Python continues the practice.
I'm proud of my Northern Tibetian Heritage
Reason? It's cheaper than bricks.
Seriously, that's what's gonna break our neck sooner or later. We strip systems of their redundancies to make them cost less, we use cheaper components and the lowest bidder, we downgrade specs to the bare minimum because price makes right.
Technology already starts learning from nature, copying structures and models from millenia proven concepts. I think business could learn a thing from them too. Because nature has down what business wants to achive: Maximum output for minimum input. There is no such thing as waste and surplus in nature's makeup, if there was, it would be used for more output instead. So why do we have 2 kidneys, why is our brain able to adapt to damage, if it wasn't for the simple fact that this proved to be the more successful way in the long run.
But as long as companies are run by managers who care more about their next quarter report than the company itself, this won't fly.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Blackberry down, we got a blackberry down.
Leave No Email Behind
'cuz mine still works like a champ. Damn near everyone else in the office is calling and whining that they can't send or receive any emails. I tried rebooting the server a few times, rebooting other servers, rebooting the blackberries, and then I find this.
I gotta start checking teh internets before I start working...
I was on RIM's rolling conference call last night and received some additional information. It seems that somehow they lost connectivity in their NOC. When they failed over to a co-lo they couldn't get the SRP communication up and running, causing all BES to fail in their connection to RIM. Fast forward a few hours and SRP is back up, but they cannot get critical components for email delivery to connect to their DB. Fast forward a few more hours and they get this up and running also. Currently, Sprint and Nextel are up and running, but the GPRS/EDGE service providers are still not receiving consistent mail flow.
Yippee!
God invented whiskey so the Irish would not rule the world.
Good ole' government. Last night at 11:37 PM the DoD distributed an email to all Blackberries informing them that the Blackberry service was down.
I heard the blackberry mainframe was about to spill the beans on 9/11, thus blowing the lid off of civilization.. but it was taken down at the last minute, mysteriously.
I had about an hour of it working 100% this morning - email, web, IM clients - from about 6:48 to 7:45 eastern. Using Cingular.
A single point of failure is always a bad thing. Especially if a lot depends on it. But it can be some cascading effect too, for example power plants running on top output that couldn't swallow the sudden additional load when some parts of the network blew up.
It's a lack of redundancies. Redundancies cost money, and we want everything as cheaply as possible. So no redundancy. But hey, it "works". Usually. The question is, though, can we afford the blackout?
Imagine communication breaking down. No cell, no net, no data transfer, nothing. You could hear commerce grind to a halt. Nothing could be scheduled, nothing could be delivered on time, we'd simply break down. And that scenario isn't as impossible as it seems, because telcos don't have a lot of redundancies in their networks anymore either.
But we're depending on them. Often enough with our very lives. Yet we're not willing to pay the price.
But even if we did, would it be invested? I mean, afaik it's not like RIM made some kind of promise that the service would work. And as long as you can't hold them responsible for the loss, of money and maybe even life, they certainly won't add anything that costs more than it has to.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
It'll be interesting to see just how much detail we get concerning what exactly failed and why the current level of redundancy didn't kick in.
I mean, it's not like the power supply failed on an NT 4 server (you know, the one with the post-it taped in place that says "East Coast B-berry server, DO NOT POWER OFF!!!"), it's not like somebody accidentally drove a nail trough some coax in the wall at RIM's HQ, it's not like somebody accidentally typed "rm -r *" at the wrong prompt. There has to be some serious "Thank God I'm not the one stuck cleaning up that mess" stuff going on here.
Funny unrelated story. We had an exec looking at one of the blackberry's. He put in back in the hard case and was fumbling around with it and saw the "RIM" on the back. Then he asked, in the innocence that only an exec can have, "So, how does one go about getting one of these 'RIM' jobs anyway?" When I am canonized as a saint, one of my miracles will be "not laughing at the VP who asked how to get a rim job."
There are some people that if they don't know, you can't tell 'em.
We too have our own BES server.
None of our Verizon users are receiving or sending mail (or able to use other network functions). The phone still works as a cell phone.
Of the three T-Mobile users we have heard from, two are fully functional and one is down.
On my own system, the last e-mail I received was at 11:40PM last night. That e-mail was sent from our internal e-mail system and was received by me on that system at 7:14PM. So, it appears that there was intermittent connectivity last night.
Perhaps they are bringing systems back on line in stages so as not to overwhelm everything?
The yuppies follow with Outage Outrage
RIM: Get better at communicating with your client base or they will go elsewhere.
philo
I don't understand how the ENTIRE network can be down. Hasn't anyone learned from the design of email?
But then again I can't see what Blackberry gives you that you can't get with an IMAP server anyway.
Now what's a busy executive supposed to do when you're giving them a presentation? Actually pay attention to you?
Does this affect us? We haven't had any issues but I am wondering if it is because we have the enterprise server.
The greatest revenge in life is massive success.
And people think I am on crack when I tell them that Windows Mobile and Push technology is the way to go...
Totally independant from any type of telecommunications, only requires a data connection be present....
In the meantime, my Treo is working fine. Why? Because it relies on the POP protocol used over a TCP/IP network. Traditional, boring, standards-based and reliable.
Yes, my TCP/IP is provided over GPRS, but I hope that my next Treo (linux-based?) will offer a Wi-Fi connection as well.
Pascal
God love the Government, it is actually common practice for people in the Govt (my experience was with the US Air Force) to send an email out that literally said: "Email is currently down, we will let you know when it has come back up" - and they do of course send out another email later when Email comes back up. You get 2 emails right after another, one saying Email is down ("No freaking way!") and one saying Email is up...
Hence the oxymoron: Military Intelligence
Sleep: A completely inadequate substitution for Caffeine.
... 9:52am EST (Rochester, NY)
www.fiscaltimes.com
They're also posting updates as they come in.
then again it's a treo and not dependent on a provider with a single point-of-failure.
Mobile outlook 1, crackburry 0.
Website Hosting
Shameless Plug for you DoD folks. Full disclosure, this is my employer and I directly work on setting up new installs of this compliant mobile email alternative.
f m/
You should look into Apriva and our Sensa windows mobile mailer and PIM software. We are just starting to get traction with our product as it took some time to get all of the needed approvals. Our infrastructure is not very different from the other guys, with a "Sensa server" sitting next to your Exchange server, forwarding to a NOC (US Based) then out to your device.
I cannot tell you that our network will never ever have problems, but I am more just informing that there are alternatives.
rmatthews at apriva dot com if you would like to contact me.
To quote our site.
Sensa® is an advanced, secure mobile email system. The system is designed to support S/MIME v3, secure email with robust interoperability with Microsoft Outlook® desktops, the ubiquitous RIM Blackberry® devices and emerging best-of-breed tri/quad-band Windows Mobile PDA's.
Sensa® supports multi-vendor handheld wireless devices and multi-vendor wireless networks with common server components to provide unique budgetary flexibility.
The Sensa® secure mobile email system was designed from its initial conception to comply with DoD Directives and NSA Type 4 (SBU and FOUO) Protection Profiles. Additionally, full DoD PKI enablement was a key design objective.
http://iss.apriva.com/solutions/securemessaging.c
You raise a really, really good point. This is why it's important for their to be a wide variety of hardware and software makers out there, all using like protocols to communicate. Email as a whole is never going to go down because there are too many different ways handle it... even if every exchange server in the world were to blow up, there are alternatives, and the system as a whole keeps going.
In other words- beware things that are popular, as you'll be subject to the same problems as everyone else when the system goes down.
Hey, I work in Waterloo (at a different IT shop). The 'RIM job' line is very old hat up here...
Hey! That last one is MSTRKRFT! thanks for the pic meng
Nothin like having a Severity 1 issue, finding the cause, and being able to say "Hah! Not my problem!" :D
Verizon, T-Mo and Cingular are still limping along. BES and BIS affected to the same degree. I am surprised that my Blackberry internet uses the same data channel from the T-Mo network, I knew that BES did, but I suppose the fact that google always defaulted to google canada was a giveaway.
Nothing for me to d but play Brick Breaker or actually work. I will choose the former no doubt.
The EDGE/GPRS issue (where the signal type is showing up lower case), may have been resolved. RIM is claiming that there was a backlog of registrations and messages hitting their database. This was causing the wireless provider's inability to get a better signal from the relays. RIM states that they may have resolved this issue by removing burden from their DB. If users are still seeing a lowercase edge or gprs, the device may need to have the radio recycled or the power recycled. This is assuming that the BES are trafficing information to RIM. If not, power cycle the BES first.
This is Russell Shaw. I do the BlackBerry beat blog for ZDNet. I've just called all the carriers, and RIM, and have just posted the latest at: http://blogs.zdnet.com/blackberry/?p=135
From the statement: "Root cause is currently under review, but service for most customers was restored overnight and RIM is closely monitoring systems in order to maintain normal service levels."
6 9
http://www.networkworld.com/community/?q=node/141
Maybe people will actually pay attention during meetings today. It always stuns me when my team has meetings to discuss, say, a plan to realign billions of dollars of procurement funds, and some senior jerkoff is too busy shooting off emails to pay attention to the rather more important matter at hand. I always joke that we need to slip in a random bit of classified info so that everyone will be required to check all their devices at the door.
Karl Rove in ER being treated for shock following Crackberry failure. Nation's government falls. Peace breaks out, we run a surplus, Crawford is consumed by tornados that miraculously miss the protestors.
Wait, am I awake now? Damn.
People are ridiculous. "Oh no, I can't receive emails while I'm in my car!" Can't these people find anything useful to do with themselves in the interim? Barring very, very specific and anamolous exceptions, if I have an employee who basically shuts down because of a Blackberry outage, I think I need to find another employee. It's like, wow guys, it's amazing anyone got anything done back in the Dark Ages of 2002 when we didn't have Blackberries!
Grow up, Blackberry users, and find something to do.
mirrorshades radio -- darkwave, industrial, futurepop, ebm.
I would prefer Microsoft's Exchange Activesync better than Blackberry. Both configuration wise (EAS is out of the box in Exchange 2003/2007) and in usability.
Cheers!! Abdul Aziz
Has anyone tried the free (as in beer) alternatives to Blackberry for Push email?
For instance this one
You need a data plan, so not completely free, but it works pretty well for me. Check the list of supported phones before going through the registration though.
Email is not mission critical. I work for a fairly large corporation, we are dealing with sar/ox auditors at the moment. Plain and simple, the exchange servers are not on their list of mission critical servers. Why? Exchange servers don't deal with the financials in any way (and shouldnt). The company itself may decide that email is critical, but the auditors and investors dont care. Without email we can still take orders, phone calls, fax, MANUFACTURE GOODS. You gotta draw the line somewhere. The business will go on without email, you better have a way to take orders without email. What, you rely on the availability of 3rd party internet routers for your mission critical applications?
I wish I was joking.
In addition to a BES are running a Sprint/Seven product known as Business Connect Server Solution which offers the same sort of middleware solution as a BES does but for a client (BizConn) that can run on PPC/PalmOS/and some Java-based handhelds. We have a saying in the office - "If RIM is down, it makes headlines. If BizConn is down, it must be Friday". My own experience is that RIM's managed network is more reliable and better supported. Sprint's continued mismanagement of the BCSS back-end has caused us no shortage of grief. They refuse to provide an SLA for the service, which has forced our team to dial support back to 8x5, M-F for the app. They push client updates with little notice and no advance download available for testing. Their support line offers perpetual catch-22. They refuse to issue ticket numbers for incidents and then request ticket numbers on subsequent callbacks. I can see the rational for the arguments that RIM should decentralize, but I would much rather deal with them in times such as these than the telecom proper. Then again we also run Exchange and offer ActiveSync. Don't ask why we offer 3 flavors. Please don't ask.
From every single person using a BlackBerry's email functionnality?
I know that emails are a plain-text technology... But how comes this transit trough a RIM-only network!?
Btw is there a web browser in BlackBerrys allowing to connect to a company's "webmail"?
Where the hell were you in 1999?
Of course, if the crackberries just communicated via standards, then you wouldn't have to have this problem. The only special thing needed is new mail notification. They elected to build a special server to handle everything, and now they have to deal with it when it blows up.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
There's a mail list setup at http://www.c2security.org/mailman/listinfo/bb-outa ge/ to notify people of this.
BB-Outage is a mail list for people to notify others if RIM's service is down in a particular area. if RIM is down in China, US, A particular state, city, etc, for people then post the area to the list so others are notified. It's probably best to subscribe from an address that's not associated with your Blackberry since if RIM's service is down, you won't get the e-mail anyhow
I was planning on purchasing a Blackberry shortly. I know the e-mails sent have to go through the RIM servers, but I guess this would not effect SMS or phone calls on the newer Blackberries right?
Payroll, on the other hand, is not on the criticality list at all. Our DR strategy is to write paycheques by hand based on people's last pay stub (that info can be recovered quickly from backup tape) and square up discrepancies a month or two later.
--
.nosig
Now might be a good time to apply for a RIMjob
"...have made me consider returning it"
What are you waiting for, you fucking switcheur? GTFO.
I've avoided crackberries and similar products like the plague since they came out.My pda has a cable that connects it to my cell phone if I really need to get on the net for something... I picked up a treo a while ago and immediatly put it back down on the shelf. If I want to make a phone call, I'll use something small that resembles a phone. if I want to surf the net or send email, I'll use a computer, and if I want to set up a meeting on my planner, I'll use a palm pilot that is designed to be an agenda. Combine too much together and you have a device that's too large for it's main purpose, to small for it's secondary purposes, and generaly not equiped to handle any of it's jobs properly. Now with blackberries proving they don't have a solid network, it's just one more excuse for me to keep avoiding them. My desk phone forwards to my cell, my voicemail sends a SMS message to my cell, urgent emails send a sms message to my cell... servers page me by sending a SMS message to my cell... I don't really need any more instant information... if it's a critical issue call me... Any important server issues that are detected by the servers can easily be summarized into a SMS message, which has been around for a while and has been relatively stable. Actually 90% of the server messages I receive fall into ("Application log full", "Power fault, UPS active", "Power restored", "Server X is rebooting" or "Server X is MIA")