Detroit's Emergency Dispatch System Fails

dstates writes "For most of Friday, police and firefighters in Detroit were forced to operate without their usual dispatch radio when the emergency dispatch system failed. The radio system used for communication between 911 dispatchers and Detroit's police, fire and EMS crews went down around 5:30 a.m. Friday morning, causing a backlog of hundreds of calls and putting public safety at risk. Michigan State Police allowed Detroit's emergency system to use the state's communication towers, but access was restricted to top priority calls out of fear of overloading the State system. More than 60 priority-1 calls and more than 170 non-emergency calls were backed up. With no dispatch to communicate if something went wrong and backup was needed, police were forced to send officers out in pairs for safety concerns on priority-1 calls. Detroit's new police chief, James Craig, says he's 'appalled' that a redundant system did not kick in. The outage occurred only days after Craig took office. The $131 million Motorola system was installed in 2005 amid controversy over its funding. Spokesmen for Motorola said parts of the system were regularly maintained but acknowledged that backup systems had not been tested in the past two years. They said the problem was a hardware glitch in the link between dispatch and the individual radios. As of 9 p.m. Friday, a Motorola spokesman said the system was stable and the company would continue troubleshooting next week."

Comments

well

[Anonymous Coward]

Knowing their police force, the 911 outage might have saved some lives.

Duhhhh....

[Mr. Dop]

...its Detroit! Michigan is the only reason why both California and Florida dont fall off into the Ocean. It sucks that much.

Expected

[girlintraining]

Detroit has had massive funding and infrastructure problems for some time now. It's a dying city with much of the suburbs either abandoned, being reclaimed by nature, and generally being both in appearance and substance as a 3rd world country. It's so bad it has gotten national attention -- an emergency financial planner was sent in to try to right their budget, with limited success.

You can't judge Detroit the same way as you could, say, Chicago. They're no longer really part of the first world. This wouldn't be news if it happened in Afghanistan, for example. It's a sad state of affairs, but this is the inevitable result of a slide into the third world... our bridges and other key infrastructure is also rotting. Detroit is just foreshadowing what will happen to many of our cities over the next 15-20 years as our economy continues to slide into the ocean of wealth inequity.

Re:Expected

[sjames]

The thing is though, all of TFAs indicate that the city had a valid contract with Motorola to maintain the system including routine testing. In spite of that, no testing happened. While your observations may have bearing in general, in this instance it seems like a well known vendor with a (perhaps undeserved in retrospect) good reputation is the source of the problem.

Re:Expected

[mjpollard]

If you're talking "suburbs" within the Detroit city limits, then yes, I agree with you. (I went by my grandma's old house in northern Detroit a while ago -- the 7 Mile/Gratiot/Hayes area, for the natives among us -- and "reclaimed by nature" doesn't begin to describe it. I nearly wept at the sight as the memories of my brother and I playing in the back yard when we were kids surfaced.) Most in the Metro Detroit area, however, know "suburbs" as the cities and towns outside the city limits, cities such as Grosse Pointe, Royal Oak, Southfield, Dearborn, etc., all of which are alive and thriving.

Re:Expected

[ttucker]

Except again, from TFA, the city of Detroit was paying an enormous sum of money to a reputable vendor to maintain the system. How does that coalesce with this third world, wealth inequality theory?

Re:Expected

[DigiShaman]

Globalism. This is what happens when it's cheaper to move automotive manufacturing overseas only to be compounded further by the unions squeezing out all the profits and stonewalling the change that's necessary to survive. It's one giant death spiral that was enviable. We've had ghost towns in the past, there's no reason to think we wont have them again in the future. Hell, Midland TX (currently a boomtown) might be the next one should all the fracking stop via legislation.

Re:Expected

[Anonymous Coward]

different Motorolas -- it split up, the part Google got is the phone making one, but the radios one is Motorola Solutions

Re:Expected

[Anonymous Coward]

Shhh, it's all about wealth inequity. Everything is. Didn't you get the memo? It turns out that the secret to happiness in life is worrying more about other peoples' bank accounts than you do about your own.

Re:Expected

[theskipper]

I think there are subtle differences in interpretation when discussing wealth inequity.

Wealth inequity won't ever go away and in itself isn't bad. If you work harder than your neighbor then your net worth should eventually be greater. That's the American Way (tm).

The problem is when you reach a tipping point where 1% of the population owns 30%+ of the gross net worth of a country. Because of that overwhelming wealth, it results in the owners having a huge amount of influence in the political and legal processes. Meaning they can fund lobbyists to bend lawmaking to their wishes, and they can afford the absolutely best legal representation after breaking laws (i.e. powerful enough to be above the law).

That's seems to be crux of the wealth inequity argument imho. Viewing it as some type of "jealousy" by your viewpoint, and its systemic effect on the other (the OP).

Re:Expected

[FatdogHaiku]

Maybe it should be Motorola, Solutions?
Actually, I'm betting this was a failure at the local level, a couple of techs or more likely a middle manager that spreads their people too thin...

Re:Expected

[girlintraining]

Globalism. This is what happens when it's cheaper to move automotive manufacturing overseas only to be compounded further by the unions

Okay. Pardon the french, but I'm gonna have to ask you to take a step back, and literally go fuck yourself. Every major industrialized country except the United States has a labor party, and strong unions. The wealth gap in every other G20 country is significantly less than here in the United States. If your argument had even the slightest rootings in reality, the story would be very different. Unions had nothing to do with this; Rapid deregulation brokered by large corporations and a cozy relationship with Congress did. Unions have exactly dick to do with this -- it's just propaganda pushed out by Fox News and rabid conservatives who think profits are people too. Unions act as an economic stabilization force -- they cool the fluxuations in unemployment, wages, etc. Even Adam Smith in Wealth of Nations said Unions were a necessary part of a capitalist economy -- and he said the same about health care and unemployment insurance. He then went on to provide examples of how the long term growth of an economy improves with such policies; But they are usually not implimented because of short term focus on profit. He also advised governments to step in and create public works projects during periods of higher unemployment, back-filling the natural boot/bust cycle of capitalism, and then increasing taxation during periods of economic prosperity as an investment into the next cycle.

As much as many conservatives think they have a handle on what capitalism really is and what's best for it, they have a remarkable lack of education on the positions of its strongest supporters. It's unfortunate, really; If they weren't so fiscally irresponsible with their short term thinking and focusing on things like reducing government spending during a recession, etc., we wouldn't be stuck in these "stagflation" situations where inflation rises but unemployment remains constant. Such an (unpredicted) economic stall-out was first observed during the Reagan administration courtesy of "trickle down" economics. Its successor has resulted in the longest period of elevated unemployment in American history. And none of this has dick to do with Unions.

It's one giant death spiral that was enviable.

The death spiral isn't actually a spiral so much as a cycle. Deregulation leads to market crashes, which lead to regulation, which lead to market crashes, which lead to deregulation... Capitalism itself is fundamentally and systemically unstable, especially in its pure form. This is why almost every major world economy is a hybrid of socialist and capitalist policy, and any divisions between the two are largely arbitrary and based more on the political beer googles of the person assigning labels than what the economy is in actuality.

Re:Expected

[RubberDogBone]

Wait wait wait. Perhaps you hear the Motorola name and think this is a vendor with a proud and respected name and a great reputation. And that's true. 30 years ago.

The Motorola Solutions of here and now is notorious among Public Safety agencies for installing crap equipments, failures like this where the failover doesn't work, system problems, interface problems, junk radios, and all the while overcharging for all of it. Even when they low-bid something, they can be counted on to deliver a solution that doesn't actually, you know, work, and then negotiate extra fees to fix it. And fix it again, and again, Because by then, the agency has already spend X+Y dollars on it so they can't throw it away and get Kenwood or Harris in there. Besides, bad things happen when Moto loses contracts. Like it drops dead suddenly and mysteriously and doesn't come back for a long time. Sabotage? No. They just quit with the zero-day hot fixes.

Chiefs of Police and Fire and other agencies know this stuff goes on but good old /\/\ has lots of lobbyists and sales weasels throwing dinner and junkets at anything that breathes and can vote for or endorse a Motorola contract. If the local Chief doesn't want to play, then they go for the city managers, the city council people, the Mayor, the dog catcher, anyone they have to. They totally act like a big defense contractor except on a local level where the local yokels have no response this sort of action except to vote for Team /\/\ every time.

So in other words, the crap that happened to Detroit is FAR from the only similar situation where the system dies. NY has had problems, San Francisco, DeKalb County GA, Denver, systems all over the place. It happens so often that it's considered normal Moto behaviour and THAT'S scary.

Where are the other vendors? They're out there bidding too. But Motorola is sort of the penis enhancement of radios where "all the BIG GUYS have Motorola so you KNOW you want to have it too, don't you?" is what the sales reps say. You know that big city you want to be like? Yeah? They have Motorola. So sign right here. No no, that's not a contract change. We just corrected a mistake for you. Just sign it.

Re:Expected

[JBMcB]

Unions had nothing to do with this; Rapid deregulation brokered by large corporations and a cozy relationship with Congress did.

Sorry, in regards to the automotive industry, you have absolutely no idea what you're talking about. The Japanese, German and Korean car companies ATE THEIR LUNCH in the 80's, 90's and 2000's. That's what tanked the US car companies. Know how many foreign car companies have unionized factories? ZERO. OK, Toyota had ONE - NUMMI - that was a joint venture with GM (which was the only reason it was unionized) It's closed now, and they moved production to a non-union plant.

The union can be an important and positive force in the labor market. However, the auto worker unions have gone completely bonkers over the last 50 years. A couple years ago there was a minor scandal when Chrysler fired a dozen or so line workers for drinking and smoking weed on their lunch breaks. The UAW went to bat for these losers and got them reinstated. Got that? It's the UAW's policy that you can ingest legal and illegal intoxicating substances, then go work with heavy machinery, and it's perfectly OK.

I have dozens of stories about the self-destructive behavior of the UAW if you'd like more. Here's a quickie: the UAW retreat and conference center that costs the UAW $4 million a year. Good use of dues, right?

http://www.reuters.com/article/2011/09/22/us-usa-autos-union-property-idUSTRE78L29Y20110922

Re:And...

[djdanlib]

Last time we tried that, we got Australia.

Should have turned to the HAMs!

[blocsync]

I live in Florida, and when weather gets bad it can destroy critical communications equipment (including redundant systems). One thing I've seen done in the past is pushing communications through Amateur radio operators. Who (unlike the name would have you believe) are EXTREMELY professional and they tend to be able to very rapidly deploy communications equipment from the inner cities all the way out to the rural areas. Some of their equipment is capable of city and state coverage, but some of them can also establish international communication on a moments notice. This would have been a good fail-over for the lower priority calls. Just my 2 cents... http://www.arrl.org/ares has some info on the group I'm referencing.

Re:Should have turned to the HAMs!

[Technician]

A Motorolla trunked radio system consists of a pool of repeaters. A single repeater is placed into the control for the repeater site. On failure of the computer, all the repeaters switch over to a failover mode and become simple repeaters. All talkgroups vanish and all communications are shared by the repeater the radios have been assigned to. If properly assigned the dispatch and patrol should be all on one repeater. A second radio in the dispatch should be on another repeater along with top level "staff" communications.

Upon failure of an entire radio site due to power failure, antenna catastrophic failure, etc, then the backup site should kick in.

A poorly managed radio system will have radios assigned to repeaters at random. I have seen this, so when failover happens, teams can't talk to each other as they are not on the same repeater and other services are blended in so some patrols and some fire may be on the same repeater and some may be on another with no communication between repeaters.

This poor management happens when the person laying out the system is suddenly downsized and someone without knowledge of the plan has to add or replace radios and have no idea want the default radio assignments should be so they are assigned at random.

Re:Should have turned to the HAMs!

[Mr. Freeman]

Nonsense. HAM radio operators are less relevant today than they have ever been. HAMs provided useful emergency communications two decades ago, but no longer. Nowadays, cellular providers truck out COWs (Cellular On Wheels. i.e. mobile cell sites) within hours of a disaster, or even preemptively if a disaster is expected. After a disaster, you'll see people talking on Nextels, not relaying messages through HAM operators. Sure, the red cross will accept volunteer radio operators, but only because policy dictates that they do so. They'll be told to set up useless radio stations and relay worthless information until they get bored and leave, or they're told that they're no longer needed.

If you want the truth about HAM radio involvement in emergency communications then you need to talk to HAMs that have actually attempted to participate in such activities within the last decade. Organizations like ARRL are going to talk up how important HAM radio is because that's the point of the organization. They'll tell you that some number of HAM radio operators participated during some disaster, but they won't tell you that the only traffic sent over those nets consisted of nothing more than periodic radio checks. They won't tell you that all HAM radio stations were set up in tactically unimportant locations. Half the time they're in areas which don't need a radio operator and the other half of the time they're set up next to a fully-functional radio truck which doesn't need any kind of HAM support.

In the situation submitted by the OP, HAM radio wouldn't have helped anyway. HAMs can only practically set up fixed locations, which are already served by landlines or personal cell service. A mobile solution would involve putting a HAM radio operator into every single squad car, which would be impractical for obvious reasons.

Re:Should have turned to the HAMs!

[Anonymous Coward]

After a disaster, you'll see people talking on Nextels, not relaying messages through HAM operators.

Might want to check on that, there are no more "Nextels", as the iDEN network was shut down on June 30th (though there is a regional provider in the southeast (SouthernLinc) that still supports the iDEN technology). Amateur radio does not rely on the cellular/landline/satellite communications infrastructure, and in a real disaster would be available instantly to assist with emergency communication needs.

HAMs can only practically set up fixed locations, which are already served by landlines or personal cell service.

Not sure where you are getting this, however Amateur radio operators are able to operate mobile as well as fixed locations. And during a disaster, there is a real possibility that landlines and personal cell services could be disrupted. There are a number of HAM organizations that operate repeaters throughout the country that provide mobile HAMs extended communications areas. Perhaps a HAM in every squad might not be practical, however claiming that Amateur radio for emergency communications isn't relevant simply isn't true.

Re:Should have turned to the HAMs!

[blocsync]

Actually, I don't need to speak to "HAMs that have actually attempted to participate in such activities within the last decade." As I am one. I'm not sure what bad experience you've seen with what sounds to have been a poorly organized net. However, it does not describe the entire community and it definitely doesn't apply to all situations. I don't think I've ever heard an actual "radio check" on a live emergency net either. Net Control tends to get very annoyed about low priority traffic like that.

Perhaps you're speaking only of a specific area or a specific group of HAMs, but I don't believe your comments apply everywhere.

Clearly in this situation, All Police/Fire/EMS/Dispatch personnel could have used Cell Phones to fill the void, but they didn't. There's a string of failures here, not just one system failing. My suggestion wasn't to replace their coms completely with HAMs, rather to use them in an organized NET to handle the lower priority calls, due to the concern over high volume on the state radio system.

I think people underestimate the degree to which people will volunteer and assist public services when called upon. You can criticize HAMs/Red Cross/etc... all you want for their failures, I'll judge them on their successes when few others are stepping up at all.

Re:When seconds count, help is minutes away

[Lunix Nutcase]

Except the fault was Motorola's. But don't let facts get in your way.

Re:My question

[Rockoon]

Because this is a very serious thing indeed. Somebody breaking into your house? 911

The 1911 isnt a very good home defense weapon. Shotguns are much better. The 1911 is for when you are out and about.