Failures Mark First National Test of Emergency Alert System

An anonymous reader writes "The first full-scale test of the National Emergency Alert System failed on Tuesday at 2 PM. Some radio and television networks did not air any alert, while the performance of others was inconsistent. 'Some DirectTV customers reported hearing Lady Gaga's "Paparazzi" play during the test. Some Comcast subscribers saw their cable boxes turn to QVC before the alert, while Time Warner Cable customers in New York did not see any alert at all.'" If you were tuned to any American broadcaster at the time, did the alert system reach you?

Comments

Government failure?

[Anonymous Coward]

Those words never go together. I am shocked.

Re:Government failure?

[ackthpt]

Each failure is an opportunity to learn and improve.

The real failure would be to not identify failures and not improve - then we'd have to be blasted about it by the sensationalist media, trumpeting how inept government is.

Re:Government failure?

[BradleyUffner]

Those words never go together. I am shocked.

Yeah... except it was the private broadcast companies that failed to properly show the alert not the goverment.

Re:Government failure?

[Anonymous Coward]

I have to say, I think an actual failure would be if it were during an emergency.

As I say at work, this is why we test. Debugging finds bugs. That's kinda what it's for.

Re:Government failure?

[Anonymous Coward]

You know what the word "Test" means, right?

Re:Government failure?

[neokushan]

^ This

Testing for something and finding that the test didn't pass is NOT a failure of a system. It's exactly what it said - a test. Now they know where the faults are they can work on fixing them.

Re:Government failure?

[timeOday]

Government negativism has reached a questionable new high. Most often the criticisms are not even quantified (as in this story), not compared to private industry (where possible) (e.g. Social Security fraud), and not compared to historical or global norms. The economy sucks and the public is united only in being unhappy about it, in which they feel some solidarity, yet can't form a consensus on what to do about it.

Re:Government failure?

[smpoole7]

>private broadcast companies ... failed ...

Been waiting for people would say that.

The engineer at the local PEP (Primary Entry Point) in our state was standing at the transmitter site, watching the equipment, when the test began. He was on the phone with FEMA, as a matter of fact.

The test never came through. The (FEMA-supplied) equipment never responded. As a result, most of central Alabama never even got the test.

The failure was on THEIR end, not ours. We had done TWO statewide tests just prior to the national one and they worked fine. Don't blame us, dood. :)

Re:Government failure?

[cant_get_a_good_nick]

Err, Human failure....

I'm never sure why people like to pile on the government. Like any social organization, and Soylent Green, it's made of people. Just like Citibank screwed itself, Enron self destructed, Goldman Sachs enabled Greece to self-destruct, all governments and companies are made of people.

One additional thing to add, government employees rarely get high pay (remember that those $600 toilet seats were paid to private contractors). So, you're blaming fallible humans, a group made more likely to be fallible by the fact you don't want to pay (taxes) to hire the best.

Re:Government failure?

[quetwo]

I was at the PEP in our area. The CAP message that FEMA sent out was coded wrong. They sent out the CAP message using the format they dictated last year, not the newer format dictated this summer. Most of the equipment I know threw out any message it did get, and it simply didn't forward it to any downstreams.

Those who supported the backup method of the older EAN system, which they were supposed to foward regardless of the CAP messaging did so.

10136: EAN NATIONAL EMERGENCY ACTION NOTIFICATION 'LP 1(L1)'(MI-TXPEP) ORG=PEP
                'Wed Nov 9 14:03:00 2011 EST' to 'Wed Nov 9 14:18:00 2011 EST'
        Forwarded : 'Wed Nov 9 14:00:34 2011 EST'
                  United States(000000) District of Columbia, DC(011001)

All in all, in our area, we had 1 TV station, 2 radio stations, and 1 cable system (out of 5) that did any type of notification -- wether it came from IPAWS or EAN. That's a failure in my mind, as we were supposed to have our older EAN system as a backup.

Re:Government failure?

[Obfuscant]

Testing for something and finding that the test didn't pass is NOT a failure of a system. It's exactly what it said - a test.

And when the test fails, it is, indeed, a failure of the system as a whole. What was the intended outcome? (A nationwide alert.) Was the outcome achieved? (No.) This is almost a "by definition" kind of concept, you know. You test something and it doesn't work, that means IT FAILED.

The only way this test didn't fail "because it was a test" is if you think the important operational criterion is the ability to test, not the ability to notify people in an emergency. Or maybe you are confused by the use of the word "test", and are thinking of how when a student takes a test and doesn't pass it doesn't mean the system failed. Well, in this case, the test wasn't applied to students, it was a direct test of the system as a whole and yes, it did, really, overall, fail miserably.

Now they know where the faults are they can work on fixing them.

Now they know where some of the faults are and can work on fixing them. If you don't get a successful test of the entire system, then the parts that didn't get tested may still have faults that weren't detected. If a major switch somewhere failed, you can spend a lot of money fixing it and then feel safe, without realizing that every leaf node that it fed would have failed had it gotten the alert.

There is no question that we are going to need another test, after a significant period of time. And then another. And another. Soon we will be getting tired of the testing and it will take place at 3AM like the regular regional tests already do.

The important, unasked question is just why do we still need a national alert system? Is there someone out there ready to plop 100 nukes into all our major cities all at once? Unlikely. Would notifying all those people all at once really have much positive effect? We already have localized alerting systems that are tested on a regular basis, and we have better means of distribution -- NOAA weather radio. Do we need to fix this system, or should we just pull the national plug and let the regional ops continue?

As for the original article mentioning that Comcast switched to QVC before the test message, yeah, so what? That's how the system is designed. It is much easier for a cable system to send the "everyone change to channel X" command downstream instead of trying to insert the alert message into every digital stream. Or insert the alert into one digital stream and then copy it to all the other streams. Of course, the ANALOG channels need to have individual alerts for those people who use analog TVs, but all the digital subs who have digital cable boxes can get by with one channel for the alert.

I was listening to local AM radio -- not a peep. The news story leading the 11AM (2PM EST) report? "If you can hear me now, the test failed."

Re:Government failure?

[RockClimbingFool]

Hey, you do know the difference between DEVELOPMENT testing and ACCEPTANCE testing, right?

The national alert system is a product in development. This was a test to determine what is working and what is not working. You can simulate and test individual pieces all you want, but until you get the opportunity to test the entire system, you have no idea what links in the chain are broken.

This country is full of fucking idiots that have no clue how engineering is performed. Just keep your misinformation to yourself and stop trying to make those around you dumber.

Um... That is why it is called a "TEST"

[cwgmpls]

Tests are supposed to fine failures. That's what they are for.

Re:Um... That is why it is called a "TEST"

[DikSeaCup]

Because there's a fiscal penalty for failure when it's found!

Re:Um... That is why it is called a "TEST"

[Anonymous Coward]

Under what circumstance would everyone in the nation need to be informed instantly? To warn you the nuclear weapon that's been pointed in your general direction since birth actually launched? That the hurricane we've been watching for over a week is finally going to make landfall? That the [insert party not running the warning button] is holding up some piece of legislation? That someone flew planes into buildings in downtown Manhattan?

Pretty sure we can live without the national wide warning system for another decade or 100. News of 9/11 spread very rapidly just from word of mouth, flaky internet access and even before everyone older than age 1 had a cellphone. Not to mention the local emergency responders reacted without it as well as the local citizens who were away from ground zero didn't have a panic trample everything in their path reaction.

Re:Um... That is why it is called a "TEST"

[Anonymous Coward]

I don't think you understood what he was saying. Hurricanes don't require a NATIONAL alert. Communities affected already receive regional emergency broadcasts. Same goes for tornadoes/earthquakes/etc. It is unlikely in the event of a nuclear/terrorist attack the government would alert the american public, since odds are it will either 1) Do no good and just cause widespread panic and be too late to allow evacuation or 2) Turn out to be a false alert and piss everyone off. Like the other AC, I can't see any actual use for the system. And this is totally ignoring that a radio/television broadcast alert system is going to miss a *LOT* more people than it did even 10 years ago as we move away from those mediums. Colleges have it right: an opt-in text message alert system that warns you of danger no matter where you are.

Re:Um... That is why it is called a "TEST"

[GameboyRMH]

I was wondering what kind of situation could make such a system necessary. When a disaster happens the whole world is watching the news channels and sites within minutes anyways.

Re:Um... That is why it is called a "TEST"

[khallow]

I don't think you understood what he was saying.

I sure did. A national alert system is not only useful for national alerts, but also local ones.

Like the other AC, I can't see any actual use for the system.

In the event of a disaster, you're more likely to have a working radio than a working cell phone.

Re:Um... That is why it is called a "TEST"

[Locutus]

except fire drills and things like that where they tell you the time it'll occur and even stage evacuations to pass "the test". I once worked at a new facility where strange tripod sensors were all around and we were told to not use certain equipment for the week. It was only much later that I learned those sensors were some kind of environmental sensors required before long term occupation of the building was allowed.

I agree that properly run tests are supposed to find failures and proper procedures solve the problems found and future tests find other failures if there are any. Ten years after 9/11 and this is just coming about is my question. Remember how the dead hijackers were given visa extensions something like 3 years after the enacted the attacks? I think it was the 2008 election before Bush and his party started talking about immigration issues.

I think the geeks would be better off relying on their own form of warning system instead of relying on a government operated one. Maybe something tied in with HAM operators and their data passing system. Handhelds and base station radios are not that expensive these days.

LoB

Re:Um... That is why it is called a "TEST"

[EdIII]

I think the geeks would be better off relying on their own form of warning system

Yeahhhhhhh..... maybe not.

I remember the Internet on 9/11. It went basically this way:

1) IT people talking/screaming with other IT people relating information as it was happening. Information got *slightly* altered from one "hop" to the other.
2) Mailing Lists and IRC channels on fire with reports about everything from aliens, aliens raping people, mad cow disease attack, the Russians invading on the East Coast, ICBM launch confirmed by a friend at an undisclosed military location, etc.
3) Screams of, "But I have not gotten laid yet! It's NOT fair!"
4) Fuck it. Meet me in Everquest. We're taking those bastards down before we die.

The "Enemy" could not have created a better disinformation system if they tried.

Facebook? Twitter? ... Farmville? It would be an even more glorious cluster fuck if it happened twice.

Spotty

[DrgnDancer]

I was in the car listening to NPR for this. The NPR (WGBH) station did a nice little lead in story and switched smoothly to the test. As soon as it did I started jamming presets and none of the other station I had programmed got the test. Local Alt Rock station, local R&B station, and the other NPR station all failed to broadcast the test as far as I can tell.

Re:Spotty

[PolygamousRanchKid]

No, the government is just being selective about who they save in the event of a catastrophe. Apparently, they believe that we will need NPR listeners in the post-apocalyptic world. Alt Rock and R&B listeners, they think they can do without.

Re:Spotty

[jmcharry]

The alert is sent to a primary station in each area and daisy chained to others. WHQR is, I think, third in a chain. The alert hit there at 2:00:39. It got the start and stop "duck farts", but not the message itself. The scuttlebutt is that FEMA messed up the head end audio.

Re:Spotty

[Phreakiture]

There are different message priorities. I don't remember what the priority levels are called (it's been about eight years since I've been involved in broadcasting), but the options essentially allow some messages to be stored and rebroadcast later (with a limit on how much later). Higher priority messages go out in real-time; lower priority may be discarded.

The radio station's EAS ENDEC is supposed to manage this for them. In the event of a top-priority message, it just takes over the airwaves in real-time. Middle and lower priority alert the engineer to the situation and let him/her decide when to send the message. If the message is not sent before the time is up, middle-priority messages will seize the transmitter and lower-priority messages will get dropped.

I would expect this message to have been encoded with the middle option -- store it for up to xx minutes, then take action automatically if the station didn't do so voluntarily. This would result in it going out over different stations at different times, and that would be desired outcome.

Re:Spotty

[bennomatic]

I was at my computer, with several browser windows and a couple of email clients open. My cell phone was right next to me. When the time came, my Outlook reminder popped up and told me it was time for lunch.

Too bad these notifications don't reach those of us who don't rely on antiquated broadcast media.

Re:Spotty

[cayenne8]

Just remember you can play bluegrass and jazz without electricity. Unlike all rock.

I dunno..I remember an OLD special on MTV (back when they actually played music)...with Aerosmith Unplugged...one of the first unplugged specials.

Those guys rocked in acoustic!!

I had it recorded on VHS back then, I wish to hell I could find that whole 30 min special ( or longer if it exists) out there somewhere either to buy or download.....

Re:Spotty

[choprboy]

Came across here at 12:02 MST and the audio stream was screwed up. The audio alerts came thru fine, but the message was extremely faint and unintelligible. About half way thru the 60sec test someone at the radio station cranked the input volume all the way up, horrible high-pitched whine of background noise, but you could at least understand what was being said then. Still, it sounded like trying to tune into a radio station a thousand miles away... The normal monthly tests have never seemed to have that problem.

Re:Spotty

[Talisein]

IPAWS has a (just barely out of development AFAIK) private RSS feed that you need a special pin code or something for. It is just for broadcasters. They also have a private SOAP server that you need some X509 certs for to pull public CAPs from (this is a superset of EAS alerts).

IPAWS eventually will have a public RSS feed for EAS messages, but they don't seem as concerned about making sure that it will be properly provisioned to serve millions of clients hitting it up constantly.

I'm developing an OSS application to feed IPAWS messages from their SOAP server to a public xmpp server: https://github.com/talisein/Stormee

Its not really ready for prime time yet, but I should have something that works in a couple weeks.

Failures, what a surprise...

[dstyle5]

... because that is exactly what you would expect in a test of such a large system. The real surprise is if it would have worked without any issues on the first go round.

Re:Failures, what a surprise...

[PRMan]

They didn't use the last system on 9/11, which was the largest emergency in most of our lives, so who cares?

Re:Failures, what a surprise...

[Anonymous Coward]

Probably because 9/11 in all actuality was only a threat to a ludicrously, miniscule number of people compared to say... the rest of the country. I wouldn't be surprised if 98% of the country wasn't even slightly, remotely affected by it (ignoring the after side-effects of the shredding of the constitution and soforth).

Re:Failures, what a surprise...

[bws111]

The purpose of EAS is to alert people to take action that can save their lives, not to act as a source of breaking news.

"Incoming missiles! Get to a bomb shelter!" is a valid alert.

"Planes Hijacked!" is worthless. What action that could have been broadcast on EAS would have saved a single life?

Oblig

[2names]

I did not hear a test, but I did see a black cat walk by, then the same cat walked by again.

FiOS in DC worked without any issues.

[gcnaddict]

and from what I heard, FiOS all over the place worked without a hitch. It might have to do with the fact that Verizon's infrastructure was built inside the last decade.

Lost Channels

[Anonymous Coward]

The local cable broadcaster here lost approx 10 channels after the test, including CNN, FOX, and DISCOVERY. They all switched to the NAT GEO channel without audio for upwards of an hour after the test ran.

In addition, the test video was jumpy, kept blacking out, audio kept dropping out, etc.

All in all, if it had been a real emergency, losing the 2 major news channels would have been real motivation to start loading ammo and supplies and gassing up the bug out mobile. ;)

Re:Lost Channels

[jafiwam]

What causes fatigue on the magazine spring is tensioning and untensioning the spring. If it's not in motion, it's not wearing. Magazines will need to be replaced over time due to normal metal corrosion, but that's over a different time scale than "loaded with .223 for two years."

Seriously?

[Oxford_Comma_Lover]

So Time Warner NY failed to implement the national emergency system that we use in the event of an *inbound ICBM attack*? When it had been announced for weeks in advance?

Curse their sudden but inevitable betrayal.

Re:Seriously?

[kheldan]

ICBM attack

Seriously: If that happens, you're better off not knowing anyway. At least you won't spend your last few minutes of life scared out of your mind because you know you're going to die in a nuclear firestorm.

Re:Seriously?

[GodfatherofSoul]

Fool! Without that alert, how on earth are you going to talk the closest girl to you into impending disaster sex?

Re:Seriously?

[bennomatic]

LOL. I wonder how many babies were born out of the War of the Worlds radio thing.

Re:Seriously?

[Leebert]

Nonsense. Thousands or tens of thousands could be spared secondary effects from the heat and blast wave. Just the simple act of not standing in front of a window can be the difference between a horrible death and surviving relatively unscathed. Surely you've seen this famous picture from Hiroshima: http://en.wikipedia.org/wiki/File:The_patient's_skin_is_burned_in_a_pattern_corresponding_to_the_dark_portions_of_a_kimono_-_NARA_-_519686.jpg

Yes, if you're sitting at or near the hypocenter, your opinion holds true.

Complete waste

[bradgoodman]

Like probably 99.99% of america, I was neither watching TV or listening to the radio at 2pm on a weekday afternoon.

I do have a cell phone on me all the time, and received no alerts on it.

I can tell you from experience however, that if it were an Amber Alert, I would have been aware of it immediately.

CONCLUSION: EAS is another complete misguided federal program.

Re:Complete waste

[aiken_d]

Because any system with a test intentionally scheduled at a time of low disruption is necessarily misguided. I'll let the IT guys know the restore test they were planning for 2am is pointless because I won't be around to notice it.

Re:I don't understand the purpose

[0racle]

It is also capable of being used locally, NWS uses it like this. This test was simply the first top to bottom national test of the system, this does not mean national alerting is the only function of the system.

Damn straight!

[Anonymous Coward]

We should leave emergency notifications to the free markets! You want to know about disasters and what to do? Well, just subscribe to a disaster notification service. I'm AT&T or your cable companies will provide that service as part of a package of some sort. And we all know what superior service cable companies have over pathetic government!

Re:Damn straight!

[sorak]

We should leave emergency notifications to the free markets! You want to know about disasters and what to do? Well, just subscribe to a disaster notification service. I'm AT&T or your cable companies will provide that service as part of a package of some sort. And we all know what superior service cable companies have over pathetic government!

We already do! There is no law preventing any organization from creating such a service. Our options are "free market and government" or "free market only"...Sorry for posting a serious reply to satirical comment, but one of my pet peeves is when the government steps in to solve a problem and conservatives reply "the free market would have done it faster/better/cheaper".

No, they wouldn't have, and they didn't. That's why the problem existed.

Dentist

[Oswald McWeany]

I was at the Dentists and one of my fillings started broardcasting the alert.

Cox Communications

[ClayJar]

I was watching the test on a friend's Cox Communications cable service, and they also switch to a shopping channel (cable channel 8) for emergency alert activations. Their cable system apparently is incapable of showing the alert on all the (digital?) channels, so they simply show it over analog shopping channel 8 and have a system in place to switch everyone to that channel automatically whenever an alert is triggered. It's a bit annoying if a test is scheduled during, say, an important football game... er... episode of Mythbusters... whatever. On the other hand, it is even more jarring than the alert tones, so you'll certainly know something's afoot.

If you have one of their Motorola digital cable boxes, when it goes into emergency alert mode and auto-switches to analog shopping channel 8 for the message, the front clock display changes to "EAS" as well. If you're suddenly watching the shopping channel and "EAS" is displayed on the cable box *and* you have the wonderfully annoying (and intentionally so) alert tones, you *should* be able to figure out that now's the time to read or listen. At least, that seems to be the general idea.

I did notice that I didn't get the alert over cable until after I'd finished watching it on OTA TV (and chatting about it afterward), so chalk up a minute or two of additional latency to the cable company.

Your conclusion does not fit the facts.

[Anonymous Coward]

1. Millions heard it.
2. Using all communication methods to broadcast a message of national urgency is hardly misguided. It's a common sense idea.
3. They'll add other methods soon enough.

Terrible news

[uncanny]

Some DirectTV customers reported hearing Lady Gaga's "Paparazzi" play during the test.

The terrorists have won

Even better

[publiclurker]

the could have different warning tiers. for the Platinum level, you get a full 30 minutes warning. For the common folk, we have bronze with 30 seconds.