Slashdot Mirror
The Tech Failings of Hawaii's Missile Alert
Over the weekend, Hawaii incorrectly warned citizens of a missile attack via their phones. According to The Washington Post, the error was a result of a staffer picking the wrong option -- missile alert instead of test missile alert -- from a drop down software menu. Hawaiian officials say they have already changed protocols to avoid a repeat of the scenario. The report goes on to add: Part of what worsened the situation Saturday was that there was no system in place at the state emergency agency for correcting the error, HEMA (Hawaii Emergency Management Agency) spokesman Richard Rapoza said. The state agency had standing permission through FEMA to use civil warning systems to send out the missile alert -- but not to send out a subsequent false alarm alert, he said. Though the Hawaii Emergency Management Agency posted a follow-up tweet at 8:20 a.m. saying there was "NO missile threat," it wouldn't be until 8:45 a.m. that a subsequent cellphone alert was sent telling people to stand down. Motherboard notes that new regulations require telecom companies to offer a testing system for local and state alert originators, but because of lobbying by Verizon and CTIA, this specific regulation does not go into effect until March 2019.
In a piece, The Atlantic argues that the 90-character messages sent by the system aren't suited to the way we use our devices.
In a piece, The Atlantic argues that the 90-character messages sent by the system aren't suited to the way we use our devices.
You need a mechanical physical switch with a switch guard. The very fact that an actual alert would be triggered by a menu item, indicates a completely incompetent design. I seldom call for people's jobs, but I'll make an exception in this case..
The shepherds did so well protecting the flock that the sheep no longer believed that wolves existed.
They need to add some AI:
"Hi, I'm Clippy! Are you sure you want to send a missile alert?"
"No, Clippy"
"OK then, launching missiles".
If the selections were in the same menu then that's just horrible UI design. I assume both selections require a strong confirmation of the action too.
For all we know this menu-option-no-confirmation approach was dictated during a 'pair programming' session with an over-the-shoulder manager.
Requiem for the American Dream
Yes, there are tech failings in this incident. There were also human failings. Let's not let the tech failings overshadow the human ones.
I mean, sure, let's get better tech solutions for this. But we can't ignore the fact that the President, who tweets about anything that upsets him, couldn't be bothered to interrupt his golf game to say that this was a false alarm.
Mr. Hu is not a ninja.
Seriously, contact all the major TV and radio stations in the area first. The expectation that everyone should get critical information from "social" media is a joke.
Who hasn't had the same issue with drop-down menus in standard software? Unfortunately there is no 'bitch-slap' feedback button to the designers or the software producer.
Menus are designed with so called logically ordered groups, but in many cases have things underneath each other which look the same, but have different effects. And a slip of the mouse sometimes makes the wrong selection.
No, you're wrong. UI design plays a major role in the correct operation of a system. Very few people in my experience are detail-oriented people, and even the ones who are still make predictable mistakes. The system must account for how real people actually behave. To do otherwise is bad system design. Looks like this was just a test of connectivity. I don't know why they didn't automate the test (send a test file once every 8 hours, write in the log that it got sent, and write in the log that a confirmation came back, then have another job that looks for those log entries in the appropriate time range and alerts the operators if it didn't work). Yes, you still need to manually test, but not as often. In a case like this, there should be a prior action required to "arm" any of the "real" messages, so there's two different processes that you won't mix up. A generic "are you sure" query isn't good enough because it's the same message whether you picked a real message or a test message. Muscle memory kicks in and you just click Yes, after all that's what you did the last several hundred times.
"I have never let my schooling interfere with my education." - Mark Twain
Everyone is talking about bad UI, and they are right, but isn't the bigger problem that this is all being tested in production? Why does the "test missle alert" option even exist in production -- that should be in a sub-prod system that isn't actually connected. Maybe it has something to do with how the EBS works but seems ridiculous to me to even have those two options in the same system.
Seriously, contact all the major TV and radio stations in the area first.
Which should take some time, unlike sending a tweet on an account already owned by the emergency center.
Also, the contacting of TV and Radio station might be hampered by people actually attempting to follow the instruction of the previous wrong alert.
Though most TV and Radio crew might wonder how come there's an alert about a missile attack on their *phones* while, at the same time they do not receive a full list of information that they have to broadcast immediately to the population while interrupting the normal programming.
So, while the HEMA guys are heading for the simplest thing to do to communicate information (blasting it on accounts that they actually own, like Twitter), the TV and Radio station should be the one trying to contact HEMA to understand why they weren't asked to broadcast any emergency information (it might have been an error like in this case. Or in the alternative case of an actual live attack, the general population might be missing critical information that the Radio should have been broadcasting and that got stuck somewhere in the process).
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
Give that some systems are worse than others in inviting operator error, you can't just assume it's not the tech because operator error was involved. However even if the tech is as good as humans can possibly make it, that still wouldn't prevent operator error.
This kind of fault is hard to test for, because it's a non-functional requirement. You can't simply do a functional test and check off "prevent accidental message from being sent". At best you can simulate various scenarios, but those simulations are unreliable because you're dealing with testers, not people who are habituated to the system and who thus use it differently.
Clearly there were several kinds of operational faults here that may have been compounded by design flaws. But one of the operational mistakes was purely a matter of planning: not programming in a "false alarm" message to be sent after the inevitable operator error. This also suggests a design shortcoming in the system in that designers didn't anticipate the need to ever issue an ad hoc message on short notice.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
I wonder how many doses of Plan B were used this weekend?
It would have a menu with consecutive items reading "kill prisoner" and "release prisoner".
Starships were meant to fly, Hands up and touch the sky - Nicky Minaj
The UX probably already asks if you are sure you want to send something.
Ah, the old AC favorite - I'll just make some facts up which I assume are true.
Starships were meant to fly, Hands up and touch the sky - Nicky Minaj
What the dialog box probably was:
Send Message?
Test: (check box...off by default)
(Send Button)
When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
the alert wasn't sent by WOPR.
Is it because we put art majors in charge of UI design? Is that it?
Could be.
I've almost given up on pushing back on UI design.
Their two overriding and incompatible drives are to 1. hide complexity and 2. make things super easy.
The result is that it's super easy to do things that you don't understand.
I don't pride myself on UX work (mostly write tools for other engineers) but seriously, wouldn't you have a different (and much harder) confirmation for "Real Missiles Ahoy" vs "This has been A Test?" e.g. confirmation modal dialog box for the test (Are you sure? yes/no) vs. do a CAPTCHA && type a word && click something that moves around, etc. If everything in the system has a same (or materially similar) confirmation mechanism, you are basically training folks to ignore it; this is why only serious things (delete, etc.) have them.
In Soviet Russia jokes are formulaic and decidedly non-humorous.
You're right. If I had mod points, I'd give you a bump. Your insight that the blessing here outweighs the cost is one I haven't seen given enough attention. Fresh eyes will be looking at how the process should work to prevent mistakes and that's a good thing. Likely they'll find other areas that need improvement.
Using a system intended for conveniently notifying the public with information to instead notify the public of an emergency is a dangerous mistake, one of which they're now aware. Finding out that the public doesn't know how to respond is priceless information that they have now. The guy who clicked the wrong menu option may not deserve a medal, but put him on the committee determining how to fix the system and plan responses. Redemption is a strong motivator.
Now the public knows that they need a response plan for such an emergency. Having public pressure to get prepared is perhaps the greatest thing that could happen. People trying to get the public prepared would have been frustrated before this, but now they'll have the public on their side. That's the kind of thing that makes budgets happen.
B) Eliminate all the stupid users. This is frowned upon by society.
Judging from the number of "posting to undo mis-moderation" posts I've seen, maybe Slashdot could learn from this fiasco and group the "up" and "down" moderations in the drop-down list.
While it was certainly a bone headed mistake, it was one what was easily possible for someone in a hurry. As this fellow was just wrapping up his shift, he was clearly trying to get everything done in time.
It this was indeed the setup the mistake was idiotic programming and software design. The end user screwing it up was entirely predictable and probably inevitable. The problem occurred when the system was designed. If a system can fail because of the design, it almost certainly will fail sooner or later.
Part of my day job is to write work instructions and design procedures. When something goes wrong the first question I have to ask is "what did I do wrong", NOT "who screwed up"? 90+% of the time the problem was unclear/wrong/misleading instructions, a badly designed process, or some other problem where the person tasked with carrying out the instructions was set up to fail. In other words, my fault. We as engineers tend to take too little responsibility for our own failures and blame user error when in fact the error was a badly designed program or procedure. We tend to think we are the smartest people in the room and while that may be true sometimes it doesn't mean we are perfect.
What failed was the operator not paying attention to their work.
If you're designing software without assuming users occasionally make stupid mistakes, then you, the UI designer, are both lazy and stupid.
"you're dealing with testers, not people who are habituated to the system and who thus use it differently."
This is a direct violation of Agile. developers should be given close contact and ability to collaborate with the end users. Not having 5 degrees of separation between developers is the key to bad software. You end up playing telephone and with no understand of the real problem.
I am appalled at the SW development I have seen in large SW companies. the waste, mis-management, slippage, and distain users and developers develop for each other. Layers of management who seem to have as a purpose only in preserving their jobs.
That's why I prefer smaller companies with in-house projects, as opposed to larger companies with outhouse projects.
This is
putting the 'B' in LGBTQ+
No, we put BAs in charge. Arts majors would probably know more about it than your typical manager.
putting the 'B' in LGBTQ+
They are concerned enough to spend money on the warning system, but have they spent the money on enough bunkers to hold the population of the islands? Are they located so that everyone has a reasonable chance of getting to one regardless of traffic/panic of everyone else trying to get there?
A reasonable question with and unfortunately unreasonable answer. You have to weigh the costs of providing such shelter against the likely benefits. Odds are you'll find that building and maintaining such shelters is too costly to justify even presuming they would work as intended. (and it's not clear how useful such shelters would be) Folks in Hawaii are thinking about the problem seriously but the answers aren't simple ones.
Force people to think.
HA! Good luck with that. In my experience far too many people will fight tooth and nail to not have to engage their brains.
I really hope North Korean UI designers made a separate button for "Wipe Seoul Off the Face of the Earth" and "Test Wipe Seoul Off the Face of the Earth"
Open Source Network Inventory for the masses! Kuwaiba
No single person can be blamed for that, however.
File under 'M' for 'Manic ranting'
In addition to a poor design it is a happy path system and doesnâ(TM)t account for real world exceptions. The project obviously chose fast and cheap over good.
First of all, Agile doesn't work in every situation unless you stretch the definition to include non-agile practices where warranted. Second, the distinction between users and testers isn't as clean as you suggest. Users *are* testers until they become habituated to the system.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
At least they now know it works for all users, not just a few test users.
Don't fight for your country, if your country does not fight for you.
I was there, vacationing in Hawaii. Got the alert, noted the time. Then I finished eating breakfast while listening to the morning riot of birds as the last hints of sunrise's color faded into daylight. I'm old, and my kids were thousands of miles away...
The biggest failure wasn't the bogus "BALLISTIC MISSILE THREAT INBOUND TO HAWAII. SEEK IMMEDIATE SHELTER. THIS IS NOT A DRILL."
The biggest failure, in my opinion, is the waste of a whole lot of tax money on stupid shit, and unaccountable politicians, government agencies and defense contractors that have inspired zero public confidence.
If the military can only 'maybe' defend it's own assets against such an attack from an adversary such as North Korea, why are we allowing our elected leaders to spend so much money on a questionable approach to national defense?
Why would any rational entity ever deploy a weapon system without also committing to developing a means to defend against it? An no, mutually assured destruction has not been, is not, and will never be a 'defense'. It's simply a guarantee of more destruction.
"Every time I see an adult on a bicycle, I no longer despair for the future of the human race." - H. G. Wells
The test can go at the click of a button, but the "Live" message should have an "Are you sure you want to send a LIVE message? This is NOT a test!" prompt before shooting out.
If nothing else, the chaos caused should be used as a talking point so people are a little more prepared for a live event. Though I doubt that will happen.
Why the hell do they have the authority to send a live message, but not the authority to send a false alert message? That was a dumb decision.
Yea the guy made a mistake. It sucked, but he learned from it. So move on. If you fire him and bring in a new inexperienced guy who could make the same mistake down the road you're not doing yourself any favors. The original guy will make a point of not doing that again.
I refuse to sign
This system seems to have been designed by programmers who made a habit of ignoring error conditions, This is called incompetents, I think.
Mistakes made by people are still "error conditions" and should be handled as needed. 8-)
Humans are part of the system. If a human error is made, then the system design has failed.
Blaming it on the operator is a "pointy-haired-boss" excuse, even if it is (partly) true.
If the summary is true....that an incorrect drop down was selected...then why did it take 37 minutes to correct? Please explain why it wasn't corrected immediately.
Something is fishy here....
Yeah, well, in his defense the system was Emacs-systemd, and he got a little confused as to whether it was C-x C-m M-a or C-x C-m M-t in the current runlevel...
(Yeah, couldn't decide whether to hate on emacs or systemd, so here's both!)
Have gnu, will travel.
Wrong. The screw-up here is a system that makes it far too easy to screw up. It was bound to happen given this abysmally bad design.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Classically, you have to break a piece of glass or at the very least turn a key to trigger something like this. The UI design bears all of the blame here. It was asking for something like this to happen. It is absolutely no surprise it happened. The ones at fault are the ones that did design this broken UI and the ones that signed off on it. These should at the very least lose their jobs and probably face criminal penalty, because negligence does not get any more gross than this.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
At the AGU meeting meeting last month a talk said that 75 character message was optimal. Some ancient computers still have 80 character buffer. A short, direct message like "A nearby earthquake has just occurred. Take cover." Messages with more details could be broadcast later. Damaging earthquakes have shorter warnings versus ballistic missles- 5 to 120 seconds vesus 17 minutes. It is based only a single station impulse be interpreted as an earthquake with a rough magnitude estimated. More precise determinations of quakes like at the NEIC require hitting several stations across the world. This is too late for a warning in most cases. The US is fourth country to implement a quake alert system. (Painfully slow due to low funding.)
You need a mechanical physical switch with a switch guard.
No, you DON'T!
If you had such a switch, pushing it would have to be part of the test. Otherwise you've created a single point of failure that causes the live function to fail even though the test psses - and you don't find out until the missiles are inbound.
Yes, they should have done things like word and position the menu items differently, so hitting the wrong one by accident was less likely, and have glaringly different text and graphics (by selection, with the function still identical) for the confirm popups. But the further the test and live functions diverge, the more opportunity you have to build a system that passes the tests but doesn't work when you need it.
Conelrad (cold-war predecessor to the Emergency Broadcast System) had a similar failure: The test and inbound-nukes kickoff keys were paper tapes on adjacent pegs, and one day the low-ranking communications guy put the wrong one in the teletype tape reader on weekly test day, telling the whole country to duck and cover. Nothing new here.
(The teletypes had a bell and the newswires had a number-of-bings code for how urgent a message would be. I think major stories rated about a three. Max was ten, which was reserved for nuclear war warning activations. I recall one time in '65 or '66 when the AP wire tape got stuck on the bell code and that thing rang something over 30 times before they got it unstuck... Fun times.)
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
The USA seems to be surrounded by these failing island nations like Puerto Rico and Hawaii. They just can't get their act together. Very sad.
The system should _never_ _ever_ send "false alarm".
Sure, but how do you design a system where that never happens?
Adding ad hoc messages is an even worse idea. At some point a politician will get the bright idea of using the system for trivial notifications (like "air quality alerts") that are better handled via other channels.
And how do you design a system that cannot deliberately be misused?
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
To sound a "real" alarm should be easy, and 99.999% idiot proof.
It should rely on the minimum amount of technology, as we don't want a situation where the alarm can't be sounded because the software crashed or the mouse has dust on the sensor
The activator for the real alarm should be a large red button, which sits in a red box with the instructions:
"break the glass, and hold the button down for 5 seconds",
next to is there should be a white button with which can't be mistaken for the "real" button with the text:
"hold the button down for 5 seconds to sound the test alarm" if it is a part of the procedure an alarm cancelled message should be sent automatically
This analysis was posted by Scott Roberson, Chair of Information Systems & Computer Science, U.Hawaii before the screen shots and related information was available. Even with that info, his analysis is relevant and insightful... https://medium.com/@scottrob/h...
Have a GUI that works.
Verification and a clear command structure to initiate an alert.
Domestic spying is now "Benign Information Gathering"
Independent confirmation?
A generation of gov/mil/clandestine services workers are back to their cabins (bunkers) in Idaho, Montana, Wyoming, Oregon and Washington days before?
NORAD will get US leadership to a safe place.
DEFCON get the US mil ready.
Civilian notification would not be part of that kind of alert as its not linked.
A tsunami warning system would be what an Operations Centre would be doing.
Domestic spying is now "Benign Information Gathering"
But at a higher level, why is this-- as our duffer in chief calls it, "...purely a state exercise?" Isn't national security a national issue-- provide or the common defence or something like that? Apparently not in Hawaii, Puerto Rico, California or those pesky blue states.
If in this age of de-federalizing and privatizing, POTUS wants to pass the buck and treat this as a state issue, why does the Hawaii Emergency Management Agency have that "DOD" prefix? Imagine if we spent $80 million (roughly the DOD's ED expenditure) on a national emergency communications system and as a condition for consuming public RF bandwidth and government-subsidized internet infrastructure, the communications cartels (Comcast/TimeWarner/AT&T/Verizon/Disney...) would provide a channel for this information.
The other option is to follow the "every man for himself!" libertarian approach of the Trump fork of the Republican party. In which case, I'd like to direct you to our subscription-only missile warning communications service where for a monthly price of $59.95 per family member ($25.95 for pets), you too can receive notification of impending doom. (Ask about our premium $99.95 astrology-assisted version where you'll receive missile notification 14 minutes earlier than all of your neighbours!)
Back 30 years ago, there were shareware games. These were basically "demos" or the first level, distributed freely everywhere, and encouraged gamers to copy them to friends, in the hopes that you'd then buy the complete game.
One game in particular -- and I may never remember which -- had a splash screen. This splash screen described the shareware concept. But, instead of "press enter to continue" (this was back in the keyboard-only days), it forced the gamer to type the sentence: "I support the shareware concept" in order to continue. Simple, effective, I remember it thirty years later.
Any button can be pressed accidentally. Any two buttons can be similarly pressed in sequence. Any swipe. Any confirmation. Hey buddy, I need you to press this button too -- also goes without a second decision-maker.
On attack subs, two officers, each with a key, standing twenty feet apart, both need to turn the keys together. But what makes that so much better is what came before -- breaking the glass and revealing the code and confirming the radio transmission.
If you're going to send a message to a million humans, it's never going to be good enough to confirm the sending of that message. In this case, they wanted to send a message, so they confirmed sending the message. Any number of humans would have backed up that decision.
What they needed to do was to confirm the message itself.
So, here's my thought. For all messages that the system is going to send to the public, (i.e. not test messages) the operator simply gets to re-type the message as displayed on-screen. Five seconds to type that message. And I promise, if I were to find myself typing that message, I would have understood what I was doing.
Each letter is effectively a confirmation. So that's what, fifty confirmations of the message content itself. And of the fact that it's being sent. And of the fact that it's going out to the public -- because tests aren't confirmed like that.
The real problem now is the very simple notification fatigue. How long will it take for you to believe it next time?
Not to mention the fact that your government just terrorized its own people -- making your government a terrorist organization.
How many times have you been to a website, and after selecting something from the last drop-down box, you go to scroll further down the page, but your scroll wheel is still focused on the drop down box?
You end up changing its selection (barely noticeable since your eyes have moved away) before you finally move the mouse far enough, or click off the boxes, or just scroll to the end of the drop down list, and then it finally does what you expect, scrolls the page, with the change to what you had intended already scrolled up out of sight.
I could see that happening here.
To put a positive spin on it, they also know their system works as it should if a real one was required. I mean that isn't nothing. Obviously they need some safe guards in place. As you say, it is likely a bit of a heads up for public awareness of preparation (or lack thereof currently) as to what to do in such an event (apart from bending over and kissing your ass goodbye, or looking for a good vantage point to watch the final show).