Slashdot Mirror
Software Bug Behind Biggest Telephony Outage In US History (bleepingcomputer.com)
An anonymous reader writes: A software bug in a telecom provider's phone number blacklisting system caused the largest telephony outage in US history, according to a report released by the US Federal Communications Commission (FCC) at the start of the month. The telco is Level 3, now part of CenturyLink, and the outage took place on October 4, 2016.
According to the FCC's investigation, the outage began after a Level 3 employee entered phone numbers suspected of malicious activity in the company's network management software. The employee wanted to block incoming phone calls from these numbers and had entered each number in fields provided by the software's GUI. The problem arose when the Level 3 technician left a field empty, without entering a number. Unbeknownst to the employee, the buggy software didn't ignore the empty field, like most software does, but instead viewed the empty space as a "wildcard" character. As soon as the technician submitted his input, Level 3's network began blocking all incoming and outgoing telephone calls — over 111 million in total.
According to the FCC's investigation, the outage began after a Level 3 employee entered phone numbers suspected of malicious activity in the company's network management software. The employee wanted to block incoming phone calls from these numbers and had entered each number in fields provided by the software's GUI. The problem arose when the Level 3 technician left a field empty, without entering a number. Unbeknownst to the employee, the buggy software didn't ignore the empty field, like most software does, but instead viewed the empty space as a "wildcard" character. As soon as the technician submitted his input, Level 3's network began blocking all incoming and outgoing telephone calls — over 111 million in total.
Check the spec - perhaps it was by design or not called out to ignore empty entries?
A null/blank input taken as a wildcard is certainly not a feature.
Even labeling that as a mere bug is putting it mildly. More like gargantuan fuck-up.
Ha ha. No, this is not just a bug. The fuckup goes much deeper than that. "An empty field acts as a wildcard" is the least of your problems. It may or may not be expected behaviour for a GUI. "Not finding it during testing" is par for the course for GUIs for this sort of thing. You're not supposed to give wrong input, even accidentally!
The real problem is thinking a GUI is appropriate to feed lists of boring numbers through. By hand, no less. It's way too easy to accidentally leave a field empty or --if it's a micro-managing form like windows IP address entry type things-- copy part of a numer in the wrong line, shift it a sub-field, or something else similarly silly.
What we have here is a mismatch between user interface and purpose, cooked up without thinking. This is the same mode that makes users stupid, but now it was the designer who wasn't thinking. The focus was on "getting some input fields done", not on "how will this be used and what might the consequences be?" The deeper problem is TFIing such lists. GUIs are entirely stupid for this.
Compare "here, have a GUI" with this sequence: Check the list then feed it to the system as a textfile. It gets queued. Then check the list as it appears in the system against your original list. The system probably should make explicit just what it will do with each entry, like "block one number" or "block a range of numers". Possibly have someone else look over the proposed actions. THEN activate it.
So the problem is that the workflow is entirely too stupid to live. And it was shaped into that form by a GUI.