TSA Software Bug Creates Airport Bomb Scare

192939495969798999 writes "An article at CNN's website reports on a serious software bug at the Atlanta airport." From the article: "TSA screeners are given tests around the clock to check their alertness. Images of bombs and other suspicious devices that are hard to detect are put up on the X-ray machine, followed after a brief delay by an alert that reads, 'This is a test.' After reviewing a tape of the images, Hawley said the software failed to alert the screener of the test."

Comments

Fun with false images

[TripMaster Monkey]

I didn't know the TSA employed such software to test their screeners. This incident raises the possibility of tampering with the software to either:

1. purposely display an image of a dangerous item where none exists, inciting a scare like the one witnessed Wednesday, disrupting thousands of lives and paralyzing a major terminal, or:
   
2. display an image of an innocuous item instead of the actual image of the luggage containing a dangerous item, allowing terrorists to smuggle said items onto aircraft. Obviously, this scenario will require far more sophisticated timing of the false image than the previous scenario, but it should still be possible.
   

Given these possibilities, and given the fact that Wednesday's incident proves that such a thing is possible, I'm betting the TSA is currently debating whether or not the decision to make the scanners capable of displaying false images in the first place was a wise one.

Re:Fun with false images

[Ihlosi]

This incident raises the possibility of tampering with the software to either

3. Display "This is a test" right after Mr. Terrorists luggage containing dangerous items has passed through the X-Ray machine.

Re:Fun with false images

[apoc.famine]

Worse than that, it shouldn't be too hard to display the "that was just a test" message on a more and more frequent basis. As the screeners are already familiar with this notice, they'll probably start to become desensitized to it. Then it becomes pretty easy to slip stuff past them.

This is very, very similiar to the "click ok to continue" problem which plagues Windows, and is really the root cause of many spyware installs. If warnings are too frequent, users treat them as irritations that they need to get around rather than important info that they need to read, understand, and pass judgement on. In this case, all that needs to be done is to up the frequency, something that shouldn't be too hard to do.

Re:Fun with false images

[Daniel_Staal]

It is a wise one: it keeps the screeners from getting to bored with their jobs. Since something they have to react to comes up moderately often, they will stay alert enough to react to it. If it didn't the fact that months go by without them having to actually react to any of the bags will mean they stop expecting to react, and then stop noticing what's actually in them.

This is all standard psychology: People aren't good at finding rare exceptions in repetative data. That is one of the reasons we invented computers. Unfortunately, a computer can't spot a weird bomb, so we need an actual intelegnce manning it. That means a human. So, we play these tricks on ourselves to keep those humans working at an acceptable level.

Re:Fun with false images

[C10H14N2]

capable of displaying false images in the first place was a wise one.

I've had design meetings practically come to blows when similarly asinine suggestions were made in the context of things that by comparison were about as critical as a recipe database. Yes, you would think in "system to positively identify bombs" the flowchart box labeled "automatically and without further inquiry disregard positive image of bomb" would raise a few eyebrows. Geezuz.

Re:Fun with false images

[maxwell demon]

purposely display an image of a dangerous item where none exists, inciting a scare like the one witnessed Wednesday, disrupting thousands of lives and paralyzing a major terminal

More importantly: After enough false alarms, the screeners will more likely not react should a real bomb appear. "Oh well, surely just another software fault, just like the three we've had earlier this week. We better don't scare our passengers again ..."

Re:Fun with false images

[Illserve]

It is not only a wise decision, it is essential.

The TSA funds fundamental research in sustaining human performance in search tests to ensure that these baggage screeners are performing well.

One thing that has been found is that the human brain cannot keep searching efficiently for something that never appears, you just tend to zone out. We're not robots after all, and searching day in and day out for a 1 in a million event that may not occur for months or years is not a task we're equipped to do.

By giving the visual system periodic targets, it stays frosty. So some kind of periodic fake bomb is necessary.

Now you can do this in two ways: with real fake bombs, or images of bombs. One of these options is going to cost about 100 times as much to implement as the other and at the end of the day, if properly implemented, both will serve the same purpose. It all comes down to how much security can we get for our dollar, and paying actors to play dress up terrorists and slip fake-bombs through the baggage system is hugely inefficient compared to a software solution.

So you ca argue that the software solution is too vulnerable... but your suggestion is going to need to be accompanied by a list of other systems that can be scrapped to pay for the more costly alternative because it has to come from somewhere.

Re:Fun with false images

[Daniel_Staal]

they've got devices coming out for cars and trucks that test driver awareness far more subtly than just popping up a test picture at random... the software actually monitors the drivers eye movements and other parameters... so there shouldn't be anything stopping them from doing something similar for this x-ray scanner application...

Those can tell you if the driver is awake, but not if they are paying attention. Fortunatly, someone driving a car has to pay attention fairly routinely just to stay in the lane and on the road, so 'awake == aware' (generally) in that situation.

For the this screening application, it is quite possible to be awake while not being aware. They can be fully focused on the screen and just not notice because their brain didn't realize that it had to notice. As I said; this is a well-known failing of the human mind. You are likely to see what you expect to see in a situation that closely resembles a common situation.

Oh, and yes, a good AI would be perfect for this job. Unfortunately, we don't have one good enough for it yet. An AI can spot a known weapon, but not an unknown one. A human can spot an unknown weapon. If they are awake and aware.

Re:Fun with false images

[Anonymous Coward]

No. It's nothing like the Windows "Click OK" problem at all.

With "Click OK", you are expected to respond to a warning/message box. You become desensitized to the warning and just click "OK". The system provides no feedback about this whatsoever. Click OK, and you're finished, you don't learn the repercussions of your act until weeks later (if ever).

With the airport software, the screener has to respond to images of contraband on the screen. In theory, after each test image appears, there will be a message, saying "That was just a test", providing valuable feedback, not only for the screener, but also for his/her boss, who will likely reprimand the screener if they miss any test cases.

No feedback=no learning
Feedback=learning

Re:Fun with false images

[scherrey]

Actually they do the "mystery shopper" thing too. Recently something like 21 airports were testing in this manner and 100% of them failed.

Honestly I'm not terribly concerned about safety. The ONLY reason the 911 terrorists succeeded was because of our policy of cooperating with hi-jackers which was based on the presumption that they wanted to survive the effort themselves. That policy is no more. Frankly I feel we'd be better off if everyone came on board armed with knives or sidearms (if properly trained). Regardless, any terrorist taking on a plane full of passengers these days knows he's going to be instantly attacked from all directions and shot down if necessary. We're putting way too much emphasis on things that have no measurable effect on our safety at serious detriment to our own freedoms, convenience, and financial situations.

inconvenient but reassuring

[God'sDuck]

better than the parallel-universe headline: study shows screeners oblivious to obvious bombs in test images...

There is No Software Design That Is So Good ...

[rewinn]

... that is cannot be implemented badly.

Sounds pretty good to me

[Billosaur]

While screening carry-on luggage, a TSA employee identified the image of a suspicious device but did not realize it was part of routine testing for security screeners because the software failed to indicate such a test was under way, Hawley said.

Willie Williams, the airport's federal security director, said the screener saw something suspicious and notified a supervisor. The two manually rechecked all the bags on the conveyor belt but could not find anything resembling what was seen on the screen, Williams said.

Put aside the software failure and I'd say this was a more successful test than the actual test. I mean, if screeners know this kind of thing is going to happen every so often and they see something suspicious, they may become a bit jaded after a while and assume it's a test, even if the indication doesn't appear. This screener took no chances and called a supervisor and then went about trying to find the device. I believe that's how the system is supposed to work.

So the software failed, but in the end it didn't really fail, because it showed someone was doing their job as they were supposed to be.

Good!

[AntiTuX]

To be honest, I think it's a great thing. Least I know that they're following protocol. The guy did exactly what he was supposed to do.

As for the software, all software has bugs. I'm just glad that someone found out that it wasn't something terrible getting on a plane.

Re:Not clever to desensitise them

[GreyPoopon]

Images to test their alertness sure, but images of bombs? That's just plain crazy. All you're doing is desensitising them and guaranteeing that even if they're alert they won't get the adrenaline rush they should.

It depends on what they do with the tests. If there are severe consequences for the operator if they miss one of the test images, then I doubt they'll be desensitized. On the other hand, if there's no consequence for being a slacker, you'll see a group of operators hudding around the display laughing at the "fake" bomb image while a terrorist walks right on through.

Re:Not clever to desensitise them

[rayde]

should they get an adrenaline rush? wouldn't that lead to potential panic? I think i'd rather if they were able to calmly react to such situations, knowing that most often it will be a test. i think they'll more likely play by the book in those situations, than do something more emotionally driven.

Why malicious items?

[VxJasonxV]

Why put in images of bombs and such? Someone eyeballing that that isn't a screener would blow a gasket if they saw it.

How about pictures of assorted dildos/vibrators? No, I'm serious. That'll catch your eye, male or female.
How about 'to scale' midgets (wow, that sounds awful... as much of a joke as it is) fighting in a mini suitcase?
Or a very carefully and perfectly laid out bra of panty?

Seriously, give these people something they wouldn't mind seeing (well, sans the dildo/vibrator) and you'll get (1) a chuckle and (2) some extra energy for productivity.

You know, on second thought, I'm going to patent the concept, brb.

Re:Why malicious items?

[akozakie]

Even better: a nice, well laid out bra, a large dildo, and... a less exposed pipe bomb, or something like that. Now that is a test! Plus, it's likely to happen - if you're going to risk getting a suspicious item on the plane this way, the least you can do to raise your chances is provide a distraction.

Re:Why malicious items?

[LnxAddct]

You wouldn't recognize a bomb in a screening device if it was slapping you on your face. You have to be trained to pick these things out, they are usually hidden very well and shaped to look like they belong with everything else. It isn't like a cartoon, you don't see 8 sticks of dynamite tied together with an alarm clock on top. Some explosive devices I guess you might recognize just by shear suspicion and it appearing to be out of place, but a good chunk of them are much harder to decipher, especially using screening equipment. And to counter you're other point, people shouldn't be in a position to be peeking at the screening device anyway. Maybe a quick glance, but if someone is standing at the edge of the security area glaring at the screen, I think that'd be something more worth being concerned about.
Regards,
Steve

Re:Not clever to desensitise them

[TrappedByMyself]

That's insane. Images to test their alertness sure, but images of bombs? That's just plain crazy. All you're doing is desensitising them and guaranteeing that even if they're alert they won't get the adrenaline rush they should. What brainiac thought this one up?

That doesn't even make any sense. This is training, you WANT people to see these things. You WANT them to have experience reacting to stuff they think is real. How do you expect them to identify bombs in suticases if they've never seen examples, especially in real world situations. Watching films in a classroom is nice and all, but not real enough. That "desensitising" comment is out of touch. You would rather them get an adrenaline rush and panic as opposed to getting maybe less of a rush, but have the experience to handle the situation?
You need to learn the difference between education and training. Education is a good start, but "first responder" types need training to know how to apply their education to a crisis situation.

"calm" is not "desensitized"

[ScentCone]

That's insane. Images to test their alertness sure, but images of bombs? That's just plain crazy. All you're doing is desensitising them and guaranteeing that even if they're alert they won't get the adrenaline rush they should. What brainiac thought this one up?

The same ones that know that combat simulations help cops and soldier generally make more level-headed decisions. The same ones that know that simulating in-flight emergencies in flight simulators takes the "holy crap!" out of handling such things. There are VERY good reasons that you want your bag screeners to be able to react calmly or subtly to what they see on the screen in front of them. They may need to be able to signal armed support, depending on their assesment of the person in line, without Freaking Out while they're looking at their equipment. These are supposed to be professionals, and it sounds like the person involved acted like one (absent the "this is a test" message).

Re:"calm" is not "desensitized"

[syousef]

*shakes head* I can't believe lots of people are saying things like this. No offence but you have no idea what you're talking about.

To use your own example you don't simulate in flight emergencies on real flights. You do it in a controlled environment usually in a simulator. If you don't have access to that or want to do more realistic simulations you're very careful about recovery conditions (eg. you simulate an engine failure by throttling back to idle, but you don't actually cut your engine).

Similarly its only in the movies that you train soldiers and police by making them think their friend's just really been shot. In the real world you do controlled excercises that are separate to normal day to day operations to avoid psychological trauma and desensitization.

In the case of these baggage handlers they should be able to identify the bomb and calmly deal with the situation but the adrenaline should be flowing nonetheless.

I do agree with you on one thing. Yes the person did the right thing given the circumstances.

Re:"calm" is not "desensitized"

[HardCase]

To use your own example you don't simulate in flight emergencies on real flights. You do it in a controlled environment usually in a simulator. If you don't have access to that or want to do more realistic simulations you're very careful about recovery conditions (eg. you simulate an engine failure by throttling back to idle, but you don't actually cut your engine).

Unfortunately, there are plenty of situations where training has to occur in situ for it to be realistic. Obviously you're not going to perform in situ training that creates a life or death situation and this case wasn't one. At worst, it has the potential to create a very inconvenient situation and I guess that this case turned out to be just that.

As an example of in situ training, when I served in the US Navy, we had the capability of injecting simulated sensor data into the real-time sensor stream of our sonar system. The data is indistinguishable from the real thing, so the potential exists for that information to go out over the world-wide tactical data system. It was an excellent tool for training and to make sure that operators were actually paying attention to the screen - generally speaking, on a surface ship sonar watches are brutally boring, unless there is an actual target to prosecute. The downside to the tool is that if it's not used properly, a whole lot of panic can ensue for a short time (and, since shit rolls downhill...)

That particular training tool was very successful, by the way. It increased the proficiency of the operators significantly.

-h-

This Braniac did

[Illserve]

What brainiac thought this one up?

Jeremy Wolfe, possibly the world's foremost expert on human performance in visual search tasks did.

You can read about his research on his publications page here.

http://search.bwh.harvard.edu/recent_publications. htm

Check out the one called "Rare items often missed in visual searches. " This research, among others in the field, is funded by the DHS for precisely this purpose. May I add that the turnaround time from primary research to application is excellent. Jeremy and his lab are to be commended as an example of how pure research can contribute directly to the public good.

And why would you want an adrenaline rush anyway?

Worked for TSA

[Anonymous Coward]

I worked for the TSA for a year and it was important to see the images of bombs and knives and grenades to keep people on their toes. In case you're wondering about the machine itself it's a german machine running linux and is updated by zip disks. So if you want to put new images in, which they do quite often, then it is put in through there.

The bombs by the machine are often obvious and are placed in funny spots where normal packing wouldn't be, so it's usually fairly easy to identify them.

Re:Not clever to desensitise them

[honkycat]

There's no indication of how often these false images are injected, so it's not clear they're being "bombarded" with false events. If it's too many and there's no penalty for missing a few, then it's a bad move. However, 99.99% or more of all airport screeners will never see a real event. It's not something you're going to get experience seeing or handling if there are not drills.

The only way to test the screeners and keep them alert is to give them events to respond to. The problem with the system as described in the article is that it sounds like only the machine knows that a fake event was generated until an audit later. Really, the people who the screener would call should be notified ahead of time that there is a fake event. That would prevent escalation. If this is done, though, there must also be an identifier of some sort attached to every image so that they don't mistake a real event report for an anticipated false one.

As long as the screeners are seriously penalized for failing to respond to any false event, this is not a bad thing. It's absolutely nothing like your "live ammo" analogy. A false positive event like that which occurred is acceptable. A few people were inconvenienced by an airport shutdown and nobody gets hurt. Imagine the consequences of a false negative.