Slashdot Mirror
False Positives, Few Matches Plague 'No-Fly' List
lindner writes "According to a recent article in the San Francisco Chronicle, the United States No-Fly List uses a soundex algorithm to match names. Designed 'to quickly summon passenger names or to catch deal-hunting passengers making duplicate bookings.' The system has only managed to rack up a slew of false-positives, including everyone matching soundex ("J. Adams") at one point in time. The problem has gotten so bad that there is now a "Fly List" for chronically misidentified passengers."
I understand that the airline industry is a little tight right now, but that's just insane.
Unfortunately, the officials implementing a system such as this are going to get crucified either way. If they let a known terrorist onto a plane and a terrorist act happens, their heads are going to roll. Every journalist will be screaming that, "this terrorist has been on the FBI watch list for 2 years, a simple misspelling of his name allowed him to foil the multi-million dolar no fly system".
On the other hand, false positives are going to make the system useless as the boy who cried wolf one too many times found out. There doesn't seem to be an easy solution to this problem.
The more you know, the less you understand.
It should be obvious to anyone that any mechanism designed to target a small group out of a large group will would have to have an extremely small false positive rate to be of any use.
And the false negative rate had better be small, too.
Something 99% accurate is far from good enough; if only 0.01% of possible individuals are actual targets, you'll be getting 100 times as many false positives as correct positives.
Will I use my alias name which is Alain Williams, or will I use my real name which is Osama Bin Laden the next time that I book a flight to the USA ?
The trouble with this sort of thing is that it inconveniences Joe Public while doing little to deter a real terrorist.
The same yahoos who came up with the no-fly program will be in charge of future lists, such as those for traffic stops, and lists that flag people for enhanced surveillence under tia-like programs.
I wonder if there is a higher incidence of wiretapping done on homes that have residents named "J. Adams."
So you see nothing wrong at all with a customer reserving a seat on multiple airlines for days on end, possibly preventing each airline from making a legitimate sale, only to "release" those seats a couple days before the flight after they have finally decided who had the best deal.
It's fine to go deal-hunting. It's something entirely different to hold something of value hostage while you're doing it.
Customer: "Yeah, I like this shirt, but I'm not sure you guys are selling it at the best price. I'm just gonna take the shirt with me while I shop around, mmm 'k?"
Your point is very valid if there is a reasonable and rational discussion of the tradeoff's - You know kind of Type I and Type II errors. But the Bushies don't believe in that. Goebellian Ashcroft said that they are willing to use every legal tool available to them to achieve their goals - even if it means ignoring the spirit of the law, and reinterpreting the letters of the law to do whatever they want.
The willingness, in fact eagerness, to overlook collatoral damage is the Hallmark of the Bush Administration. They have rammed policies that wouldn't pass muster support anywhere. It is almost as if they are willing to kill 9 innocent people to prevent the 10th guilty one from escaping.
This mentality shows up in the No Fly list. It shows up in how the Arab immigrants were rounded up, and are now being deported by the thousands. It shows up in how to get to the Saddam "WMD's" they were willing to slaughter Iraqi's. Two or 3 Sept 11 bombers entered with Student visa's so everyone on that visa now gets grandly screwed.
So, logic applies only when the hysteria subsides. If you want you can never let the hysteria subside. And Donald Rumsfeld is a genius - almost lunatic - in that. Like he said in almost poetic form, on Feb. 12, 2002, Department of Defense news briefing, (which means that he could use the concept described in his "poem" below prove anything that he wants - it is almost like dividing by Zero.)
To see a world in a grain of sand, and then to step back and see the beach where the sand lies
One of the major clinical automation systems used in American hospitals uses soundex as a primary matching algorithm for patient lookups in the admitting department. Everyone is smart enough not to use it for names like "Juan Garza," but for names like "Steve Franklin" the chance of getting false-positives on your search algorithm is REALLY high. This is largely because of how the system itself implements things.
Two notable occasions have occured where patients were admitted as the incorrect "Steve Franklin" (name make up for use here, of course). Needless to say, this might be a bit of a problem when the medical and nursing staff then takes that admission record and looks back at labs, radiographs, and such ON THE WRONG PATIENT.
Of course, this same "highly advanced" system is really just a set of SQL tables that don't even use variable lengths for fields like comments (instead restricting the user to something obscene like 38 characters). The user interface is really just a Curses program that reads the columns on the table and displays them, allowing the user to edit them. Nearest I can tell, SQL functions handle all the data verification and such, and don't even do a good job at it.
I've worked with this computer system for four years, suffering through it's stupidity.
The point is that one should never assume that sucky, disgusting software is written by first year comp sci majors. There are enough professional programmers out there to cause a far bigger disaster.
Never underestimate the power of stupid people in large groups, or in corporate culture.
Who cares what algorithm they use? Why someone would support a 'No Fly' database is beyond me.
I think people are either criminal (means they should be kept in prison) or not.
Guys like you make me really afraid. For you it's only a technical problem, is it?
You US folks could really do with a constitution to stop this sort of crap happening. Oh wait, you do have one. Oh well, back to the drawing board. Land of the free indeed.
...as long as they're barred from entering the cockpit. The success of the 9/11 attacks can mainly be credited to 1970s-era hijacking guidelines directing pilots to comply with the terrorists' demands, on the assumption that they were going to fly the plane to Cuba or something similar, rather than use it as a weapon. Those guidelines made sense in their time, but clearly, they're no longer applicable.
Here's an idea -- instead of inconveniencing millions of innocent passengers, how about securing the cockpits instead? So long as the pilots remain in control of the plane, it's a flying prison for anyone who commits any criminal act back in the passenger compartment. Let the cockpit crew notify the ground of a failed terrorist attack and land the plane at the nearest airport, with the police and FBI waiting. End of story.
Perhaps if airlines weren't so elusive about their pricing, potential passengers would be able to easily compare various flight options without having to do this. But obviously it's in the interests of the airlines to keep passengers in the dark.
I had to change a flight that I was booked for a couple of months back, and I couldn't even get them to give me a firm figure on how much it would cost to alter it until I'd committed myself to doing that. Now that is ridiculous.
-- Even if a god did exist, why the fsck should I worship it?
Interestingly enough, the original Soundex was based on English language only. So when feeding it foreign names, it will obviously match names from different languages that in reality are far from sounding alike. Admittedly, their algorithms are merely based on Soundex and maybe a bit better.
But to me, finding terrorists by checking their names against no-fly lists sounds just about as useful as checking IP packets for an Evil bit, doesn't it?
Do they really think that they'll stop terrorists by asking them their names and refusing service if they appear on a list?
Like a terrorist is going to walk up to the counter and say, "Hello, my name is Ibrahim Salih Mohammed Al-Yacoub and I'd like to buy a one way ticket from New York to Los Angeles, preferably on whichever flight has the fewest american infidels. Oops, did I say infidels? I meant passengers, good american passengers."
Terrorists just have to use fake names, or steal someone's identity.
As for soundex, it's a very useful tool for matching words based on how they sound. If someone asks you to search for "alan", you might type in alen, allen, ellen, etc. and still find what they're looking for. This is just a case of it being used in a foolish manner.
Here's a list of the most common ways that people died in the US, 2000:
Total: 2,403,351
"Terrorist attacks", if listed using 2001's figures (~3000?), would be clumped into the "other" category, being 10 times less dangerous than suicide, and 5 times less likely than regular old homicide. "Terrorism" is a whopping 240 times less likely to kill you than heart failure, and would account for a mere 0.12% of all fatalities.
Think about it another way - every 2 days, more people die through heart failure than were killed in the WTC disaster. Worrying about "terror" is only likely to increase your odds of dying of heart failure or hypertension.
handguns killed 11,000+ people in the U.S. last year.
Really? How did they manage that?? I always thought of handguns as fairly inanimate objects!
I've had enough abrasive sigs. Kittens are cute and fuzzy.
What is this 'leftist' creature that you talk about? I have seen a dramatic increase in the use of the term in the last couple of years. No doubt its rise in popularity only coincidentally coincides with 2001/09/11. Unfortunately, it seems primarily to be used as a sort of epithet meant to discredit some or other class of opinions. The snobbish might call it an ad hominem attack--the rest of us recognize it as name-calling. His points aren't valid, because he's a leftist.
So, what is this 'leftist' creature? Dusting off my history books, the closest fit seems to be 'Commie'. A bearer of uncomfortable political ideas. Anathema in the halls of power. Dangerous to be seen with anywhere else. Might even get you put on a watch list...
Incidentally, I suspect that the shortcomings of watch lists as enumerated by those wicked 'leftist' sympathizers are valid concerns. Terrorists aren't the only ones with limited resources--if airlines and the government are wasting significant time and effort on a very large proportion of false positives, couldn't that time and effort be better spent elsewhere?
~Idarubicin
There's a simple, easy, inexpensive and effective solution. Unfortunately that's not what the government wants; they prefer expensive, complicated solutions that let them hire more people and expand the bureacracy, even if it happens that those solutions are ineffective. Anyhow, the simple, easy, inexpensive solution is:
The flight volume for all these countries is tiny. Israel has only a few thousand flights per year to only 10 airports. Their procedures work for their number of flights.
There are 30,000 flights each day in the US. We need something that scales.
It's bad enough when a single isolated program fails completely, as this one has. But if, as you suggest, this program is a part of a larger anti-terrorist system then the failure on the governments part is that much greater! As the saying goes "A chain is only as strong as its weakest link.". The failure of the "No-Fly" list to accurately idenify terrorists creates a critical weakness in our national security infrastructure. For you see, this system was designed to deny terrorists mobility and access to a weapon that they have used successfully in the recent past. In this the "No-Fly" list has failed completely.
This list reminds me of the blunder the French made in constructing the Maginot line. They covered most of their eastern border with a line of fortifications that are tough by even modern standards. But they left the Arden forest uncovered because they thought that no mechanized army could pass through the thickly wooded area. But when war broke out that's just what the Germans did, and the rest of the line was rendered irrelevant. In other words when a line or net or wall is breached then all the remaining sections are worthless.
"The moment "pride" is lost, "freedom" is also lost." - Ramza.
In the US, more people are killed in car accidents _every month_ than were killed in the attacks on the WTC. Even a tiny 2% decrease in the number of car-accident deaths would save more lives every decade than were lost in all terrorist attacks the US has ever suffered.
Over the last 10 years, an American's odds of dying in a terrorist attack are about 1 in 100,000. That's less than your odds of drowning in your own bathtub, less than your odds of drinking yourself to death, and less than your odds of accidentally suffocating in your own bed! (http://www.nsc.org/lrs/statinfo/odds.htm)
Frankly, the current atmosphere of fear of terrorism is little more than hysteria. Why on earth aren't we showing the world we have some balls and are strong enough to not let a few terrorists make us live in fear? If you live in fear or give up freedoms, you've let the terrorists win!
X-raying shoes doesn't make for effective security, but it's intrusive enough to give the impression that at least something is being done.
Articles and editorials that call attention to the violations that come with the bogus no-fly list are essential components of the system -- they make everybody else experience it, vicariously. Everybody who is a little bit stupid (i.e. most people) feels a little safer for it. Sure it inconveniences some people, but not enough to make much political difference.
Even better than the impression of intrusive security, it leads to demands for what amounts to a system of internal passports, where you can't travel by air without registering, and getting -- and maintaining --- official permission. "What, no internal passport? Sorry, sir, I can't let you board." At first felons will have their passports pulled, then "suspected terrorists", then political undesirables of all sorts.
They also conduct 10 minute interviews with every person who goes through customs. Haven't flown there myself, but a good friend of mine has described it in detail, having gone back and forth dozens of times over the last few years. The security is far more extensive than air marshalls and a few gadgets on the planes.
... I could go on.
Israel has a massive, complex security network, and you can't just extract a single entity, El Al, from it and pretend that none of the rest of the government's activities bear on how El Al works.
I've flown El Al several times in the past year, and I have been living in Jerusalem, and I can vouch for both: there's a lot more than a 10 minute interview to El Al if anything about you raises their hackles, *and* security in Israel is (has to be) a whole different animal than it is in the States. (I'm a U.S. citizen, a student living in Israel for a year.)
How is a 1 in 100 chance of having a few questions asked and your bags searched because of a false positive "completely destroying" any of your rights?
Here I have to take issue with you. "A few questions asked and your bags searched" is not what they are talking about in this article. I happen to trigger a couple of profiles that make airlines sit up and take notice (I'm a middle-aged single female, for starters) and as a result, flying is a nightmare. While I totally understand the need for careful security, if you are one of the lucky people on the list, you can look forward to the things that have actually happened to me in the past year:
-- kept off of one international flight, questioned for 4 hours, finally allowed on the next flight going out.
-- required to spend 3 hours clearing security, therefore missing my flight (while my laptop was disassembled so completely that tools were required, and all the parts x-rayed.)
-- when travelling with a friend, said friend was also taken off into a separate room, where he was questioned to see if his version of my story matched my version
-- Searches that require undressing.
-- Humiliating "interviews" in front of a line of people (who are also angry that the line is held up.)
My point is, we're not talking minor inconvenience, as you suggest. I have no desire to be blown up or crashed into a building, and I am quite clear that anything I do to protest this treatment makes matters worse, so I just grit my teeth and fly as little as I can.
Some airlines are more courteous than others; I've become a real fan of Continental, because they take their time checking me out but do it with a minimum of humiliation.
OK, now what?