Face Recognition - Real or Science Fiction?

An anonymous reader writes "Facial recognition software has been touted as one of the technologies that will change our future, particularly in law enforcement. How close are we to being recognized by a computer anywhere we go, as portrayed in movies like Minority Report? According to the industry's recent Public Relations releases, these products are closer than we think. The reality though, is that current products work only when utilizing a small comparative sample, and any attempts for an individual to disguise themselves typically throw off the results. To see how far this technology needs to go before becoming mainstream, one site utilized Government-tested face recognition software, available freely through MyHeritage.com, to compare hundreds of famous people, animals, and cartoons to a database of 2,000 celebrities. Some of the results showed promise for the technology, but most were just funny — for example, who would mistake Barbara Streisand for Shrek, or Lance Bass of N'Sync for a Teletubby?"

Comments

I've heard this for years

[2.7182]

After working in computer vision for 5 years I've realized that most problems aren't hard - they are not well defined. Mathematically face recognition is not a problem that can be stated.

Many other problems in CV are like this - edge detection, segmentation, etc. But people write hacks that work in restricted conditions and say they've solved.

And look, you could always just put on those Groucho Marx glasses.

Re:I've heard this for years

[PieSquared]

I think the real problem is what it looks at. The shape of your face is what it looks at. What if you put a little clay or really thick makeup around your jaw and cheek bones to change your visible facial structure... and of course facial hair can be shaped to look like pretty much anything is under it without even adding anything artificial to your face. And of course, you'll need multiple frames of reference with a wide angle between them to get any useful information anyway... you can't really judge depth from a single frame and if you try a little eyeshadow will throw it off. I can only see facial recognition as proving that you aren't someone smaller then you are, not that you are a specific person. And of course you could always get one of those masks from mission impossible! Yea, that's what I thought!

Re:I've heard this for years

[bzipitidoo]

I also worked on a project to compare images. The idea was general purpose, not just for faces only. Unfortunately, it didn't really work. I took the woefully slow prototype and rewrote it so it worked correctly, and was far faster. And all that did was help destroy their illusions and delusions that it was going to work. Before, they could be optimistic because it wasn't fast enough to do hundreds of tests, and so they were able to point to extremely small sets of data upon which it had apparently mostly worked. Progress of a sort. Meanwhile, potential sources of more funding were not interested in the general purpose. Some wanted facial recognition for security uses, and another group wanted pr0n detection for censorship uses.

I don't know why facial recognition is so sexy and hot right now. Quite a few ventures claim to have something that works, but when you dig into the few details they provide, you find a lot of vapor. Some sort of do work, but they need lots of help. The faces have to be positioned, scaled, and oriented the same. Some employ methods for handling that automatically, and some lean on people to do that. The lighting has to be just so. Pretty much every method must be trained. And if those difficulties aren't enough, the methods have to be fast. Of course we'd like them to be always right, but even after all the help, a method is doing awesome if it's right 90% of the time.

Currently, people are far, far better at this problem, and we're still fooled quite often, otherwise no one would bother with disguises. But some expect computers to do better. Check a photo of a person against a database of a _million_ photos to see if there's a match? Could people do that? I doubt it. Facial recognition by computer is fantasy, and probably will remain fantasy for many years yet.

"Do you see anything there?"

I looked at the broad plumed hat, the curling love-locks, the white lace collar, and the straight, severe face which was framed between them. It was not a brutal countenance, but it was prim hard, and stern, with a firm-set, thin-lipped mouth, and a coldly intolerant eye.

"Is it like anyone you know?"

"There is something of Sir Henry about the jaw."

"Just a suggestion, perhaps. But wait an instant!" He stood upon a chair, and, holding up the light in his left hand, he curved his right arm over the broad hat and round the long ringlets.

"Good heavens!" I cried in amazement.

The face of Stapleton had sprung out of the canvas.

"Ha, you see it now. My eyes have been trained to examine faces and not their trimmings. It is the first quality of a criminal investigator that he should see through a disguise."

"But this is marvellous. It might be his portrait."

"Yes, it is an interesting instance of a throwback, which appears to be both physical and spiritual. A study of family portraits is enough to convert a man to the doctrine of reincarnation. The fellow is a Baskerville--that is evident."

recognized

[AcidLacedPenguiN]

This is all well and good, but the minute I get falsely identfied as a criminal just for being in the bar district late at night in the wrong place/wrong time I won't be too happy. . .

Legal hoops

[solevita]

I'm wondering about the legality of all this, especially in a criminal justice system. My DNA, for example, can't be used in court as evidence unless certain hoops have been jumped through; the prosecutor needs a reason to obtain a DNA sample and then procedures must be followed.

I wonder if the same systems will apply to a computer analysed image of my face; will there be a criterea for when this image is admissable in court? Will I have rights concerning my image? Or are we just going towards a 1984 style system. Interesting because this hasn't been the result of DNA admissions to court, despite the seemingly more robust nature of this evidence.

Belittling technology

[MobyDisk]

Some of the results showed promise for the technology, but most were just funny -- for example, who would mistake Barbara Streisand for Shrek, or Lance Bass of N'Sync for a Teletubby?"

That's just trolling. The software was instructed to find the celebrity who most closely matched a cartoon character. It didn't mistake anyone for a cartoon character. And since cartoon characters are not within the scope of what the software is for, it shows that it worked better than expected. Attempts like this to belittle the success of the technology are akin to Ad Hominem attacks, and have no merit in a discussion.

re: hacks and restricted conditions

[King_TJ]

But don't we almost always get a computer to solve a problem that's not strictly a mathematical one using "hacks that only work in restricted conditions"?

Our spell-checkers in our word processors don't actually know anything about the rules of a language, phonics, etc. They just do lookups from a dictionary. If a word's not listed, it has no idea if it's spelled properly or not -- even if the misspelling is one that's simply not a possible correct sequence of letters for the language. Most don't even realize if a word is misspelled in the context of the sentence, as long as it matches a correct spelling in the word list.

Until we figure out how the human brain recognizes faces as individuals, we can't expect anything *but* a clever hack for a computer to do the same. And truthfully, I suspect the human brain takes many things into account to do a "recognition" on a person. How often do you see somebody in the store that you're pretty sure you know from a previous job, school, etc. but you're not quite sure? I've had this happen a few times, and to make a better determination, I had to take other factors into account, like the sound of their voice if I heard them speak, the way they walked, or maybe an expression that came across their face. Humans "key in" on specific things that help them remember a person. And depending on which "features" they chose, they may or may not be effective. (Say you remember a gal really well because of her long, flowing hair? If she cuts it real short, there's a good chance you won't recognize her at all anymore if she walks by you.)

Re:GA guided NN's

[MBCook]

The problem is the inputs. Do you inputs sets of geometry (eyes are X" apart, at an angle of 0.53 degrees, chin is .5" below lips, blah blah blah), the raw image, or something else? If you use the raw image, you'd need a system in the front end scale/rotate the images to be in about the same place otherwise you probably have no chance (unless you want your neural net to do that TOO, which would make training harder and take longer).

Even if you use geometry (we have a vague understanding of what makes people look similar or beautiful) you'll still run into problem. You have problems of perspective (not all pictures are taken straight on).

Garbage in, garbage out. The best solution is to provide tons of information and let the neural net sort out what matters and what doesn't (they are quite good at that) but that will require more training which means more time.

So in the end you may build a good system. But to use it you must provide it with geometry of a face that someone picks out after fixing the perspective on a photo. Or it works much like our brains and accounts for all that, but it will take you 6 years of non-stop training alone.

And what is a success? Two people who look similar? A perfect match? What if your software rates a picture of a celebrity impersonator (looking like the celebrity) over a picture of that celebrity looking different (movie role, disheveled mugshot, etc)? Is that a success?

And how do you rate the people for the training input? Sure a neural net can figure out the way to something where we know the end, but what about when we don't quite know the end?

It probably took evolution a VERY long time to get good at recognizing individuals. And even then, we are not that great (mistaken identity, all cocker spaniels look alike until you spend more time with them, etc).

It's a neat problem, but it is seriously tough even with the "voodoo magic" that a neural net would provide over trying to come up with a straight formula.

Initially, cameras would be spaced like eyes

[benhocking]

If I was training to match V1-4, I'd have the input come from two "eyes" with inputs similar to what our eyes actually provide to our brain. We know quite a bit about visual cortex, but there's a lot we don't know. Initially, I'd train it using a batch of photographs for a single person (we'll call her "Momma") and then I'd train with a few others (where a match is a match only if it's the same person). From there, I'd create histograms of parameter settings that seem to do an adequate job on this small set, and then use this reduced parameter space to create populations that are evaluated after training on millions of photographs. (The photographs can be placed in front of the eyes - once for each photograph, mind you, and not for each "individual" being tested - just like we can recognize photos and not just people.)

I could imagine narrowing the parameter space down to 100 or so unknown parameters, and each training session might take several hours. Given enough resources (e.g., the Pittsburgh Supercomputer Center), I'd run population sizes of 500 or so (in parallel), so that you could possibly go through 4-5 generations per day. In a month, you might have some pretty good individuals. Of course, my research area is the hippocampus and not the visual cortex, so it might take significantly more than 100 parameters to even begin to set this up.

Now, someone else pointed out that such computers would not have the biases that we humans have, but that's not necessarily true. If you train the computer using an input set of 950,000 "white" people and 50,000 "black" people, it would tend to make the mistake of thinking that "black" people look a lot like each other. (Studies done with speech recognition have shown that neural networks trained on Japanese have a much harder time telling "l" from "r" than those trained on English.)