What Are You Looking At?

Ensign Stinky writes "The NYTimes has a story, with some spooky-cool pictures, about software to extract exactly what image a person is seeing with their eyes, just from the reflection on their cornea. You can see even a wider image than the subject and tell what they're specifically focusing on. It's too bad the coolest tech is immediately subverted for evil. The possible applications listed include 'surveillance cameras that spot suspicious behavior.' Remind anyone of that scene in the movie 'Wild Wild West' where they extract the last thing the dead guy saw?"

Comments

Here we go again...

[jlgolson]

Why is surveillance cameras that spot suspicious behavior bad? It seems like it would be good, because the cameras will not be watching the vast majority of people walking by. Just the ones that are darting from person to person, or back and forth looking for cops.

Also, why didn't the poster mention "use in interfaces for quadriplegics who use their gaze to operate a computer". Sounds like that is a lot more interesting to the Slashdot crowd than surveillance cameras.

Sounds kinda nifty to me. As far as the surveillance part, they won't learn that much from me. Guys look at breasts a LOT. Wow. Newsflash.

Re:Here we go again...

[Anonymous Coward]

It's amazing how many people believe that a camera can pick out the people with suspicious behavior without looking at everybody.

The camera will record everybody. The person/computer program reviewing the recording might choose to keep only the recordings of 'suspicious' people-but I doubt it. Bureaucrats are CYA types-and it's much more CYA to keep *everything*.

Re:Here we go again...

[Lumpy]

It doesnt matter anyways. 99% of all surveillance cameras are extremely low end and have even less resolution than the televisions and VHS recorders that are viewing/recording them.

Only extremely high end professional Tv cameras have anywhere near the 700 lines of resolution that NTSC is capable of and most CCTV or surveillance cameras not only have much less than 2/3rd that resolution, but their optics, I.E. lens sucks horribly.

Nobody has a surveillance system with cameras that have $30,000.00US lenses on them and $50,000.00 cameras.

It's a neat idea, but you can not extract information from nothing. and at that low of a resolution that most all video equipment is at they will extract nothing from the blurry-blob that is the reflection in their suspect's eyes.

Unless they are standing within 18 inches of the camera... then I would syspect that the "criminal" would be a tiny bit suspicious.

dont get me wrong, it's neat but the journalist stretched the truth and extrapolated ideas that were way out in outerspace and 100% impossible without insanely expensive equipment.

Re:Here we go again...

[pz]

IAAVR (I am a visual researcher) who professionally studies eye position. We use a number of methods to do this, but one of the easiest and quickest way to measure a person's eye position is to arrange an off-the-shelf video camera with telephoto lens to point at the subject's eye. Plenty of software then exists to extract the iris position and therefore the position of the eye in the orbit, and therefore the point in space where the user is looking. Naturally, a more expensive whiz-bang camera will give you better data, but with a run-of-the-mill consumer grade camera you can do better than 1 degree of accuracy. This sort of thing is already done for quadraplegics.

How do you turn this into a high-resolution image of what the subject is looking at? You point a (better) camera in the opposite direction and either adjust it's position to match, or computationally select out the portion of the image where the subject is looking.

Now, that isn't exactly what these researchers did, but it would be a whole lot easier (and it's what we do on a daily basis).

And, for those who don't have a photography habit, many of the current-issue SLRs (Canons, specifically) already read your eye position with some nifty technology that uses reflections of IR LEDs off your cornea and focuses the camera where you're looking in the frame. (If you haven't used a camera which does this, try it; you'll never go back.)

The point? Technology to read eye position exists, and some of it is pretty old (eg, if you're willing to put a contact lens in your eye, then techniques from the 60s work fine). The ONLY interesting part these people did was to use the reflection off the front surface of the eye (which despite what another poster suggests is very high fidelity if captured with high-quality hardware) and applied the appropriate reflection model to undo the optical distortion of looking in the equivalent of a curved mirror. Think of it this way: if we all wore those mirrored sunglasses from the 70s, despite not having exact eye position information, just approximate gaze direction from head angle, we'd be able to tell more-or-less what each person was looking at.

Comment removed

[account_deleted]

Comment removed based on user account deletion

If you're gonna ...

[XP-Elwood]

quote only half of the sentence (and spread FUD by doind so) at least use the *whole sentence*. "Because the algorithms can track exactly where a person is looking, the system may one day find use in surveillance cameras that spot suspicious behavior or in interfaces for quadriplegics who use their gaze to operate a computer."

Re:The first thing I thought of....

[Scrab]

Don't they already have something like that built into a helmet... Like this?

Re:The first thing I thought of....

[GeekZilla]

Well, actually they already have that-sort of. The Apache attack helicopter uses a targeting system that aims based on what the pilot looks at. Except that it uses a monocle over the right eye of the pilot. The monocle displays targeting information and presents a cross-hairs to the pilot. The pilot merely puts the cross-hairs on his target by turning his head and "looking" at it with the monocle and then pressing the trigger for the appropriate weapon. However, it's not REALLY based on what his eyeball is focused on, it's what the cross-hairs are pointed at. He could point the monocle towards the horizon and without moving his head, he could rotate his eyeballs to look down and fire, but unless he moves his head, the guns/missiles will still fire at what the monocle is pointed/looking at. Here are just a few pages that a quick Google search turned up: How Apache Helicopters Work-Controls and Sensors or "PBS-Frontline or this page that talks about the M142 INTEGRATED HELMET AND DISPLAY SIGHT SYSTEM (IHADSS)specifically.

Re:Exactly

[BWJones]

There is no way you can tell what the person is mentally processing by virtue of the fact that a particular image happened to be reflected in their eye. All you can reasonably conclude is that they were facing in a particular direction.

You CAN however correlate what a person is looking at with a brain waveform called a P300. That waveform is essentially an evoked potential that signals recognition. It does not tell you anything else about that recognition, only that the person has seen the image or object or person before.

not tremendously impressive

[SpootFinallyRegister]

regardless of privacy/big-brother/thoughtcrime issues, this doesnt seem that impressive to me.

algorithms have existed for a number of years for facial recognition that keys on features of the face, most notably eyes. being able to find irises in a picture with faces has been done; and not even requiring a picture of just one face as the article seems to suggest.

from there, its extracting a transparent reflection off of a constant backing with multiple frames. again, previous work. nothing new.

yes, its a neat application, but this is no breakthrough. this article is like someone going out and taking a picture of something nobodys taken a picture of before, and then saying they invented a new camera.