What Are You Looking At?

Ensign Stinky writes "The NYTimes has a story, with some spooky-cool pictures, about software to extract exactly what image a person is seeing with their eyes, just from the reflection on their cornea. You can see even a wider image than the subject and tell what they're specifically focusing on. It's too bad the coolest tech is immediately subverted for evil. The possible applications listed include 'surveillance cameras that spot suspicious behavior.' Remind anyone of that scene in the movie 'Wild Wild West' where they extract the last thing the dead guy saw?"

Comments

Thoughtcrime

[BWJones]

It's too bad the coolest tech is immediately subverted for evil. The possible applications listed include 'surveillance cameras that spot suspicious behavior.'

Hey guys, like much of the popular sci-fi literature will illustrate, its not what you might be looking at or visually or cognitvely attending to or even thinking.......its what you actively do with those thoughts or attentions. Prosecuting folks for visual attention to things that stand out (like items folks covet such as that rather nice looking Porsche below and outside my window) will be fruitless. Same goes for prosecuting "thoughtcrimes". However, cheating on exams.......could be more easily documented I suppose.....

Re:Thoughtcrime

[Anonymous Coward]

In the US at least we do prosecute throught crimes under the guise of "hate crime".

If I shot you because I didn't like your race the punishment is more severe than if I shot you just because I thought it would be fun.

Re:Thoughtcrime

[GTRacer]

Well, I once got a parking ticket for "intent to park" in an unauthorised space. I pulled into a parking garage (dedicated to customers) to ask for directions to an appropriate employee lot as my assigned one was full.

I got the directions and was ticketed for parking in the customer garage. Mind you, I wasn't IN the garage yet (it has a long driveway leading to it), and I never exited my car. In fact, the first thing I did when I saw the guard was to ask for the directions.

He gave me the directions, a ticket, and turned me around. His rationale? He knows how employees like to take advantage...

GTRacer
- Find the umbrella.

Here we go again...

[jlgolson]

Why is surveillance cameras that spot suspicious behavior bad? It seems like it would be good, because the cameras will not be watching the vast majority of people walking by. Just the ones that are darting from person to person, or back and forth looking for cops.

Also, why didn't the poster mention "use in interfaces for quadriplegics who use their gaze to operate a computer". Sounds like that is a lot more interesting to the Slashdot crowd than surveillance cameras.

Sounds kinda nifty to me. As far as the surveillance part, they won't learn that much from me. Guys look at breasts a LOT. Wow. Newsflash.

Re:Here we go again...

[Short Circuit]

So people like me, who are inherently paranoid, are at higher risk?

Great...I knew this would happen. :)

Re:Here we go again...

[tsg]

Just the ones that are darting from person to person, or back and forth looking for cops.

Or scanning the crowd looking for someone they're meeting. What, exactly, about "darting eyes" indicates criminal or suspicious behavior?

Sounds kinda nifty to me. As far as the surveillance part, they won't learn that much from me. Guys look at breasts a LOT. Wow. Newsflash.

They won't just know that guys look at breasts a lot. They will know whose breasts you were looking at. Big difference.

Re:Here we go again...

[Lumpy]

It doesnt matter anyways. 99% of all surveillance cameras are extremely low end and have even less resolution than the televisions and VHS recorders that are viewing/recording them.

Only extremely high end professional Tv cameras have anywhere near the 700 lines of resolution that NTSC is capable of and most CCTV or surveillance cameras not only have much less than 2/3rd that resolution, but their optics, I.E. lens sucks horribly.

Nobody has a surveillance system with cameras that have $30,000.00US lenses on them and $50,000.00 cameras.

It's a neat idea, but you can not extract information from nothing. and at that low of a resolution that most all video equipment is at they will extract nothing from the blurry-blob that is the reflection in their suspect's eyes.

Unless they are standing within 18 inches of the camera... then I would syspect that the "criminal" would be a tiny bit suspicious.

dont get me wrong, it's neat but the journalist stretched the truth and extrapolated ideas that were way out in outerspace and 100% impossible without insanely expensive equipment.

Re:Here we go again...

[Minwee]

"Our security cameras showed that you spent twenty six minutes and eighteen seconds staring directly at Ms. Jones' chest in the last month alone. I'm afraid we're going to have to let you go before she files a sexual harassment complaint with the board. Have a nice day."

Re:Here we go again...

[pz]

IAAVR (I am a visual researcher) who professionally studies eye position. We use a number of methods to do this, but one of the easiest and quickest way to measure a person's eye position is to arrange an off-the-shelf video camera with telephoto lens to point at the subject's eye. Plenty of software then exists to extract the iris position and therefore the position of the eye in the orbit, and therefore the point in space where the user is looking. Naturally, a more expensive whiz-bang camera will give you better data, but with a run-of-the-mill consumer grade camera you can do better than 1 degree of accuracy. This sort of thing is already done for quadraplegics.

How do you turn this into a high-resolution image of what the subject is looking at? You point a (better) camera in the opposite direction and either adjust it's position to match, or computationally select out the portion of the image where the subject is looking.

Now, that isn't exactly what these researchers did, but it would be a whole lot easier (and it's what we do on a daily basis).

And, for those who don't have a photography habit, many of the current-issue SLRs (Canons, specifically) already read your eye position with some nifty technology that uses reflections of IR LEDs off your cornea and focuses the camera where you're looking in the frame. (If you haven't used a camera which does this, try it; you'll never go back.)

The point? Technology to read eye position exists, and some of it is pretty old (eg, if you're willing to put a contact lens in your eye, then techniques from the 60s work fine). The ONLY interesting part these people did was to use the reflection off the front surface of the eye (which despite what another poster suggests is very high fidelity if captured with high-quality hardware) and applied the appropriate reflection model to undo the optical distortion of looking in the equivalent of a curved mirror. Think of it this way: if we all wore those mirrored sunglasses from the 70s, despite not having exact eye position information, just approximate gaze direction from head angle, we'd be able to tell more-or-less what each person was looking at.

Comment removed

[account_deleted]

Comment removed based on user account deletion

Okay...

[Agent Green]

So I need to wear my tinfoil hat AND dark sunglasses!

Forget the government...

[Trigun]

This is really going to get me in shit with the wife.

as long as the ladies...

[Digitus1337]

...don't get any real-time version of this, i'm in the clear.

Sexual Harrasment claims up by 500%

[Anonymous Coward]

BEEP!

Female worker: Stop looking at my breasts!

Male worker: I wasn't!

BEEP!

Female worker: Argh! You did it again!

BEEP!

Re:Sexual Harrasment claims up by 500%

[grub]

I keep the sexual harrasment forms in the bottom drawer of my desk. That way when a woman goes to get one I can check out her ass.
[rimshot]

Old technology

[FerretFrottage]

Women have been able to detect what men are looking at for centuries.

(.)(.) ---> Hey you, read the comment above first

Can help spot fakes

[EnnTeeDee]

Very cool! Seems like this might be used to help spot Photoshop modifications -- for example, in a group picture, just compare the reflections in each person's eyes.

I Spy

[barcodez]

I'm going to get this and become I-Spy Champion of the world! Mu ha ha ha ha (etc).

Hmmm

[jdtanner]

I can imagine it now...

Spy1: What is he looking at?
Spy2: Hang on...it's still processing...
Spy1: Well?
Spy2: He's looking at two guys wearing shades and dark coats operating a massive camera and computer!
Spy1: Doh!

John

Comment removed

[account_deleted]

Comment removed based on user account deletion

Looking through animal eyes

[base_chakra]

Although the author of the article declares that "the system can automatically recover wide-angle views of what people are looking at" (emphasis mine), to me one of the most exciting potential applications is to further human understanding of what animals choose to look at.

With our current knowledge of ocular biology we can make some assertions about what color ranges different species can see, but being able to study more precisely what they choose to focus on and what conditions attract their attention would advance our understanding of other species tremendously.

Resolution

[wowbagger]

I find the statements about "we can go back to old pictures of JFK and see what he was looking at" to be questionable at best.

You need a LOT of pixels of the eye itself from which to reconstruct an image. Now, look at how much of a given normal picture the eyes of a person represent.

You *might* be able to reconstruct where the person is looking. You probably aren't going to have enough pixels to reconstruct what they saw.

To do that level of imaging you are going to need a picture of the person's eye at high resolution.

So the government spy cameras will have to zoom in on your eyes - call it about a 500 to one zoom. They will have to track your eyes as you move about.

And yes, if you wear sunglasses you can defeat this.

Now, what this WOULD be very useful for would be in combinatino with a head mounted display - since the display device has to subtend a large angle as viewed from the eye, the display device must have a good view of the eye. So combining the display device with an imaging device would allow the system to see what you at what you are looking, so you now have a pointing device. Theoretically, a wink or slow-blink could be a "select" operation.

Now, if they could get the focus point of the eye, they could REALLY make an interesting system - if you are focusing past the image, they could mute it - reduce the brightness, possibly even reduce the amount of information (iconify apps, reduce update rates, show only "critical" items, etc.) When they detect you've shifted focus to bring the display into focus, brighten up. Think of looking through a dirty windshield, then shifting focus to the dirt on the glass.

Re:Exactly

[symbolic]

There is no way you can tell what the person is mentally processing by virtue of the fact that a particular image happened to be reflected in their eye. All you can reasonably conclude is that they were facing in a particular direction. What if, for example, someone was merely staring into space, with their thoughts wandering between and betwixt something completely unrelated? Isn't that what we call daydreaming? What rational conclusion could you you possibly draw in a situation like this, and how could you refute someone's claim to the contrary?

Re:The first thing I thought of....

[GeekZilla]

Well, actually they already have that-sort of. The Apache attack helicopter uses a targeting system that aims based on what the pilot looks at. Except that it uses a monocle over the right eye of the pilot. The monocle displays targeting information and presents a cross-hairs to the pilot. The pilot merely puts the cross-hairs on his target by turning his head and "looking" at it with the monocle and then pressing the trigger for the appropriate weapon. However, it's not REALLY based on what his eyeball is focused on, it's what the cross-hairs are pointed at. He could point the monocle towards the horizon and without moving his head, he could rotate his eyeballs to look down and fire, but unless he moves his head, the guns/missiles will still fire at what the monocle is pointed/looking at. Here are just a few pages that a quick Google search turned up: How Apache Helicopters Work-Controls and Sensors or "PBS-Frontline or this page that talks about the M142 INTEGRATED HELMET AND DISPLAY SIGHT SYSTEM (IHADSS)specifically.

Mouse replacement?

[DrCode]

Would be cool if this could work with a computer. Instead of "focus-follows-mouse", I'd like to have "focus-follows-eyes". Lots of times, I'll look at a window and start to type in it, then realize that I hadn't moved the mouse over it to get focus.

Inherently flawed

[fyngyrz]

When I started in martial arts (decades ago - I teach now) I had pretty normal vision. Look right at something, see that something, everything else is pretty much tuned out. I could see some motion at the periphery, but that was really all.

I was trained to use my peripheral vision - exercises like counting fingers further and further out from the target you're looking at progressively increase your ability first to discriminate detail that you usually don't process, and progressively widen the field of view so that you take in more at a glance.

In martial arts sparring, it is very useful to see something coming, essentially, to see it early. There is plenty of reinforcement, both positive and negative, in that environment. Learning this well pays numerous dividends in the arts. It is an interesting general ability as well.

At this point in my life, I can "look" right at you in the sense that a centered axis out of my pupil draws a line to one of your eyes. At the same time, I can actively study something I can see very clearly that is considerably off that axis, behind you, somewhat off to your side, and way out of the same focus plane your face is in. You won't know, and gear like this wouldn't know either. I'm "looking right at you" as far as any observer is concerned.

I learned to do this - I certainly couldn't do it at all before actively training to do it. I teach my students to do it. The initial level of ability varies from person to person, but I've yet to encounter anyone who couldn't improve markedly over six months or so of daily exercises. I suspect that if the technology being discussed here comes into any kind of use where it is actually a social/legal issue, others will learn it just as well. You could probably detect the focal plane being different (the eye's physical configuration after all does change based on the focal plane) but this whole center of attention thing is absolutely defeatable.

I have high confidence that until or unless you can actually read minds and determine cognitive intent, this kind of technology will be very limited in application and reliability. We should ask, who will be motivated to learn to defeat such a mechanism by it becoming a law enforcement tool? It seems to me that the most obvious answer is those who have some kind of subversive orientation. Criminals, to put it more bluntly.

Action, reaction.