Apple's All-Seeing Screen

Based on a recent patent we may be seeing a new kind of display coming from the Apple store in the near future, one that can capture images as well as display them. From the article: "The clever idea is to insert thousands of microscopic image sensors in-between the liquid crystal display cells in the screen. Each sensor captures its own small image, but software stitches these together to create a single, larger picture."

Comments

Obligatory: Facecrime

[RobertB-DC]

"It was terribly dangerous to let your thoughts wander when you were in any public place or within range of a telescreen. The smallest thing could give you away. A nervous tic, an unconscious look of anxiety, a habit of muttering to yourself -- anything that carried with it the suggestion of abnormality, of having something to hide. In any case, to wear an improper expression on your face (to look incredulous when a victory was announced, for example) was itself a punishable offence. There was even a word for it in Newspeak: facecrime, it was called."

Found it here: http://www.newspeakdictionary.com/ns-dict.html

Touch screen, not camera!

[isaac]

Think touch-screen here, not camera. Regular touch screens typically register only a single point at a time. There are alternatives that use frustrated total internal reflection, but currently these require rear projection - not feasible for a tablet. See http://mrl.nyu.edu/~jhan/ftirtouch/ if you haven't already.

Incorporating sensing elements within the display will permit sensing multiple simultaneous points of contact of arbitrary size/shape in a tablet form-factor. Neat!

Apple's been patenting lots of touch-interface concepts recently, too. Vide.

This patent is probably more about touch-screens than screen as scanner (that'd be a neat trick too, but probably would require too much resolution) or camera (would require a different but perfectly calibrated refractive element at each sensor - probably impractical).

-Isaac

When does a camscreen become mandatory?

[Catbeller]

When does a camscreen become mandatory?

I'm not kidding here. After all, if I'd told you ten years ago that by 2005, all cell phones would have a mandatory GPS tracker broadcasting your location to the phone company as you move about, with a nominal abilty to be switched off (ha), would you have believed me?

I see no outrage over Homeland Security, your phone company, Scientology, and any random corporation with a legal staff being capable of tracking your movements for the rest of your lives. Where is the outrage?

I see no problem with camscreens becoming mandatory in the next 15 years. Even the techiest of the techies have no problem with the tracking devices in their phones, cameras on the streets, and eventually mandatory trackers in our cars, so letting Mr. X watch you as you all watch your computer screens is not a biggie. I can see an infinite number of excuses to make it required by law. Hell, even the emergency health care bit that they used for the cell phones could be re-rigged for this one.

And the generation of kids coming up through school have been seen drug tests, dog searches, RFID trackers, and lie detectors. They've been told they have no rights as minors, and I doubt they'll be any more rebellious as adults. They're also convinced they are surrounded by enemies wanting the kill them in their schoolbuses and office buildings, so the fear excuse is a big Go.

Such a neat device, a camscreen. Here's what I'd like: separate power circuits for the screen and the camera element array. So I *know* that the thing cannot operate without my permission. But I wanted that for my cell phone's tracking device, and so far the phone salesmen look at me like I'm bin Laden or a specially-abled adult who left his house without his nurse. (big thought: look overseas for a phone capable of giving me the option of being untracked, import the damned thing. Maybe I am a little slow).

Enough w/ the creepy stalker stuff, and "on" LEDs

[maggard]

First for all of those posting "Heeeey, way to spy on chicks!": You're why many women dislike /. You're not funny; you're sad, creepy, and need to get a life.

I'll also point out a relative of mine had this happen to her. She's a pretty, vivacious, young woman, married, was then working in a public relations firm. The IT fellow was always a little too attentive for her comfort, to the degree she actively avoided calling him for issues.

Eventually she needed her speakers for a project, but rather then call in creepy IT guy she asked office clever guy to take a look, it was probably just a loose wire or something. That was indeed the issue, however he also discovered an additional cable, running to a camera, mounted under her desk staring into her crotch, feeding into a nearby cabinet with a VCR.

Much hullaballoo ensued, everyone in the building heard of it within a few minutes, much to the ire of the police. There were fingerprints, and all of the fellas in the office but for creepy IT guy offered theirs for comparison. none of the supplied prints matched, IT guy quit, relative had her desk replaced with a table.

That's who you sound like when you post stuff like that.

The good news is Steve Jobs has been here before. I remember NeXT bringing around one of their boxes to demo at my local http://www.acm.org/">ACM chapter. It came with a nifty built-in microphone, to which someone immediately noted "great for spying!" The NeXT rep gave a smile and pointed to the red LED next to the microphone, hardwired to light up whenever the microphone was active.

This practice continues to this day at Apple, putting in hardwired signal LEDs to indicate when a camera is active. My expectation is that this will continue. Indeed I wouldn't be surprised if Apple were to even include a camera-active screen mode to brighten it for a better picture when the camera is active, possibly swapping in a white background.

This has lots of applications

[dmoen]

The patent application mentions a number of applications: (1) video conferencing, (2) using the screen to replace the camera in multi-function portable devices like PDAs and mobile phones, (3) medical probes that must capture an image and supply their own illumination.

Slashdot user Isaac mentions the idea of using this for a touch sensitive display. I couldn't find this mentioned in the patent application, so the race is on to file a follow-on patent!

But you wouldn't actually have to touch the screen. Years ago, MIT built a user interface called "put that there" that did gaze tracking and voice recognition, so that the "mouse pointer" was pointing at whatever object you happened to be looking at on the display. No need to touch a mouse, you just use your gaze. That might be possible with this technology. It could also be used to interpret hand gestures and facial expressions, and use them as input.

I personally think it would be cool to build a software-programmable mirror. Think of a bathroom mirror with zoom functionality, image enhancement functions, etc. The extra functions are activated by hand gestures, and face recognition is used to determine the centre of zoom (because in a bathroom, you normally want to zoom in on your face).

Doug Moen