Slashdot Mirror
Minority Report UI For The Military
merryprankster writes "New Scientist is reporting that a 'Minority Report'
style interface is being developed by defense company Raytheon. Users don a
pair of reflective gloves and manipulate images projected on a panoramic
screen. A mounted camera keeps track of hand movements and a computer
interprets gestures. Raytheon has even
employed John Underkoffler, the researcher who
proposed the interface to the makers of the film. Now just wait till Billboards start scanning your iris."
Finally, a good explanation for the data-gloves Reeves used in the movie.
Shh.
The system under development at Raytheon lets users don a pair of reflective gloves and manipulate images projected on a panoramic screen. A mounted camera keeps track of hand movements and a computer interprets gestures
"Hand gestures, unlike a mouse or pointer, work really well when data is represented on wall-sized displays, for example."
And where in the field will this be used?
Am I the only one who gets scared when I imagine what a room in the pentagon might look like, with Generals wearing special glasses, and moving projected data off walls?
Rosco: "If brains were gunpowder, Enos couldn't blow his nose."
In tech, we often find ourselves referring to the Hollywood Operating System. You know, the one where every key press makes a "click" sound, and passwords are cracked one character at a time (admittedly, something that actually worked against Windows 9x file shares).
I was actually impressed with the UI in Minority Report. I'm not saying it was necessarily perfect, but it wasn't obviously ridiculous either. There is a need to monitor information flows across many different sources, to simultaneously sense them, and to have the ability to integrate on demand. A large display with linkable data nodes is one approach that deserves further analysis.
I see this as being pretty exhausting after prolonged use. Perhaps if minute hand movements were translated into large gestures on the big screen... but that's what a conventional mouse does. I think the most revolutionary part would be to make "drag-drop" thing a lot more physical, i.e. add small amounts of inertia to dragged objects. Also, Google for the copy-paste pen device - really nifty stuff.
http://zero-to-enterprise.blogspot.com/
Wouldn't it be easier to just put 5 little transmitters on the tips of the fingers and triangulate all fingers. It'd be much more accurate and have much less noise in the data. Some may argue that itd be too battery intensive, but there could be two solutions. A) The transmitters would be very low power because the distance needed wouldn't be very much and no intense processing would be needed on their part, they just have to send out some unique signal with time data. I know bluetooth devices now (PDAs and CellPhones) use bluetooth for hours without recharging in addition to full color screens and running a processor. This would just need the transmitter so a battery could last a very long time in addition to the distance being very short. B) Just use some kind of highly sensitive rfid technology so the gloves are "self powering" (not sure if the tech is at that stage yet).
Regards,
Steve
...but I believe that in Minority Report they were using OLEDs ie no projector. Wake me up when they have OLEDs in mass production, that is a lot more interesting.
Push/Pull
...So many more distinct gestures/commands are possilbe.
Slide/Spin/Twist
Grab/Grip/Grok/Associate
Wipe/Toss
I read a lot of Phillip K Dick and the interface portrayed in Minority Report was wonderfull.... not the goop-pool..... I'm refering to the the big screen Tom Cruise manipulated.... the goop-pool interface is the opposite extreme.
Nice story...original author highly recommended.
it makes me wonder. Which side are you on?
Hoppy Harrington says "Hi"gher
I wonder how accurate this would be. Would it even be really useful for first person shooters that require pinpoint precision? I would say no, but then again, I'm somehow fairly accurate with a mouse, so my hands can be accurate with training. I'm not sure about the whole arm bit though.
Canadian Cynic, canadian politics is less boring than you
Have a look at HandVu for something that works right now.
I was planning on writing something similar to this (actually, very similar, same libraries and everything) but now may just build on top of the HandVu libraries instead.
I'd mod you up insightful but I've already commented here.
That was exactly my first thought -- waving your arms around is bloody tiring. Heck, I like to have my mouse tracking set so I can pretty much move the cursor from one side of the screen to the other by just flexing my fingers, the heel of my palm pretty much rests in one spot (and in a different spot when using the keyboard).
-- Alastair
In each of your examples, the muscles involved get a chance at rest.
Holding your arms out continually puts some muscles under constant tension. No muscle in the human body was designed for that kind of punishment; Even the heart muscles, composed of the only muscle tissue that doesn't get tired, get rest periods every beat.
tasks(723) drafts(105) languages(484) examples(29106)
Tovi Grossman, a grad student at UToronto, won best paper at UIST for developing a gestural, camera-based UI for actually building CAD models in a volumetric display from Actuality Systems. Click on the "video" for "Multi-finger gestural interaction with 3D volumetric displays."
http://www.dgp.toronto.edu/~tovi/
For one thing, an orchestra conductor's hands are *frequently* at his sides for periods throughout the piece.
For another, I don't know of many people conducing 8 hour symphonys 5 nights a week.
I seem to recall that during scenes in the film where the computer interface was being used the operators were physically passing back and forth what looked like large acrylic disks and inserting them into the drives. It was a strange dichotomy to see an apparently sophisticated interface coupled with such an archaic manner of data transport (i.e. sneaker net). It was as if the idea of local area network had not occurred to them (or perhaps not to the producers or set designers anyway).
Try an experiment: fix your wrist, then position your fingertips in any configuration. Now, try moving any of your hand or finger bones while keeping the fingertips in the exact same position and orientation. It's very difficult, and you probably can't move those other bones too far.
Besides, it would be very rare indeed if the orientation of the second and third knuckles independent of the fingertips would be relevant to the task being performed. And even if that were true, trying that experiment shows me the interpolated position probably wouldn't vary by more than a few degrees at most.
Getting back to data gloves in general, I always thought the real data glove from many years ago was a clever piece of engineering. The one I remember seeing had an IR transceiver mounted on one side of each joint, and the IR beam was interrupted by variable width "vanes" affixed to the other side of each joint. The degree of flex was correlated to the amount of IR that was passed through the isolator. No moving or rotating parts to wear out (except for the glove,) and no friction for the user.
John
At Pacific Northwest National Laboratory, we are implementing something we call the HI-Space table, which uses a camera to track hand motions as well. Ours doesn't need special gloves, though. You can walk up to the table and move your hands around and it watches any number of hands, doing any number of poses. It detects objects that are placed in the space and recognizes them if they are in the database. We have voice recognition, too, so it can respond to spoken commands.
One of the best things about our system is that it is completely untethered and intuitive. There is no training period, and no device to put on. You are interacting with the digital world by manipulating in the physical world.
I write applications for the table. There are a lot of issues that come up that you wouldn't normally think about. For example, with many hands in the space, it's easy to have people doing conflicting things. Actions are not so clearly defined, either. For example, when selecting a button, do you point to it? For how long? What if your finger moves a little?
We are currently conducting user studies to see in what ways the HI-Space table is better than the desktop and cave environments, and we're looking for other applications and organizations interested in using this technology.
http://www.hitl.washington.edu/projects/hispace//
http://www.pnl.gov/infoviz/hces//
contact me at bob [dot] baddeley [at] pnl [dot] gov