Slashdot Mirror

← Back to Stories (view on slashdot.org)

See-Through 3D Computer With Gesture Controls Gives Us a Glimpse of the Future

Posted by Soulskill on from the like-a-pre-cog-you-might-say dept.

silentbrad writes with this excerpt from Boy Genius Report: "Some believe a future full of massive, gesture-controlled computer displays like the ones seen in Twentieth Century Fox's Minority Report are an inevitability, and a prototype PC designed by an intern with the Microsoft Applied Sciences Group may be among the first steps in making that future a reality. Massachusetts Institute of Technology Ph.D student and MIT Media Lab research assistant Jinha Lee recently set out to change the way we interact with desktop computers. While progress has been made with 3D display technology, 3D has not yet proliferated in the personal computing space and Lee wants to change that. The end result of his work is a fascinating desktop computer with a transparent 3D display and a unique gesture-based interface that could change the way we use computers."

63 comments

  1. Errors by DeadDecoy · · Score: 4, Funny

    Now we can have the blue cube of death!

    1. Re:Errors by rainmouse · · Score: 1

      Now we can have the blue cube of death!

      Yes to go with some very terrible repetitive strain injuries.

    2. Re:Errors by flirno · · Score: 1

      Translucent cube of death.

  2. Half cool by Ginger_Chris · · Score: 5, Interesting

    Transparent screens still make me all fuzzy inside, but gesture based UI just doesn't interest me in the slightest. I want a UI where I have to move as LITTLE as possible. There was nothing on that demo that wouldn't have been easier with a mouse and a couple of mouse buttons more accurately, quicker and with less movement.

    1. Re:Half cool by slowLearner · · Score: 2

      True, but a few points.
      1. how long did it take to get the Desktop right? If you think about how long we didn't even have mice to interact with the computer and you don't have to move hardly at all if you don't use one of those.
      2.We need to start some place with this and 3D displays and how to interact with them have been around for a while but they will only be improved on when try to figure out how we work with them. This may not be as quick or intuitive as a mouse, but if people don't do work like this then we will be stuck with mice, track-pads and touch screens and the like. for ever.
      3.If it could be linked to glasses/contact lenses instead of a screen then I think it would make a lot more sense.

    2. Re:Half cool by hairyfeet · · Score: 4, Funny

      Glad i'm not the only one that doesn't want to play Dance Dance Revolution just to check his email.

      --
      ACs don't waste your time replying, your posts are never seen by me.
    3. Re:Half cool by Anonymous Coward · · Score: 1

      3.If it could be linked to glasses/contact lenses instead of a screen then I think it would make a lot more sense.

      Just imagine all those people walking on streets, talking to themselves and frantically flapping their hands and making straange gestures and grimaces.

    4. Re:Half cool by Anonymous Coward · · Score: 1

      Just don't do those strange gestures in Los Angeles because you may gesture the wrong gang and get killed.

    5. Re:Half cool by slowLearner · · Score: 2

      A bit like the first time I was present when somebody using hands free headset for their mobile phone, I answered 3 of his questions before I realised he was't talking to me. I am pretty sure he thought I was deranged as he left fairly quickly.

    6. Re:Half cool by martin-boundary · · Score: 2

      1. how long did it take to get the Desktop right? If you think about how long we didn't even have mice to interact with the computer and you don't have to move hardly at all if you don't use one of those.

      *Is* the Desktop right? If you ask me, it's not that great. For example, controlling windows by a couple of icons on the title bar seems very wrong.

      Suppose you have a piece of paper on your desk, and you'd like to move it aside. Do you pick it up always on the top side and handle it that way, or do you pick it up any old way you like? We should be able to drag windows by grabbing them anywhere in the middle of the client area! There's also no reason why the resizing is associated with the window border or even worse, one of the corners.

      Well, there is, because historically mice have had too few buttons (wonderful innovation, Apple!), so instead of having a physical button reserved for only one task such as moving or killing a window (etc), we've reserved screen space for those tasks. Except screen space is more valuable for actual applications, so we've shrunk the borders and window icons until they're just hard enough to hit that people still put up with it.

      Of course you can fix these things if you're willing to hack your window manager, but the vast majority of people are stuck with those choices nevertheless.

      2.We need to start some place with this and 3D displays and how to interact with them have been around for a while but they will only be improved on when try to figure out how we work with them. This may not be as quick or intuitive as a mouse, but if people don't do work like this then we will be stuck with mice, track-pads and touch screens and the like. for ever.

      You're forgetting keyboards, which are a lot more expressive than mice. I'm also not convinced that aping interfaces designed by hollywood to be visually interesting in movies is the way to go. But I agree that experimentation and crazy ideas are a good way forward at least.

    7. Re:Half cool by slowLearner · · Score: 1

      you are right about point 1 I should have said "to the state of functionality that is present today" instead of "right".
      Yes keyboards are a great deal more expressive than mice, but at the same point they also take a great deal longer to learn. My daughter could use a mouse from to click on stuff age 4 (only using one of the buttons cause the other one didn't do much to her eyes) I am not saying that she could do everything with the mouse but her grasp of where the mouse was and how it moved compared to her hand was one that was learned very quickly. So now she is 7 and has just really started using the keyboard and I guess she will not be completely comfortable with it for another year or so and with complete mastery some way off in the distant future.
      The main point is, do we want to tied to a desk, or workstation or office or room or do we want more mobile computer and possibly even one that will disappear altogether. If we want all that then I am not sure how often we will be using a keyboard, but I've been wrong before I did buy shares in a bank.
      I'd also better stop before I start sounding like a futurist.

    8. Re:Half cool by mhajicek · · Score: 1

      Keyboards are very good for data input. Mice and trackballs are nearly perfect for 2D input. If you want 3D input get a Space Pilot. http://www.3dconnexion.com/products/spacepilot-pro.html

  3. UUmmm not quite but by Anonymous Coward · · Score: 0

    When I read See through Computer I thought of Orac - although the displays were pretty dodgy the computer ITSELF was see through.
          (I guess Im showing my age. As for those from the USA http://en.wikipedia.org/wiki/Orac_%28Blake%27s_7%29 )

  4. Demos by Zaelath · · Score: 3, Interesting

    The Minority Report interface is cool because he's not doing work, he's creating a display for the audience. Hence, the only place I see gesture interfaces being useful is for Jobs 2.0 explaining why we should buy yet another incrementally better iPad, or Balmer showing how much Metro sucks for desktop use :)

    1. Re:Demos by Anonymous Coward · · Score: 0

      It's also good for future pre-crime :
      You were going to click a link to ThePirateBay - break the law, go to hell.

  5. Interns? by Anonymous Coward · · Score: 1

    a prototype PC designed by an intern with the Microsoft Applied Sciences Group

    How much freedom/power do interns have that they can design completely new interfaces and requisition all that hardware, resources, etc?

    Is this common in Microsoft and/or all tech companies?

    1. Re:Interns? by zAPPzAPP · · Score: 1

      Most likely the child/nephew of some MS boss.

  6. sweet by alienzed · · Score: 2

    now you can have your screen a mere 5-6 inches from your face! Honestly that looks hella awkward and nothing at all like minority report.

    --
    Never say never. Ah!! I did it again!
  7. Minority Report Patents Challenged by Anonymous Coward · · Score: 0

    I thought Apple had already bought the Minority Report patents

  8. Gesture Computing Will Never Last by chrismcb · · Score: 4, Insightful

    Gestures as the main UI input will never last. They might work for a few things, like say doing a presentation. But no one will stand them for very long.
    Stick your arms out. Now hold them there, for as long as you can. I'll wait.
    .
    .
    How long did you last? Two minutes?
    My hands sometimes get tired just using the iPad, at an angle.

    1. Re:Gesture Computing Will Never Last by txoof · · Score: 4, Insightful

      Gesture based input is lousy for coding, managing systems, writing books and pretty much anything that depends on the accuracy and specificity of a keyboard. But that doesn't mean that a display like this doesn't have its uses. Rendering and manipulating objects through this sort of interface would be amazing! Also, think about how clunky and absolutely STUPID touch screens were just ten years ago. Heck, you can still find the same old crap displays at airport checkin kiosks. Now think about how a good implementation in iOS/Android devices has totally changed the world of touch interface. The inovations of Apple, Android and others have given us totally novel ways of interacting with our data. I would have never voluntarily brought one of those old touch screens into my home, but I cary one around in my pocket every day now.

      This implementation might not be perfect, but it's a step in the right direction for novel forms of input. Once someone cracks an awesome 3D/Gesture interface that works well and doesn't make you feel like your stomach is going to push out through your eyes, it will quietly creep into ubiquity just like the (good) touch screens today.

      --
      This one's tricky. You have to use imaginary numbers, like eleventeen... --Hobbes
    2. Re:Gesture Computing Will Never Last by Skywolfblue · · Score: 3, Interesting

      Well, that's largely because most of today's gesture recognition is absolutely terrible at recognizing fingers precisely from a long way off.

      Waving arms is generally tiresome and not all that effective.
      But say, twitching a finger to flip through photos would be a lot easier then using a keyboard.
      A better version of kinect that can turn on the TV and load that DVD at the slightest wave of your hand? Awesome! now you don't have to look for that clunky remote anymore.

      However, this particular device with the screen in between the person and their hands, just awkward and plain ungainly.

    3. Re:Gesture Computing Will Never Last by jtownatpunk.net · · Score: 1

      Bingo. Traps and delts would be on fire after a few hours of moderate use. Probably less time.

    4. Re:Gesture Computing Will Never Last by bartoku · · Score: 4, Insightful

      Like everyone else who watch Star Trek The Next Generation, I have been dreaming about touch screen interfaces since the late 80s.
      Now that they are here...it was not worth the wait.

      I downloaded BlueStacks yesterday, installed my favorite Android games, and they are ten times better with a mouse!
      My fingers do not get in the way!
      Now if I only had a dual mice pointer I could do multi-touch

      I admit there is something innately pleasurable about a touch screen and feeling you are really interacting directly with the items on screen, but in the end it is mostly imprecise and frustrating.
      Sure some advances have been made allowing it to be tolerable, but occlusion of the screen by your fingers and the lack of tactile feedback mess it all up by design.
      As a result the iPhone has made it harder than ever to make a phone call and only slightly less frustrating to send text messages compared to a feature phone given that we used to have physical keyboards with a Blackberry.

      Touchscreens have their place and uses, they are not going away; but I look forward to the day we get our buttons back.
      Case in point is the Kindle Fire, who thought it was a good idea to exclude the external volume buttons?

      I am with chrismcb on this one, gesture computing only works really well in imaginary worlds like Minority Report.
      I mean why steal a concept from a movie that does not even properly justify the main character ripping his eye balls out? The real future is EEG and muscle computer control and muscle/nerve sensing control.
      Why bother making the gesture, when I can just think it?

    5. Re:Gesture Computing Will Never Last by Anonymous Coward · · Score: 0

      The big difference being that _you_ are assumed to consume content, not to create it..

      So no need for fancy tactile feedback and all such nonsense..

    6. Re:Gesture Computing Will Never Last by flirno · · Score: 1

      I wouldn't want to consume content by suspending my arms in the air repeatedly either. Carpel tunnel is bad enough but tennis elbow and the like will follow with the whole touch screen fad.

    7. Re:Gesture Computing Will Never Last by lxs · · Score: 1

      Think of the upper body strength your average geek would gain. Millions of overweight slobs will turn into Tai Chi masters overnight.

    8. Re:Gesture Computing Will Never Last by mhajicek · · Score: 1

      Now try to filter out all the false signals. People are always moving; scratching, twitching, shifting, stretching, etc.

    9. Re:Gesture Computing Will Never Last by elsurexiste · · Score: 1

      I agreed at first glance, but then I thought of orchestra directors and music players: they play 8 hours a day and don't falter. Perhaps it's just a matter of training and adequate resting positions?

      --
      I rarely respond to comments. Also, don't ask for clarifications: a brain and Google are faster, believe me!
  9. Pointless... by bertok · · Score: 5, Interesting

    There's zero benefit to transparent screens in the vast majority of use-cases, but there's a huge downside: loss of contrast ratio. In an era where contrast ratios as high as 10000:1 are achievable, why would want to go back to something as poor as 10:1 or even lower? The only use-case where transparent screens might be useful is vehicle heads-up displays, but even there it's not quite the right solution, because the focal plane is all wrong. Take a look at vehicles that do have HUDs: they all use reflection with an angled surface because it allows for the use of optics that projects a virtual display at a focal depth much further out than the surface itself. A display embedded into a windshield would appear fuzzy and out of focus if you look out at the road through it.

    Most of the time, gadgets and technology in movies are designed simply to look cool, not for actual practicality, and the result is often pretty but stupid. The GUI in Minority Report is a great example of this: nobody can hold their arms up in front of them for more than a few minutes! Try it: get out a stopwatch and hold your arms out level for ten minutes. After the painful burning in your shoulders stops, take a minute and think about doing that for an eight hour workday. No matter how cool it looks, the mechanical advantage of our shoulders just isn't great enough to allow this kind of interface, and never will unless we all become cyborgs first.

    Just because you saw something in a sci-fi movie doesn't automatically mean that it's the "future" and only the required technology developments are holding us back. It's not like script writers and directors have some sort of personal revelation of the One True Future that we must all aspire to.

    1. Re:Pointless... by Anonymous Coward · · Score: 5, Insightful

      Heaven forbid someone try and use their arms constantly. I mean what would happen if suddenly someone were to have to hold a flute, conduct an orchestra, or even swing a hammer for more than 2 minutes.

    2. Re:Pointless... by Anonymous Coward · · Score: 1

      In an era where contrast ratios as high as 10000:1 are achievable, why would want to go back to something as poor as 10:1 or even lower?

      Maybe someone who wants to start working on transparent screens with high contrast ratios? I don't even know how that'd work, but that doesn't mean it wouldn't work.

      The only use-case where transparent screens might be useful is vehicle heads-up displays

      Or augmented reality overlays a la Rainbows End.

      Perhaps in museums where you could cover paintings with transparent screens that bring up information/video about the piece when questions. You could put sculptures in transparent screen boxes as well.

      Perhaps a shop could set their front window to be a large transparent screen as well, detailing sales and so on allowing easy updates without blocking the displays.

      Perhaps you're right in that there's zero benefit to transparent screens in most cases, but a caveat. I would recommend you instead say that there's zero benefit to transparent screen in most cases *at this moment in time*. What future advances may bring, I couldn't guess, but we'll see.

      nobody can hold their arms up in front of them for more than a few minutes!

      Yes, and nobody can lift 500 pounds. Except that dude that benched 1075 pounds.

      Hell, to go by your example of waving your arms around, a musical conductor spends an entire concert waving his arms around with very few points of rest. According to your theory, he's a cyborg. According to my theory, humans can do a lot of stuff if we try.

    3. Re:Pointless... by Anonymous Coward · · Score: 0

      All I heard was Cyborgs.

      CYBORGS 2012!

    4. Re:Pointless... by Anonymous Coward · · Score: 0

      In these user cases there's always an enormous amount of windows our hero is navigating. Strangely, modern ui user cases involve around simplified screens with a single dominant window (iPad, Metro).

    5. Re:Pointless... by Anonymous Coward · · Score: 1

      augmented reality glasses

    6. Re:Pointless... by flirno · · Score: 1

      Playing an instrument designed around the human body is vastly different from touching a flat panel all day long which is not really designed. The whole interface is a kludge. Screens will need to be redesigned to make long term use of touch ergonomic and actually enjoyable.

      AKA key and mice for current monitors is great.

      Touch with current display technology sucks.

      Touch may work with a completely different kind of display technology that gets away from the '2d display panel at eye level while sitting' setup we have going on now.

    7. Re:Pointless... by RivenAleem · · Score: 1

      That's the darndest thing. I tried this after reading your post and found that one arm got tired quickly, but the other was able to keep at it for hours...

    8. Re:Pointless... by Anonymous Coward · · Score: 0

      You gotta remember to switch arms when watching porn

    9. Re:Pointless... by Kingrames · · Score: 1

      Tactile feedback.

      Hitting something with a hammer, you FEEL how hard you hit it and how far you've driven the nail in.

      touching the screen doesn't give you that.
      If a 3d display can fix THIS, then it'll be the new thing. But right now, keyboard and mouse offer tactile feedback, and the touchscreen does not. a 3d space where you wave your arms around? If it's anything like the Kinect I tried, it won't do.

      --
      If you can read this, I forgot to post anonymously.
    10. Re:Pointless... by tragedy · · Score: 1

      The reality is that the dude that benched 1075 pounds can actually only bench press around 655 pounds, which is still extremely impressive and beats the 500 pounds you mention. The super-high bench-press records you hear about are all done with bench-press shirts. These started as safety wear, but are now used as a performance enhancer. Essentially, they constrain the athlete's torso and arms so that the arms are held out stiffly. They lie back and several spotters lift the weight and lower it onto the lifter. The lifter then lifts from that position. Basically, the shirt turns the person wearing it into a big spring. There's still technique to it, and the people achieving these very high records still need to be immensely strong, but it has the same relationship to the regular bench press that cycling has to running.

      As for the holding arms out in front for several minutes, I agree that's ridiculous. Plenty of people can do it. Pretty much any position someone can hold for thirty seconds can be held pretty much indefinitely by someone who trains enough at at.

  10. Naah! by aglider · · Score: 1

    Those things are just marketing fluff!
    I'd like you to enter a complex sentence, a program fragment or even do anything you do today without a keyboard!
    Oh, yes, you can. It only takes 10x times to accomplish the task!
    And it'll be nice see all those persons in public wving the hands and talking loud with their portable 3D computers.
    Privacy!

    --
    Sent as ripples into the electromagnetic field. No single photon has been harmed in the process.
    1. Re:Naah! by thoughtspace · · Score: 1

      And what's the point of having a transparent display if you can only read the text from one side anyway?

  11. Future! by Zaldarr · · Score: 2

    Living in the future is so cool. Although it makes me wonder just how efficient this is. Looking at the video and how he switches programs, I've never really said to myself "you know, Alt-Tab could be a whole lot quicker and simpler." I can see the applications in things like CAD, gaming and such, giving another level of control to a 3D object represented on a 2D screen, and I know some designers who would love this stuff; but for the most part this is cool but not a huge leap in interaction for your everyday computing.

    --
    I write professional videogame reviews! http://www.digitallydownloaded.net/
  12. Heavy user of gesture-based interfaces by Anonymous Coward · · Score: 2, Funny

    For many years I've heavily used gesture-based interfaces--while driving!

    Hasn't had much of an impact though--everyone else still can't drive.

  13. They're doing it backward by Anonymous Coward · · Score: 1

    They should have this in front of the screen. Touchscreens are awesome (I hate working on my non-touchscreen laptop now that I got used to my touchscreen everything else), but gestures would be even more awesome if done right. The mouse is useful for some things, but for a lot of things like switching desktops (sorry for all you windows users for something us linux folks have taken for granted for a long time); opening new programs; and selecting, moving, copy/pasting and deleting text; a gesture interface that allows me to just pick my hands up from the keyboard and "tap" the space on the screen I'm *looking* at by putting my finger in the line from my eyeball to the screen place and making a tap motion (or a swipe motion, etc.) would be great. Just think, instead of having to tab alt-tab repeatedly, ctrl-alt-left, etc., being able to just slightly lift your hand and make a swipe left or right motion with one or two fingers to switch between windows or desktops. There's potential here if they get the screen out of the line of sight to your hands.

  14. Amazon checkout by Jeremi · · Score: 2

    Your shopping cart currently contains:

    [1] Microsoft 3D visualization/manipulation interface
    [1] RealDoll
    [1] 3-gallon tub of Vaseline

    Checkout now, or continue shopping?

    --


    I don't care if it's 90,000 hectares. That lake was not my doing.
    1. Re:Amazon checkout by Anonymous Coward · · Score: 0

      Doesn't petroleum jelly dissolve latex?

    2. Re:Amazon checkout by Anonymous Coward · · Score: 0

      Yes, petroleum dissolves latex.
      But why is protection needed with a RealDoll?

  15. Not a fan of transparent displays by fragMasterFlash · · Score: 2

    If this screen is going to be the only display on your workstation then the level of transparency needs to be variable and be fully opaque to prevent distractions when not interacting with volumetric UI elements. Ideally the display screen should not need to be transparent at all with the users hands rendered into the volumetric scene based on camera input.

  16. Heads Up Display by FairAndHateful · · Score: 2

    I can't wait until you can just buy a replacement windshield so you can get stuff done on your long commute! Or combined with your onboard GPS naviagation system with the augmented reality thing going on.

    It's transparent, so it's gotta be better than texting, right?

    1. Re:Heads Up Display by Anonymous Coward · · Score: 0

      I can't wait until you can just buy a replacement windshield so you can get stuff done on your long commute!

      Actually you have plenty of stuff to get done while commuting: what about driving the damn car?

    2. Re:Heads Up Display by FairAndHateful · · Score: 1

      Actually you have plenty of stuff to get done while commuting: what about driving the damn car?

      I feel dirty for doing this for a response to one of my own posts but...

      Whoosh. I thought I was obvious enough.

  17. moving/resizing windows [Re:Half cool] by Anonymous Coward · · Score: 1

    Windows can be moved/resized by clicking (middle/right button) anywhere inside the window while pressing the Alt key!

    Iirc, this feature has been available in X for almost a decade!

  18. cmon nvidia by strack · · Score: 2

    what i want to know is why nvidias 3d glasses dont have a few leds on them, so they can implement head tracking with a webcam in their stereoscopic 3d system. i wanna be able to lean left and right and have the 3d object on the screen distort and change so it seems like the screen is a animated hologram, much like johnny lees wii glasses demo, except with added stereoscopic depth.

  19. Is that a mac keyboard by Anonymous Coward · · Score: 0

    ...that we see under the screen? ;)

  20. No no no no by paiute · · Score: 1

    Were these people born without rotator cuffs? I don't even want to think what my arms would feel like if I had to lift them up over the keyboard to manipulate objects all day.

    --
    If Slashdot were chemistry it would look like this:Cadaverine
    1. Re:No no no no by flirno · · Score: 1

      They probably did not do time consuming real world testing or their arms would have been burning too. They could be the first to experience new types of repetitive motion strain.

  21. Change the way we use computers? by nurb432 · · Score: 1

    Nah, looks like the same old interface concept, just with a bit of new aged 'shine' attached. I don't see a fundamental revolution here, just evolution to give it more eyecandy.

    --
    ---- Booth was a patriot ----
  22. Re:Wait !! by meowris · · Score: 1

    Well... He is actually a Korean... http://leejinha.com/ABOUT

  23. God no. by JustAnotherIdiot · · Score: 1

    Some believe a future full of massive, gesture-controlled computer displays

    Yeah, since most computer geeks don't have enough wrist problems, lets have you flick your hand around!
    I'd much prefer vision/thought controlled computers, thanks. |:

    --
    What do I know, I'm just an idiot, right?
  24. Haptic Feedback Interface too, please! by McGuirk · · Score: 1

    Finally, an Omni-Tool!