Slashdot Mirror
Intelligent Thimble Could Replace the Mouse In 3D Virtual Reality Worlds
New submitter anguyen8 (3736553) writes with news of an interesting experimental spatial input device. From the article: "The mouse is a hugely useful device but it is also a two-dimensional one. But what of the three-dimensional world and the long-standing, but growing, promise of virtual reality. What kind of device will take the place of the mouse when we begin to interact in three-dimensions? Anh Nguyen and Amy Banic ... have created an intelligent thimble that can sense its position accurately in three-dimensions and respond to a set of pre-programmed gestures that allow the user to interact with objects in a virtual three-dimensional world. ... The result is the 3DTouch, a thimble-like device that sits on the end of a finger, equipped with a 3D accelerometer, a 3D magnetometer, and 3D gyroscope. That allows the data from each sensor to be compared and combined to produce a far more precise estimate of orientation than a single measurement alone. In addition, the 3DTouch has an optical flow sensor that measures the movement of the device against a two-dimensional surface, exactly like that inside an ordinary mouse."
The prototype is wired up to an Arduino Uno, with a program on the host machine polling the device and converting the data into input events. A video of it in action is below the fold, a pre-print of the research paper is on arxiv, and a series of weblog entries explain some of the development.
Why a thimble in Monopoly.
I remember the thimble IBM used on one of their early laptops long ago, so...no, thanks.
I had a 3D ring based system 15 years ago. I hope this is better
ground floor opportunity: kickstarter to the rescue...pink pony thimbles, green zombie thimbles, and disposable/recyclable ones, too.
Waving your arm around to interact with the 3d space will surely tire your arm out if you are sitting at a computer. Using it in the 2d space seems like an overly complex way to get the functionality of a mouse.
However, it does seem pretty neat despite the aforementioned pitfalls. It could be a neat way to interact with a computer that you might be standing a few feat away and need to start a process or something, just do the correct hand motion and it would start. Sure I could come up with more uses if I thought about it.
That's the only thing I can think of every time something like this surfaces. Try holding your arm horizontally in the air for 15 minutes. Bet you get tired before you hit 5. And I use the computer for hours on end.
Now, put one on each finger, and we can make virtual keyboards, and have all sorts of fun with the UI. 8 fingers.... 8 bits per byte... we could have each finger represent a bit, on or off. Then without moving the hands or stretching the fingers, each key on the keyboard is represented by which the fingers being lowered or not. Saves the thumbs, one for mouse positioning, the other for enabling mouse mode. In mouse mode, the rest of the fingers would do things like Ctrl-Alt-Shift-Super-Meta on one hand, and left click, right click, middle click on the other hand.
Robots. Lots of robots.
Scott Adams predicted this many years ago, and I still agree with his analysis.
3DConexion, formerly Spacware/Spacetech and possibly a few other names have had advance 3D positioning devices since forever ago, replacing buttons and dials in CAD and CAE software. A thimble is not going to be more or less ergonomic. As with mice, I'm sure it's a personal preference so someone will like it better.The Spaceball however is designed for use with a relaxed hand and does much more than 3D positioning. Like zooming, centering, and what ever else you program the buttons to do
If you want to take "it's position" as the starting point I'll argue that the ergonomics is less than that of a Spaceball, and more in line with motion detecting devices that again we have had for well over a decade. The thimble won't be as useful in HFE, because open and closed hands are at least as important as position and rotation.
In short, this is a wheel that's already been invented. I don't see anything "novel" or even better than what we have had already. Maybe if fits a niche I'm not aware of or care much about.
-The wise argue that there are few absolutes, the fool argues that there are no probabilities.
These 3D whizmos, like for example LEAP motion (incredibly cool), all work great.... for about 20 minutes. Then you put them in the drawer because they require too much muscle coordination and energy to operate. in contrast when you REST your finger on a scroll wheel or REST your hand on a mouse it is not merely not moving, it is at rest in 3 dimensions. it only takes a small effort to move it, but you are not having to run a whole lot of muscles in coordination to keep the hand or finger in a constant position. it's hard to poise your hand in empty space. In the old days, good typists could do this with hands poised over the KB and fingers hovering above the keys. Most people now days use palm rests or put pressure on the keys. those old time secretarial pool typists had to sit up straight and brace their feet on the floor to pull that off. Girdles probably helped!
the first successful mouse replacement will have that feature. Perhaps something with haptic feedback to support your finger a little till you really want to move it.
personally I suspect the some sort of eye motion or maybe a joystick like thing will be the first 3D controller that people can use for long periods.
Some drink at the fountain of knowledge. Others just gargle.
LEAP promised similar things. Logically, their technology should work well, but the execution was piss poor. The trick to getting 3D finger interaction to work will either be higher immersion, such as proportional (to the controller) 3D displays or Occulus Rift style implementations where you can see your hand interacting. Another issue LEAP has is defining the horizontal and vertical ground planes. Their controller would work better if it detected and calibrated to you monitor and activation motions occurred when you touched the screen in many cases.
3D gesture identification and intent management seems to be a stumbling block so far as well. Seems largely that programmers figured out the hand skeletal structure and then immediately ignored that musculature, tendons, and fine motor control are not the same in all positions and directions.
Some example dumb hand / finger gestures for 3D control (I see these in LEAP motion software and in proposed hand gesture libraries for similar technology):
- Triggering a thumb against the side of the index finger - most of the hand moves, especially the index finger (which is typically being keyed off of for cursor position)
- Triggering by pulling the index finger like a trigger - surprisingly inconsistent when there is no resistive grip or button
- Holding a splayed out hand(s) horizontally, mid air as a default centered position
- Keying z-rotation off of a hand pointed at the screen as if one's arm protruded from the chest
- Expecting the hand to translate mid-air like camera dolly & track.
- Lots of other ergonomically / kinematically ignorant ideas. I think they modeled everything with those articulated wooden hands for clay sculpture. And no arms.
Just some things to consider before creating your own 3D motion controller...
This was the first thing I thought of when I read this. With a positional accuracy of 1mm (sure to get better) you can 'easily' type away at a virtual keyboard. You could do the above key chording as well for shortcuts or whatever. I would love to lean back in my chair, position my hands comfortably, and be able to type without being tied to a particular format of keyboard on my desk. My current favorite keyboard is a Microsoft 4000, I HATE the non-ergonomic straight line keyboards, and save particular loathing for laptop keyboards.
I agree.. with all 10 fingers, it might be an overkill. However, modular solutions like 3DTouch will be the future.
Johnny Mnemonic gloves where cool this seems like an mini ver of them.
It doesnt lag 3-5 video frames after the movement.
Who logs in to gdm? Not I, said the duck.
My special name is is "intelligent thimble".
We all have a special name for it and I call mine "intelligent thimble"
Can you imagine how tiring it'd be to hold your hands in the air, and wave 'em about for any length of time?
How difficult is it for most people to memorize relative finger positions as opposed to absolute positions on a keyboard? How much lag will users experience transitioning to some other interface each time they get to a character they haven't memorized?
This is a simple MPU with Kalman filtering for relative motion estimation and the sensor out of a laser mouse for absolute positioning over a table. OculusVR uses the former technique for detecting rotation and heading in their Rift HMD. Control-VR has the 10 finger version of the same idea already working as a prototype. The addition of a laser mouse sensor is new, but why is that worth a paper? Optical tracking is also used in the Rift DK2 but using a camera and pose estimation.
nobody cares to hold sth the mouse is popular because u can get stuff done with minimum energy and high efficiency, add a UP down key on the mouse or keyboard and u got ur 3rd dimension,
"Hi, welcome to MIT Tech Review. You've never read our site before, you probably know nothing about our site since you followed a link from an aggregator, and we're blocking you from reading the site now via this pop-over. WOULD YOU LIKE TO SUBSCRIBE?!?!?!?!?!"
No.
To the best of my knowledge, no.
Science is all about firing a drunk pig out of a cannon just to see what happens.
Sorry to break it to you guys, but this has already been done in India...in a more polished form. http://www.wearfin.com/
I have used and made software for a device precisely such as this one, with position and direction in space, only that it was not worn but handheld and called a "wand". This was fifteen years ago, '98/'99.
It was used for control in a CAVE environment where you are enclosed in a cube of six computer screens with the perspective adjusted to the position and direction of your 3D glasses.
"We mustn't be caught by surprise by our own advancing technology" -- Aldous Huxley
I am fucking up your UI. Pray I do not fuck it up further.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
That looks like an Asian lady!
I've been navigating 3D worlds my whole career, and a Spaceball works just fine. A Spaceball already has a comfy surface to rest your hand, and you don't have to squeeze a switch to move.
spot heu doue F]or Beeu the Pest' broPlems Hth nuless on cau ork
The smaller the "thimble", the more people will like having them on every finger.
I'm already hesitant to use others' keyboards. Don't loan me your thimble. I won't know where it's been.