Slashdot Mirror

My job at the time was to find new useful applications for Haptics, that were not in the medical field.
Apart from games and widgets (buttons, sliders etc), we built a wind surfing simulator which could teach the user about how wind drag works.
We also investigated 3D (marking) menus with "magnetic" grid-lines and industrial and artistic/craft applications (jewelery, dentistry) where you could first work with tools on an object and then let the machine mimic your actions afterwards, possibly in another scale. We also sketched on a paint program where you could feel the viscosity in the oil paint and the texture on the canvas.

We were very limited by software. The hardware drivers required realtime performance, and the graphics was running 100% in another thread, which made things problematic under Windows NT which wasn't really a realtime OS. We would also have liked to have a finite element simulation framework for simulating flexible materials.

(btw. the job sucked, but the technology was so cool so I stayed on for longer than I should have... )

That is great and all, but I believe the problem with all that, is that you -still- can't intereact with the 3D object you're seeing, at the place where it -appears-. That is, you're seeing the object in front of you, but your hand is like 30cm away on the mouse (or whatever 3D input device) trying to manipulate it. That's one thing we solved at ReachIn (a company where I used to work for) by projecting the stereo image onto a mirror, and have a 3-DOF force-feedback device installed under the mirror, so that the hand can be -at the same place as the object-!

But anyway, that is besides the point - I think technology like this is limited (with the exception of entertainment) unless you can get your hands into the 3D image and use it as if it were truly virtual reality. It is one thing to look at a 3D image of a brain tumour, but if you still have to use a 3D-mouse to manipulate it, then there is still an intuitive leap to be made when using applications written for the display - just like the intuitive leap between the 2D mouse and the 2D display. What this needs is to be combined with technology like Reachin [troll warning] where you can actually see the data in the same space where you hands are working on it. Much more appropriate :)

You can touch digital models. See Reachin and SensAble.

I'm sure this is also a good thing for getting more haptics rendering (ie. force feedback rendering, for example) working on linux.

The challenge with haptics is getting at least a 1kHz frame-rate on force renderings. This requires soft-realtime perfomance at a minimum and is tricky on most platforms.

There are some commercial companies who have been working in this area for several years.
Most notably Reachin and SensAble.

The software drivers for Windows NT comes with a mouse emulation mode, but it has some serious drawbacks. The stylus has to be touching an imaginary plane (created with force-feedback) in order to move the pointer and the stylus' only button is used as the left mouse button.

If someone would write a new mouse emulator for the Phantom, it could just rock. It would become the ultimate pointing device.

• It wouldn't have to be picked up. By applying a feedback force in the upward direction, the stylus would appear weightless and stay in place (in space) when you release your grip of it.
• It is pressure sensitive
• Senses the pen angle. The angle is actually very significant for real drawing/painting. I have only seen it being emulated in software before, and it has been done poorly.
• It has 3D tracking. Actually, that is what it was designed for.
• Force feedback. You could feel windows, widgets and icons. The resolution of this device is amazing. The software that I am using simulates rough, frictional and bumpmapped surfaces.
• With a mirror or a flat display in front of the device you could align the position of the screen image with your hand to get hand-eye coordination. (see this)

• No Linux drivers
• All surfaces feel rubbery. There will always be a delay between stylus movement and feedback. Therefore, it is not possible to simulate hard surfaces very well. When hardness increases, so does vibrations in the device and it just does not feel right.
• Expensive. It carries a five-digit pricetag (USD).

The software drivers for Windows NT comes with a mouse emulation mode, but it has some serious drawbacks. The stylus has to be touching an imaginary plane (created with force-feedback) in order to move the pointer and the stylus' only button is used as the left mouse button.

If someone would write a new mouse emulator for the Phantom, it could just rock. It would become the ultimate pointing device.

• It wouldn't have to be picked up. By applying a feedback force in the upward direction, the stylus would appear weightless and stay in place (in space) when you release your grip of it.
• It is pressure sensitive
• Senses the pen angle. The angle is actually very significant for real drawing/painting. I have only seen it being emulated in software before, and it has been done poorly.
• It has 3D tracking. Actually, that is what it was designed for.
• Force feedback. You could feel windows, widgets and icons. The resolution of this device is amazing. The software that I am using simulates rough, frictional and bumpmapped surfaces.
• With a mirror or a flat display in front of the device you could align the position of the screen image with your hand to get hand-eye coordination. (see this)

• No Linux drivers
• All surfaces feel rubbery. There will always be a delay between stylus movement and feedback. Therefore, it is not possible to simulate hard surfaces very well. When hardness increases, so does vibrations in the device and it just does not feel right.
• Expensive. It carries a five-digit pricetag (USD).

OK. First, to declare my bias, I work for a company called ReachIn and all we do is this sort of stuff.
3D is definitely worthwhile for modelling, visualisation and simulation. How worthwhile is a good question. But even these CAVES, at about ~$1 M have been bought by the oil and automotive industries and paid for in a few weeks.
3D window managers are interesting, but how usefull they are remains to be seen. Without computers, for most things people actually work in 2D. Books and paper are 2D. Whether this is because the technology is deficient or because this is the most efficient way is a really tough question.
I'm not yet convinced that you can get a whole lot more from a 3D UI, but, this is probably like some UNIX hacker in 1975 saying, what Windows? Nice gimmick, but what do you NEED them for, I can switch TTYs fast.
But you should try shaping something in 3D, it's really a new experience, anyone who has ever tried to use 3D Studio Max or Rhino or whatever will appreciate that there is some manipulation of 3D stuff that is painfull in 2D 'cause it's SOOOO counterintuitive. What we do is have a PHANTOM from SenSable and co-locate it so you can see what you can touch. It is really quite neat. We can then do medical training and shaping like you wouldn't believe. If you can, get to SIGGRAPH 2K in New Orleans and have a look around, they'll be stuff from us and hopefully some others that will show that 3D is really capable of being worked in.

The PHANTOM, as mentioned above, is really cool, it allows real tactile interaction.

I work at a company in Sweden, ReachIn Technologies that produces hardware and an API that combines graphics and haptics in the one spot, which is really impressive.

There are lots of benefits of having a proper haptics system, it allows the incredible control that we have of our fingers ( we can sense bumps in a smooth surface of 1 micrometre ! ) to be used. There are heaps of potential markets as well, the one catch is that these systems are fairly expensive at the moment, oh, actually, there is one other, we don't support Linux....