Slashdot Mirror

Something similar already exists and provides haptic feedback so you can "feel" the surface you are sculpting with your pen/tool. I have to say i'm not that impressed with the results (see their gallery), but that's not necessarily a comment on the product, rather the artists who use it. Seems people have been getting much better results with stuff like ZBrush anyway

My industry is heading into this area, after using 3D milling machines going to 3D printers was a natural evolutionary step, especially in areas where thickness is more critical. We do a lot of lost wax investing in the dental lab industry, the basic process is 5,000 years old and is used in many artistic and artisan endeavors. Basically anything you can sculpt in wax or clear resins can be used as long as it burns out ashlessly, just sprue and burn out the resin, using a 3D printer is probably over-kill. You can invest with common industrial molding plaster all the way to silver, Gold or Chrome alloys need high-temperature investments.

For sculpting you would need exact 1:1 mapping, which neither the normal Wiimote nor the add on provide. And even if you have 1:1 mapping you still would need feedback to make it really useful, so a Wiimote won't replace a haptic device anytime soon. However it might be possible to use it in the same way you use a SpaceNavigator, i.e. to navigate around in 3D space.

"Haptic" seems to be the preferred choice in the robotics/controls community.

From the image in the article, it looks like they're using a SensAble/Phantom as their input device.
(although it looks like one of the earlier models - their current design is quite a bit more polished).

See: http://www.sensable.com/

As SensAble is (arguably) a member of that community, and also refers to their product via the term "haptic", this seems to be acceptable choice of nomenclature for the article.

Sensable (http://www.sensable.com/) already has a great set of controllers, but they are currently super expensive (thousands of $$).
Novint (http://www.novint.com/) is claiming they'll have a controller out next year in the sub $100 range which is comparable to existing industry haptics devices. If Nintendo licenses the hell out of this technology, it's possible for the price to drop even further.

Briefly, haptics is the study of the sensation of touch, google for more info.

Currently haptics devices from Sensable http://www.sensable.com/ have extraordinarily accurate force sensations. The problem with these haptics devices are that they cost in the thousands of dollars per device.

However Novint http://www.novint.com/ made a showing at E3 and are claiming they'll be coming out with a device under $100 in a year. Here's a link to an ign review off their website.

http://gear.ign.com/articles/709/709246p1.html

I think it would be neat (and probably in their best interest) if Nintendo kept supports and software hooks in mind for haptics devices in the future.

The problem of "feel" is being solved simultaneously to this remote manipulation technology, though the article may not mention it. Haptic interfaces give mechanical feedback to the controls to convey sensations like texture and viscosity. Using a tool like this I have virtually stirred syrup and cut open a human kidney. The former I can vouch as feeling almost indistinguishable from reality; you might have to find a more educated or psychotic person than myself to vouch for the latter.

I''ve been trying to find out if it is possible to get the Sharp Actius AL3DU laptop to run 3DS Max in autostereo mode together with a plugin for a 3D haptic input device called the SensAble PHANTOM Omni. It also might be cool to add the 3Dconnexion SpacePilot input device for navigation.

I think this would be the ultimate interface for 3D design. Has anyone had any experience with this? I've been emailing all the companies involved and have gotten responses saying that there are problems with this setup. I'd like to try it out on my own, but putting it together would cost a fortune and I don't want to blow all that cash just to find out it doesn't work.

I''ve been trying to find out if it is possible to get the Sharp Actius AL3DU laptop to run 3DS Max in autostereo mode together with a plugin for a 3D haptic input device called the SensAble PHANTOM Omni. It also might be cool to add the 3Dconnexion SpacePilot input device for navigation.

I think this would be the ultimate interface for 3D design. Has anyone had any experience with this? I've been emailing all the companies involved and have gotten responses saying that there are problems with this setup. I'd like to try it out on my own, but putting it together would cost a fortune and I don't want to blow all that cash just to find out it doesn't work.

The current haptic devices include the Phantom OMNI. This is the only system, I've had the chance to use with 3D software. It gave good feedback on the virtual models, but it would probably be much better if you could combine it with atomic force microscopy.

The patent was file in 2000.

But in 1996 I played with exactly the same kind of haptic technology (or called force feedback) before. It was called Phantom (tm) used in scientific apps:
http://www.sensable.com/products/phantom_ghost/pre mium6DOF.asp

For example you can put on a finger thimble and feel a virtual 3D surface.

I even wrote an SGI program to use it as a flight cnotrol device.

One of the research labs in my university have purchased a couple of these laptop along with a Phantom Omni pen and stands.

The laptop can display photographs and 3D applications in stereo, with the actual depth field being +/-4 inches in front of and behind the laptop screen. The haptic pen provides force feedback to give you an idea where the surface is (resistance will increase if you try and move the pen into somewhere "solid", but will be flexible otherwise).

The two can be combined together with special stand which allows you to turn the laptop upside down, and see a reflection of the screen from a mirror, with the goal being to allow you to use the pen as naturally as possible.

Ok, I'm not as religous a microsoft hater as many here, but is there ANY good reason to have this run on XP?

You'll find a lot of exotic computer gadgets you find on the net tend to use Windows. I presume it's simply because it's probably cheaper to develop for just one platform, and they choose to use the most common one. Here are some examples of fancy displays and input devices I've found on the net...

• Autostereo displays
• Panoram Technologies
• Elumens Corporation
• Actuality Systems (site seems like it's down at the moment)
• Sensable Technologies
• Polhemus.com
• Ascension Technologies

Some of their products may be compatible with other operating systems, but most I looked at have a Windows bias.

I expect we can also look forward to seeing many great new haptic applications being created as a result of SensAble contest that the article mentions. Being able to feel and interact with simulated objects in a very intuitive way will undoubtedly become an increasingly import part of how we use computers.

I would personally appreciate hearing from anyone out there who has an interesting haptic application in the works (or wants to hear about mine). scottgilroy2000 - a - yahoo dot com

is this really the future ?

An article on InformIT.com looks at the current state of haptic technologies: "In the consumer realm, two companies dominate the field in the creation of tactile I/O devices: Immersion Corporation and SensAble Technologies. Right now, each seems interested in consolidating a position in the marketplace.

My job at the time was to find new useful applications for Haptics, that were not in the medical field.
Apart from games and widgets (buttons, sliders etc), we built a wind surfing simulator which could teach the user about how wind drag works.
We also investigated 3D (marking) menus with "magnetic" grid-lines and industrial and artistic/craft applications (jewelery, dentistry) where you could first work with tools on an object and then let the machine mimic your actions afterwards, possibly in another scale. We also sketched on a paint program where you could feel the viscosity in the oil paint and the texture on the canvas.

We were very limited by software. The hardware drivers required realtime performance, and the graphics was running 100% in another thread, which made things problematic under Windows NT which wasn't really a realtime OS. We would also have liked to have a finite element simulation framework for simulating flexible materials.

(btw. the job sucked, but the technology was so cool so I stayed on for longer than I should have... )

I'm surprised nobody has mentioned the Phantom devices. They would work better than any of the devices mentioned in the article, and actually provide force-feedback. I've never used one, but I presume that if you let go of it, it probably could stay in the same position you left it in, so that wouldn't be an issue like it would with the other devices. The site also has 3D CAD applications that use it, and an SDK.

surgeons use 'em....
Phantom units are actually pretty affordable & fun... Like working w/ clay
www.sensable.com

That is great and all, but I believe the problem with all that, is that you -still- can't intereact with the 3D object you're seeing, at the place where it -appears-.

Yes you can. There are 3D input devices such as mice, joysticks, gloves, and haptic devices.

• 3D haptic input device
• 3D scanner
• 3D printer

Cheap: no. Easy to use: fairly. 3D: oh yeah.

The phantom is the darling device of many haptics researchers right now. It is pretty much exactly what you'd expect a 3D mouse to be. It's price pretty much limits its market to researchers and serious artists at the moment. I've had the chance to play with it and I can tell you that it's a fun little toy. No one has built a desktop for it yet though.

Right now we have reached the beginning of the transition away from button inputs to gesture inputs. There are of course many projects working on gesture inputs. The first that are really viable are the 2D ones from fingerworks.com. The next will be refined versions of the P5 Glove or the sensible phantom. I think eventually gesture based input will be the type used in Minority Report, (see the 1st and 10th images in the gallery).

Finally, I think we will move on to direct input. It's been shown that people can control very simple objects, (move a ball to the top or bottom of the screen), with electrodes connected to their head. Unfortunately so far it has not been responsive enough to see application. Input may also be of the form in Ghost in the Shell where people have wireless connections through implants in their body and also physical jacks in the back of their neck. (Another thing shown in the movie are fingers that come apart on wires to type. Rather than that I'd expect a low-power data transmission in the fingers so set the fingers in appropriately shaped cavities and have the data transmitted across the skin.)

Keyboards are nice. They have worked for a long time, but it is time to replace them. Slowly we can transition from keyboards, through the 2D gesture inputs of fingerworks to 3D inputs along the lines of minority report at which time, hopefully, direct input methods will be viable.

I'm sure the fine folks at SensAble are disappointed that the article doesn't mention them or indicate that PHANTOM is an abbreviation (though I can't find the expansion at the moment). I seem to remember it being pHANTOM back in '97.

Back in my college days, I got to play with a PHANTOM at URCS. It's an arm that can exert translational forces at the tip. They're available with a thimble or a stylus on the end. We had the thimble style, which had a weird harness joint that always kept the finger tip at the "end" of the arm regardless of rotation. You strapped in the tip of your finger in not entirely unlike strapping a foot into a bike pedal.

From a programming standpoint, you could query it for position and velocity, then send back a force vector. Multiplying the velocity vector by a value between 0 and -1 gave the impression of moving through peanut butter or motor oil or water. I tried using a positive multiplier, but that got dangerous very quickly. You were supposed to be careful about hitting the edges of the range of motion.

You could simulate surfaces by monitoring the position for crossing the surface and returning a force vector orthogonal to the surface. At the time, really hard surfaces didn't work. You could get many gradations of sponginess, but past a certain point, it wouldn't get any more solid. Surface texture (bumpy vs. smooth) and shape were easy to feel, though.

One of the really, really slick things my advisor commissioned was to put two PHANTOMS facing each other so you could put your thumb in one and index finger in the other. Then you strapped on a VR headset with a magnetic head tracker. You could see a sphere each for the thumb and finger tips. There were boxes floating around that you could grab, throw, and bounce off of each other and the "walls". Although it was a zero-gravity environment, the "weight" of the blocks between the fingers was very convincing, as were the collisions. You could bounch a block basketball style, and it felt about right. The head tracking contributed greatly to overcoming the proprioception disconnect.

I'm sure the fine folks at SensAble are disappointed that the article doesn't mention them or indicate that PHANTOM is an abbreviation (though I can't find the expansion at the moment). I seem to remember it being pHANTOM back in '97.

Back in my college days, I got to play with a PHANTOM at URCS. It's an arm that can exert translational forces at the tip. They're available with a thimble or a stylus on the end. We had the thimble style, which had a weird harness joint that always kept the finger tip at the "end" of the arm regardless of rotation. You strapped in the tip of your finger in not entirely unlike strapping a foot into a bike pedal.

From a programming standpoint, you could query it for position and velocity, then send back a force vector. Multiplying the velocity vector by a value between 0 and -1 gave the impression of moving through peanut butter or motor oil or water. I tried using a positive multiplier, but that got dangerous very quickly. You were supposed to be careful about hitting the edges of the range of motion.

You could simulate surfaces by monitoring the position for crossing the surface and returning a force vector orthogonal to the surface. At the time, really hard surfaces didn't work. You could get many gradations of sponginess, but past a certain point, it wouldn't get any more solid. Surface texture (bumpy vs. smooth) and shape were easy to feel, though.

One of the really, really slick things my advisor commissioned was to put two PHANTOMS facing each other so you could put your thumb in one and index finger in the other. Then you strapped on a VR headset with a magnetic head tracker. You could see a sphere each for the thumb and finger tips. There were boxes floating around that you could grab, throw, and bounce off of each other and the "walls". Although it was a zero-gravity environment, the "weight" of the blocks between the fingers was very convincing, as were the collisions. You could bounch a block basketball style, and it felt about right. The head tracking contributed greatly to overcoming the proprioception disconnect.

The device they are using seems to be the Phantom by SensAble Tecnologies (product page). I used one of these a few weeks ago at USC's Integrated Media Systems Cetner, they're pretty cool. If you are interested in this kind of thing, the field is called "haptics," from the greek "to touch."

You can touch digital models. See Reachin and SensAble.

There are some commercial companies who have been working in this area for several years.
Most notably Reachin and SensAble.

Finally, there is effective research being done, to make computers, as Douglas Adams (RIP) would have put it, more than typewriters with televisions in front of them. By combining a 3D display with the 3D sensory technology in development by companies like Sensable people can interact with a genuine multidimensional, immersive computer environment. Beyond the prOn possibilities of merging the technologies (fondle-able images anyone?) is a whole new world of design. Though I can't see the displays improving word processing or programming much, designing products on a 3D screen, touching them in virtual space, and printing them on 3D printers offer a whole new world of simple product development. Converse is already using a 3D printing and fax system to design shoes in boston, and give their factories in Asia models to work from.

If anyone is interested in the actual user interface research going using force feedback devices like the logitech mice and the phantom then you could do worse than check out this workshop Other good resources on the web are the various phantom based PUG and PURS workshops ASME have some sort of annual winter symposium aswell but I forget the url......

I seem to recall that all of Logitech's force-feedback technology is licensed from SensAble (or maybe I'm confusing them with Haptic Technologies. When I got curious about "haptic" devices about a year ago, I remember seeing a version of the WingMan on the SensAble web site. They were pushing it purely as a GUI enhancement ("Feel when you mouse moves from one window to another!), but apparently weren't getting any takers. Only Logitech took them up, and for a long time, even Logitech considered it purely a "game device".

Nowadays, SensAble concentrates on 3D haptic devices, useful for design engineering, sculpture, etc. It occurs to me that this too would make a nice GUI pointer. Think of the idioms you could invent with a 3d pointer! But of course, the gadget is much too expensive for that market.

Which also explains the problem with the WingMan and iFeel. It isn't that physical feedback is a bad idea. It's just that they haven't figured out how to make a real feedback device that's cheap enough for the mass market.

__________

I seem to recall that all of Logitech's force-feedback technology is licensed from SensAble (or maybe I'm confusing them with Haptic Technologies. When I got curious about "haptic" devices about a year ago, I remember seeing a version of the WingMan on the SensAble web site. They were pushing it purely as a GUI enhancement ("Feel when you mouse moves from one window to another!), but apparently weren't getting any takers. Only Logitech took them up, and for a long time, even Logitech considered it purely a "game device".

Nowadays, SensAble concentrates on 3D haptic devices, useful for design engineering, sculpture, etc. It occurs to me that this too would make a nice GUI pointer. Think of the idioms you could invent with a 3d pointer! But of course, the gadget is much too expensive for that market.

Which also explains the problem with the WingMan and iFeel. It isn't that physical feedback is a bad idea. It's just that they haven't figured out how to make a real feedback device that's cheap enough for the mass market.

__________

Have you checked out Freeform from Sensable? They have a 3d stylus (sort of like a 3d wacom tablet).

The software drivers for Windows NT comes with a mouse emulation mode, but it has some serious drawbacks. The stylus has to be touching an imaginary plane (created with force-feedback) in order to move the pointer and the stylus' only button is used as the left mouse button.

If someone would write a new mouse emulator for the Phantom, it could just rock. It would become the ultimate pointing device.

• It wouldn't have to be picked up. By applying a feedback force in the upward direction, the stylus would appear weightless and stay in place (in space) when you release your grip of it.
• It is pressure sensitive
• Senses the pen angle. The angle is actually very significant for real drawing/painting. I have only seen it being emulated in software before, and it has been done poorly.
• It has 3D tracking. Actually, that is what it was designed for.
• Force feedback. You could feel windows, widgets and icons. The resolution of this device is amazing. The software that I am using simulates rough, frictional and bumpmapped surfaces.
• With a mirror or a flat display in front of the device you could align the position of the screen image with your hand to get hand-eye coordination. (see this)

• No Linux drivers
• All surfaces feel rubbery. There will always be a delay between stylus movement and feedback. Therefore, it is not possible to simulate hard surfaces very well. When hardness increases, so does vibrations in the device and it just does not feel right.
• Expensive. It carries a five-digit pricetag (USD).

OK. First, to declare my bias, I work for a company called ReachIn and all we do is this sort of stuff.
3D is definitely worthwhile for modelling, visualisation and simulation. How worthwhile is a good question. But even these CAVES, at about ~$1 M have been bought by the oil and automotive industries and paid for in a few weeks.
3D window managers are interesting, but how usefull they are remains to be seen. Without computers, for most things people actually work in 2D. Books and paper are 2D. Whether this is because the technology is deficient or because this is the most efficient way is a really tough question.
I'm not yet convinced that you can get a whole lot more from a 3D UI, but, this is probably like some UNIX hacker in 1975 saying, what Windows? Nice gimmick, but what do you NEED them for, I can switch TTYs fast.
But you should try shaping something in 3D, it's really a new experience, anyone who has ever tried to use 3D Studio Max or Rhino or whatever will appreciate that there is some manipulation of 3D stuff that is painfull in 2D 'cause it's SOOOO counterintuitive. What we do is have a PHANTOM from SenSable and co-locate it so you can see what you can touch. It is really quite neat. We can then do medical training and shaping like you wouldn't believe. If you can, get to SIGGRAPH 2K in New Orleans and have a look around, they'll be stuff from us and hopefully some others that will show that 3D is really capable of being worked in.

A few years ago, when I worked at Discovery World we ported the armlib software from UNC to work on RT-Linux with the PHANToM. Our work is available here . It was pretty fun work while it lasted. Unfortunately, when I left, the project died. We did develop an extension of the UNC client/server protocol for remote manipulation - we called it TouchU-TouchME, but we never had a chance to present a paper on it. Oh well... I note on the UNC web site that someone has created a scanning-tunneling probe using Lego Mindstorms. This would be an ideal mate to the work that WillWare describes above.

OK just as a background check I've worked on some haptic stuff. Haptic being a magic word that means force feedback for lack of a better definition. I worked specifically on two projects, which had much better hardware than this thing. With those it was possible, to feel things but not perfectly. Generally (although this was 1-2 years ago) minute details were lost. Things like textures were difficult to feel. Best example I can relate was you hand running quickly over a keyboard type texture. Friction and drag were possible but not very accurate or reliable. That stuff was all done on the PHANTOM. Stuff for this device has gotten better but still if you want realistic feeling you need a lot of computer power behind it. Typically it can be used for medical imaging with SGIs. The other device I worked with was based on magnetics. Its here. It had better precision in the feelings, but less of a range (err at least the prototype did its been about 1-2years since I was there). Basically you could feel things very precisely but only over about 1". The PHANTOM allowed you to feel things kinda sloppily over a much larger range (walls were a little spongy).

The point of all that blathering - I doubt that could get an acurate feeling in these mouse type devices. The input an computation you would need is pretty large. More likely it is what was stated in the article here - a rumble pack in a mouse. So I doubt you could tell the difference between a tennis raquet, a can of soda, or an adult site.

As for any benefits to the blind, I believe there are beter products that do braille type output to a wrist rest type thing that sits below your keyboard. This probably can't do braille due to lack of exactness.
-cpd