Slashdot Mirror

Interesting. It kinda replicates the 2004 PhD thesis of Mathias Kolsch when he was at UCSB (now at the Moves Institute, NPS). Mathias's work is known as HandVu. The source code for HandVu is available on that web site, along with videos.

Interesting. It kinda replicates the 2004 PhD thesis of Mathias Kolsch when he was at UCSB (now at the Moves Institute, NPS). Mathias's work is known as HandVu. The source code for HandVu is available on that web site, along with videos.

Why are you limiting this to voice processing? Consider a camera based system which records your hand gestures to control your computer. You should see the work of Juan Wachs for an introduction on this technology: http://www.movesinstitute.org/~jpwachs/index.html

i think vision recognition and hand tracking have the most potential for musical control. try hooking up HandVu or CamSpace to Max/MSP for example.

Open source software for gesture recognition http://www.movesinstitute.org/~kolsch/HandVu/HandVu.html

There are fundamental problems with all of these types of devices-- they 1) don't let the body handle momentum naturally and 2) don't stimulate the vestibular system in a way that is consistent with the visual or proprioceptive (the body's sense of where its limbs are) cues.

1) Momentum: On a 2-D treadmill, the omni-directional treadmill is supposedly fast enough that it allows for running. But when you are running and then change direction quickly, your body will lean into the turn to counter its momentum. Doing this on the treadmill will make you fall over. Someone once described it as "running on a slippery ice cube".

2) Vestibular cues: Our body can sense motion even without visuals or body movements. This is why some flight simulators have motion platforms [://www.simlabs.arc.nasa.gov/vms/motionb.html]. One post above said that the treadmill should reduce motion sickness because it provides body motions as well as visuals. But a treadmill doesn't cue the vestibular system. One theory of motion sickness is that it results from a mismatch of visual and vestibular cues. In the back seat of a car, your visual cues say you are still (relative to the inside of the car) but the vestibular system says you are moving. Similarly in a IMAX theater or while playing an FPS on a big screen, your visuals say you are moving but your vestibular system says you are still. Knowing how you are moving is critical for maintain balance and even surviving. The mismatch in visual and vestibular cues interferes with your ability to balance, and that's why dizziness results.

Luckily, one can fool the vestibular system, much as we can fool the visual system. Techniques include "wash-out" on motion platforms, electrical stimulation, and Redirection. Wash-out is where the motion platform moves the user to simulate the virtual motion, but then sneaks her back to the center of the room at an acceleration that is below what her vestibular system can detect. The shifting tiles look like a fabulous idea, and I wonder if one could implement a form of wash-out on those tiles.

Links

• Nasa Ames VMS motion simulator
• http://www.simlabs.arc.nasa.gov/vms/motionb.html

  Sphere http://www.vr-systems.ndtilda.co.uk/sphere1.htm2-D treadmills

  Omni directional treadmill http://www.movesinstitute.org/darken/publications/ ODT-UIST97.pdf

  Torus treadmill (great video) http://intron.kz.tsukuba.ac.jp/vrlab_web/torustrea dmill/torustreadmill_e.html

  Redirection http://www.cs.unc.edu/~eve/rdw/

One more thing, the problem with, as one post suggested, implementing VR in a huge wide open space (like a desert) is tracking. The computer needs to know where your head is and in which direction you are looking, very accurately and quickly, so it can draw the virtual scene from your perspective. By accurately, I mean with millimeter precision, and by quickly I mean it must update the images within tens of milliseconds of your head moving. If you focus your eyes on your figure at arms length, then rotate your head right and left, the reflex that moves your eyes to keep them locked on your finger is called the VOR (vestibular ocular reflex). It can react to head movements in 10 milliseconds.

Here's a PDF paper about the omni-directional treadmill with neat pictures and good description of how it works, what it's like to use it, and limitations (turning in place, crouching, sidestepping and a few other movements often cause stumbles)

To echo your sentiment, I too would like to see videos of all these in action, both with experienced users and users who have never been on it before.

-Adam

What might that navigation system be?

• I can't make out the logo under the navigation display screen. A wave, followed by "ware"?
• ECDIS-N seems to be the Navy paperless navigation design. Electronic charts (maps) exist for navigation of federal vessels with ECDIS-N.
• Navy is using two navigation systems: USCG COMDAC INS and Litton Marine's (now Sperry Marine) IBS (VMS). The Swift seems to use a new IBS, thus it is using Litton's VMS.
• Sperry Marine makes Navy's ECDIS-N: In a separate effort to extend the open-architecture concept, PEO IWS in late 2003 awarded a contract to Northrop Grumman Sperry Marine in Charlottesville, Va., to act as the Navy's sole supplier of electronic charting, display, and information systems, called ECDIS-N, in a competition that rejected two Navy-developed systems. Confirmed.
• Search for ECDIS-N on Sperry Marine produces only the Military IBS page. Apparently ECDIS-N might be there. Sperry Marine products
• Apparently ECDIS-N systems must follow DII COE for the operating environment. POSIX is also involved.
• LynxOS might be involved in real time situations: LynxOS(R) was selected as the reference RTOS implementation for the DII COE configurable RT kernel.
• Thus there may be Unix influences in those designs for the navigation system.

The phrase Joint Interoperable Mission Planning and Rehearsal System is found by Google only in this document. There is a Joint En-route Mission Planning and Rehearsal System (JEMPRS), but no hints of its platform.

The COMBATSS site doesn't have much info. Another site mentions an HP Unix workstation with COMBATSS. And the COMBATSS Platform Equipment doesn't sound like a description of MS-Windows. Using Mozilla as an interface is mentioned in the original article, which doesn't reduce the possibilities much.

Better link here. Plain text, and even links to the penny arcade showing it.

Back in the late 80's, I was at the Naval Postgraduate School in Monterey, CA. The computer graphics lab was doing a lot of work in low cost 3D simulators. They had terrain data, stereoscopic imaging from aircraft overflights, etc. to make the terrain realistic. The problem was getting good 3D models of the ships, planes, missles, jeeps, trucks, etc.

Turned out, a local company called Cyberware had developed a number of 3D scanners. Our idea was to scan a toy or model from the hobby shop and use the data set as the basis for the models.

The scanners produced cylindrical data sets of 100K and up, which was too much for the SGI workstations of the time to render in real time. So I wrote software to reduce the polygon count. For example, the side of a ship or truck is relatively flat, so it can be represented by a few large polygons instead of the many small polygons from the scanner. I had a scan of Spock (from 2nd or 3rd movie) that I could reduce from ~120K polygons down to ~12K or so (IIRC).

Of course, there is much better hardware and software available now.

The Cyberware web site has more information, including sample data sets (human heads, bodies, statues, etc.). Check it out.

I don't work for Cyberware, just used their hardware for my thesis work.