Slashdot Mirror


The Dismounted Soldier Problem

Pilchie asks: "When reading this recent article I started thinking about a problem I learned about when working for Canada's DCIEM in the simulation training technology sector. Most of you are probably familiar with some type of simulator or virtual reality. Have you ever noticed that every VR game includes some kind of vehicle? The reason is that no one has been able to develop a system to accurately simulate walking, without actually going anywhere. Before you laugh at this, think about it. How would you build a User Input system that allows a person to walk in an arbitrary direction at an arbitrary time?" Interesting thought. Anyone have potential solutions? (More below)

Pilchie continues:

"Some Ideas
These are some of the ideas that I have heard, and some of their drawbacks:

Don't walk. Use some other input system (joystick, whatever) to move the character. Problem: People get lost if they don't actually walk. See here.

2D mesh of bearings. This is probably the best idea so far, the idea is to have a 2d mesh of bearing that can rotate in any direction, each with a sensor to determine which direction it is rotating in. The drawback is the difficulty in sensing the rotation of the beads, as well as allowing them to move in any direction. Plus safety(things getting caught between the beads).

Spheres. This idea involves placing a person either inside or outside of a spere and letting the whole sphere rotate in an arbitrary direction (sort of like a mouse ball). The problem is making a sphere big enough, with little enough friction that it can actually be moved by a normal walking action.

Mime Walking. This involves teaching people to walk in a special manner to allow the system to know they are walking. See this again. Problem is that it is fake.

Large room/visual tricks. Put someone in a large room and let them walk where they want. When they start to approach a wall, rotate the picture enough that they compensate, but not enough that they realize it is rotating. Problem: obiously you can't be sure they won't hit a wall at some point.

Well what are your ideas on the issue?"

59 of 271 comments (clear)

  1. mesh of bearings by ToiletDuk · · Score: 3
    I think the approach about a mesh of bearings that can rotate in all directions is a very good one. As for tracking the movement, it could be done with optical sensors. It would probably get slightly pricy having one optical sensor for each bearing, but it would probably be effective.

    Hypothetically, it would work like the logitech "marble" trackballs. The balls would be speckled, with large contrast between the ball color and the speckle color. Optical sensors would then track the movement of the balls, and I would assume, average out the direction and the speed of the movement among all the balls.

    One problem would be after-spin. When you spin a trackball, it will continue to spin for a second or two after you let go, depending on how fast you would spin them. Maybe there would be more friction imposed on the bearing mesh. It's an interesting problem.

    1. Re:mesh of bearings by isaac_akira · · Score: 2

      One problem would be after-spin. When you spin a trackball, it will continue to spin for a second or two after you let go, depending on how fast you would spin them.

      You could solve this by having the weight of your body press the balls down, away from a friction brake. When they have weight on them, they roll easily, but when the weight is removed, they lock in place.

      And to save money you could use an optical tracking system that tracked multiple balls at once (like a video camera pointed at 8 or 16 or more balls at once).

      - Isaac =)

    2. Re:mesh of bearings by Gray · · Score: 2

      I misread that initially as 'mesh ON bearing' which lead me to an idea I will now attempt to work though in a slashdot message.. woo..

      Okay, you make a resonably sized floor of metal balls held in bearing packs. You should be able to stand on it and with some balancing or a hard support, walk normally. The friction break idea would help.. So, that gets you fairly free walking in 2d, but tracking that is still the problem. Enter the mesh..

      Surround the bearing 'plate' with a strong flexable bag. So between your feet and the balls there is a thin layer of bag material.. I think that would result in a omnidirectional treadmill wouldn't it? Then print the bag some big dark spots and use camera watching the underside to track them.. The balls themselves are totally passive.. Walking on a ball method without the curved floor or having to have a huge ball in your office..

      Problems that come to mind..

      The bag/cover would have to be made of something strong, flexable and with a fair amount elasticisty, but I'm sure some substance exists..

      The energy needed to move the bag around the plate would make each step require more force then normal.. Clever design could hopefully minimize this..

      Seems cheap and uncomplex to me.. I'm sure someone has though of this before, but just in case, if anyone builds one, I want a ride (gross points would be nice too)..

    3. Re:mesh of bearings by joekool · · Score: 2

      stop thinking of the bearings as a way to track movement--think of them instead as a way to keep the user in one place
      use some kind of boot or sensor to determine what the foot is doing--this allows significantly more interaction with the environment, as movement in three dimensions can be measured--in other words you can kick something, or march, or whatever, and it will be reflected in the simulation.
      maybe I am just missing something, but it seems like you would like to model all of the bodies movement within the simulation---I keep thinking of the sword fighting in SnowCrash

      --

      Slackware: old school feel, new school gear.
  2. Re:hmmm by rde · · Score: 2

    The problem with a treadmill is that it's only good for moving forward in back; for full movement in 2d you'd need either a sophisticated treadmill that could move along two axes or a separate control that rotates the environment around the user, so that the can continuously face one direction.
    The problem with the former is that I can't envision how it'd work (others may be more perspicacious), and the latter takes away from total immersion.
    3d movement, once 2d is taken care of, is simple as long as no-one wants to climb a virtual Everest.

  3. Direct Neural Interface? by Hobbex · · Score: 3


    What ever happened to using a neural signal inhibitor to pick up the walking signals in the spinal collumn nerve cells and translating them into computer instructions?

    We have pretty good beta of it going here at [echelon system autocensoring error 431 - no replacement known for this forbidden term] based on stuff the aliens bought left us. And it seems you can use it for a full 2 weeks before you loose your physical motoring skills completely!

    The device also has great application within torture and teledildonics (what can I say, we don't get that many woman here at [echelon system autocensoring error 431 - no replacement known for this forbidden term]).

    -
    We cannot reason ourselves out of our basic irrationality. All we can do is learn the art of being irrational in a reasonable way.

  4. The VR games I've seen... by moonboy · · Score: 2

    ...have a "ring" that you stand within (so you don't lose your balance and fall, I presume.) This could also be used as a positional system, while the user wears a belt, which would contain a small transmitter where the "buckle" would be. The "ring" could have sensors within it to provide the system with coordinates for the direction of the user. Then the user could basically walk or run in place and turn as they would in the "real world." It wouldn't be the best simulator without some sort of variable speed treadmill that would be responsive to changes in speed by the user and would allow the user to rotate on an axis. This would be the real engineering feat.

    ----------------

    "Great spirits have always encountered violent opposition from mediocre minds." - Albert Einstein

    --

    Co-founder and designer at Music Nearby: http://musicnearby.com
    1. Re:The VR games I've seen... by Pilchie · · Score: 2

      Actually you can get self contained position and directional tracking systems. They work by emitting EM fields in three perpendicular directions, then a sensor measures the angles between them and a reference field to determine Euler angles. I am not sure how the position part works, but I assume it has to do with intensity of the emitted fields. And these systems only cost about $1000 US.
      >~~~~~~~~~~~~~~~~

      --
      >~~~~~~~~~~~~~~~~
      Pilchie
  5. You Don't by Jonas+�berg · · Score: 2

    Actually, this problem occurs in other scenarios too, such as if the subject is supposed to climb a ladder, climb a rock, crawl under a fence or jump off a high building. The solutions is rather obvious; you don't simulate this. Atleast not using mechanical devices. Can you imagine how complex a machine would be that would allow the subject to do any two of the previous mentioned actions? I expect the game industry will try to do something similar to this anyway because it would heighten the experience and earn them more money.
    The real problem, however, will remain, and what you probably should do is hook up electrodes to your brain to pick up movement and simulate feelings and sensations. I admit that there's probably quite a few years until we can do that (which is why the game industry will try other things first), but I think that's where we're ultimately heading.

  6. SQUID Sensors by Crutcher · · Score: 5

    There has been a lot of development in the squid field over the last few years (squids are a type of minute voltage detector), and while it takes a little training (ususaly about 10/15 minutes of practice to get navigable, which is much less than the time it took most of us to learn the nintindo controler) sqid based controlers have been under development for a few years. The idea is you only have to THINK about moving a muscle, and its voltage changes, but it dosn't actually move until you reach a treshold value. With this tech, there are all kinds of fun ways to solve the walking problem. Personaly, I would put squids in an input glove, and have it respond to *pushes*.
    -Crutcher

    --

    -- Crutcher --
    #include <disclaimer.h>
  7. Two robot arms by Francisco · · Score: 4

    How about this: two robot arms, with shoes where the grabber would normally be. Person stands in the shoes. The arms would controlled so that they move freely when the feet are above "ground", and lock up when they hit "ground level".

    This would have the advantage of also working on stairs or gradients, or even wading through mud.

    I have seen something similar in arcade games simulating skiing. These only had one degree of freedom for each foot though. For this to work you would need six.

    Of course, things could get unpleasant if the machine crashed. Want to risk being a human wishbone anyone? :-)

  8. Dont let the feet touch the ground.. by jdigital · · Score: 2

    The problem with walking, is that when people start doing that crazy leg thing, they actually get somewhere.. If that somewhere does not correspond to whats on the VR display, then you have your problem. Solution : Dont let the feet of the walker touch the ground, ie. suspend the 'user' in a harness and attach their legs to an exoskeleton, whilst the 'user' is suspended 30cm (1 foot) off the ground. The exoskeleton serves 3 purposes : 1) Senses leg movements to feed into the VR unit0r. 2) The device can be mechanically active, so that you can have some sort of feedback from the machine. For example, walking 'up-hill' will require more work that going down a hill. 3) Its not every day that you can use the word exoskeleton in meaningful conversation. Josh

    --
    :wq ~ ~ ~ ~ ~
    1. Re:Dont let the feet touch the ground.. by Pilchie · · Score: 2

      My personal opinion is that this will be the second real solution developed, since it allows better 3D stuff. I think that first someone will come up with something like the solution proposed further down the page by Gray, and then a few years later something like this. Designing the device to be non-intrusive would be very difficult though. This reminds me of one of the fist solutions that I thought of which was to suspend the user on some kind of paddles which (s)he moves with feet. Anyway, the whole submission was just to generate some speculation, so I think it is doing well.
      >~~~~~~~~~~~~~~~~

      --
      >~~~~~~~~~~~~~~~~
      Pilchie
  9. Dream it! The ultimate VR sim by Ripat · · Score: 3

    Ok. This might be *slightly* off topic, but If it's a great VR experience you are after you shuld try lucid dreaming (basicly beeing aware that you are dreaming, when you are dreaming, and then do whatever you like in the dream).


    No problem with walking, eating, flying, doing magic etc...


    Well it can be a bit hard to learn of course, but it's well worth it

    1. Re:Dream it! The ultimate VR sim by kcarnold · · Score: 2

      You forgot one: blowing up the world! I've done that.

      Of course, there is the problem with realisticly simulating going to the bathroom... oops!

      cya, Ken

    2. Re:Dream it! The ultimate VR sim by PurpleBob · · Score: 2

      That kinda reminds me of the first time I played Stunt Race FX. It was the first truly 3D game I had ever played. (As in, both you and the camera could move and tilt in 3 dimensions, which ruled out Doom, Wolfenstein, etc. which only had 2D motion, and Star Fox where the camera hypothetically could move like that, but didn't.)

      Anyway, that night when I went to bed, I could still see race tracks turning, dipping, and zooming by while I had my eyes closed. The same thing happened in my dreams that night.
      --

      --
      Win dain a lotica, en vai tu ri silota
    3. Re:Dream it! The ultimate VR sim by PurpleBob · · Score: 2

      Wow, I never got to those last stages. I had nightmares as well, and I learned to recognize them (made easier by the fact that almost all of them were about the grandfather clock in the hallway turning into a monster). I noticed that if I lied down somewhere in the dream, it would end.

      As I gained more power over my dreams, the dreams changed accordingly... I became able to confront the monster, until in one dream I tricked the grandfather clock into eating itself. I stopped having those dreams after that, and when I looked at the clock in real life it wasn't scary anymore. I've dreamed very infrequently, and remembered even less, since then.

      Now I kind of wish that hadn't happened, after hearing you describe all the cool stuff you can do in your dreams. Oh well.
      --

      --
      Win dain a lotica, en vai tu ri silota
  10. A few suggestions by Martin+Ling · · Score: 3

    There is one core problem here: how to maintain walking motion in a restricted space.

    • A simple treadmill cannot suffice, as it can only work in one direction.
      • but perhaps the user could rotate their torso to turn? - have a sensor which picks up torso rotations (not head, mind you - that's for looking) and translates them to turning. Note that this would be a matter of turning right while the torso was twisted right and stopping when centered again.
      • ...but a giant trackball could work. Has to be relatively easy to turn by walking motion, and not have too much carry-on spin (perhaps this could be handled by some intelligent braking system). Note that an important addition to this system would be a sensor to pick up the position of the torso. There's a reason for this - it means the user can walk backwards, diagonally, sidestep, etc. It's easy to not realise exactly how important such issues are to aid realism and navigability. We do not just walk forward, turn, walk again. All good Quake players use the sidestep keys continually, and all good Mechwarrior players understand the implications of not having them (and often use jumpjets for the purpose :)

    • Fake walking allows the maximum freedom, but is after all fake. However, consider the following:

      User is suspended in a harness, legs dangling, arms free. This can be comfy if done properly. Manipulation controls, gloves etc can be used for arms. On each foot is a position sensor. A certain line just above the maximum reach of the users feet is designated as 'ground'. He moves his feet in normal walking motions, and the positions of the feet in relation to the ground plane are interpreted:
      • Feet move one by one under the ground and backwards = walking/running (speed determinable).
      • One foot faster than other or backwards = turning.
      • User rapidly pushes both feet below ground = jump.
      • User pulls both feet above ground = crouch.

    Anyway - just some ideas. I'm very interested in such issues, so go ahead and email me.

    1. Re:A few suggestions by kcarnold · · Score: 2

      Those are interesting suggestions, but the harness idea (mentioned elsewhere as well) has the problem of the feeling of opposing force (also mentioned elsewhere). If your legs are dangling, it's going to feel like your legs are dangling, not like they are touching the ground at every step. It is going to feel almost the same as fake walking. To increase the complexity (but since some outrageously complex ideas have been proposed, it seems that complexity doesn't matter so long as it works) and realism, why not have some line actually be the ground? And to maintain the ground level in a crouch or in a landing after a jump, the harness should be held up by movable wires that could change the vertical position. In general the wires would be relaxed so that the used could "walk" on the "ground", but as the user left the middle of the device, the wires would gently pull him/her back to the middle. Going along with that is that the "ground" be made up of a bunch of movable rod-like things that could approximate uneven terrain by moving up and down. The ground would be covered by stretched plastic to even it out. Then when the user was wandering away from the middle of the device and the wires were pulling him/her back, the rods could tilt slightly back toward the middle so that the natural thing to do would be to go back a little bit. If the plastic was just slippery enough, the pulling back could be accomplished without the user knowing. Of course the whole system would have to be intellegent to allow for the initial acceleration forces felt when beginning to move or stopping. This could be accomplished by delaying the pull-back as much as possible in the necessary circumstances. This seems really complicated, but most everything proposed is, and with today's fast processors, the complexity can be dealt with. This solution also better accounts for response time because most of the actual movement such as jumping, crouching, or walking is left to physics (which is the fastest processor I've ever seen), and the things the computer has to worry about is the compensation, most of which is repetitive (pulling back mostly the same amount while walking) and thus can be optimized with local, less powerful processors. The only really hard things are dealing with sudden motions, such as when you stop running. However, the delayed response time would approximate inertia in this case, adding to the realism.

      The only problems I can see with this approach are common to most every approach. First, it would be a pain in the butt to get into because of all the wires and sensors and stuff, so it probably wouldn't work for a public video game arcade kind of situation, and it wouldn't be able to deal with bending over very well. Having a harness could also allow for (but not accurately simulate, nothing proposed can) jumping off of a high ledge in a game where such a thing can't really hurt you (like a lot of them). It would also allow for the simulation of some other thing moving them around, like hitting a wall could cause an abrupt force backwards, and you could approximate the up-and-down forces of, say, a ski lift. It would be enormously flexable.

      But it would be (also like most other things) prohibitively expensive.

      Come to think about it, this sort of resembles how Star Trek NG did it on the holodecks, execpt it was a force field that pulled them back towards the middle of the simulator instead of a harness and cables. They called it a treadmill force field or something like that, except that it couldn't have really shared that much with a treadmill because treadmills only go in one direction.

      Anyway, why bother in the first place? VR is always going to be virtual, and that means that there will have to be some restrictions. Besides, vehicles are more fun anyway; you get to go faster.

      Kenneth

      PS - What's wrong with the real thing? And what were you doing on Slashdot at 6:42 in the morning? I was sleeping :-|____.

  11. Problems by CPol · · Score: 2

    One major problems with mesh (and any other system that lets the person actualy walk) is that you can't compensate for the lack of moving mass easily. The person doing the walking would still compensate for moving his body, so that he'd lean into the movement. This is basical the same as running at a threadmill, if you don't learn to run without correcting your balance you'll fall off. And if you manage to avoid shifting your weight you'll learn a 'wrong' movement pattern, thus making practical application of any skills you get pretty difficult.

    One solution to this is to fake acceleration by letting people wear a vest and connect it to wires placed above and around the person. The wires then pull on the vest in a manner that lets the person lean into the movement. The drawback is that the system doesn't react fast enough to follow small changes, and the wires severely hinder arm movement. It also leaves the problem of inner ear unsolved, as you will never 'feel' the movement.

    --
    Phase 1: Where do you want to go today? Phase 2: This is where you want to go today. Phase 3: You're not going any
  12. Problem of acceleration by rat · · Score: 5

    As I see it the general problem here is the inner-ear based sense of acceleration. We feel it when walking, riding an elevator, falling, turning a corner in a car, etc. When the visual sense of place and the inner-ear sense of acceleration are in conflict, most people feel disoriented, sometimes to the point of nausea or vomiting.
    In constant-speed walking, the acceleration perceived is largely up-down, except during the first and last steps. Perhaps an appartus with a wide range of movement could duplicate both the up-down acceleration (a simple matter of "bobbing") and the initial and final accelerations, each of which would be followed by a gradual (enough not to be noticed) acceleration in the opposite direction, to ensure an eventual return to center. My gut feeling estimate is that to be "believable" a machine of this design would have to have a diameter of about ten meters. Big and expensive, but perhaps doable.

    rat

    1. Re:Problem of acceleration by the+hopthrisC · · Score: 2

      But you can actually get used to situations with different dynamics. Just think about moving staircases! Last time I tried to use a broken one (it did not move, but the lights were green - I was tired, so I didn't recognize it at first) I almost fell to m knees, beacause I automatically leaned against the acceleration which was - of cousre - not there.

  13. Fool the sense of balance by PG13 · · Score: 2

    The most promising solution to this problem appears to be the large room with some sort of "fooling" effect to make the user always slightly turn away from the wall. It may sound farfetched but consider most of our "realistic" simulations come from the fact that the human brain relies on a couple little tricks to understand the outside world not on very good sesory input (for example use of two slightly differnt images to simulate 3d when in fact the objects are not so far away and could be detected via a focusing mechanism).

    What about some sort of mechanism to slightly fool the inner ear? This would cause the user to (unconciously hopefully) list to one side keeping them inside a fixed area...if we combined this with a mildly sloping floor it might be used to good effect. The difficulty hear is of course to fool the inner ear without having to stick needles in their or anything.

    Has anyone done any research into what happens when the image presented to the user is slightly tilted from the angle expected by the tilt of their head? Do they walk differntly?

    A sort of simple proof of concept of this idea is to put yourself in a large room close your eyes and try to walk straight across...extremly difficult to do! Imagine if you were being actively fooled!

    --
    Marriage is the "pseudo-ethics" that cloaks the messy truth of sexuality in the raiment of propriety -- it's "Don't Ask,
  14. Re:dont laugh by plunge · · Score: 5

    Pretty funny, but not totally far from the best solution. We're actually reasonably close to being able to decifer nerve impulses, so controling walking via a tap into the spine isn't such a pipe dream. But there are some serious drawbacks regardless. 1) Pattern generators. Walking is a standard "program" the nervous system can "run", meaning it CAN do so without sensory input, and we can measure it. The problem is, PGs also use various sensory input to do "error correction," which means we could get all sorts of addative errors and confusing screwups. 2) Without actually walking, how are we going to make it "feel" like walking? Reproducing sensory feelings is way harder than measuring standardized spine output. It would feel extremely uncomfortable to "walk" without the proprioceptive feeling of walking. 3) Related problem- how are we going to shut down the muscles without shutting down the proprioceptive nerves in the muscles, which would also exacerbate the problems of 1 and 2

  15. Inner ear sense of acceleration by Pilchie · · Score: 3

    I decided to respond to this at the top level because I have seen numerous people state things about it, and I wanted to address all of you. Contrary to popular beleif, the inner ear problem is not much of a problem at all. If you provide an immersive enough visual environment, the human brain will compensate for not receiving the acceleration inputs for moderate accelerations, such as those involved in walking, etc. This is one of the things that the group I worked with researched. In most applications, true acceleration inputs are only required to differentiate self motion from external motion. As an example of this, in the helicopter simulator I worked on, a motion platform was required for the pilot to be able to determine which motions were of the helicopter, and which of the ship. However, for someone standing on the deck of the ship, not watching another moving object, there is no need for a motion platform. Yes it makes it more realistic, but it is sufficient for training purposes to provide the visual stimulation. In fact most of the time when we were testing we didn't have the ship moving, and therefore didn't put the motion platform on, because it was unnecessary.
    >~~~~~~~~~~~~~~~~

    --
    >~~~~~~~~~~~~~~~~
    Pilchie
  16. Some related work by Gene+McCulley · · Score: 4

    I work in the modeling and simulation domain. One team in our local organization is working on Individual Combatants (the more general term for what we used to call Dismounted Infantry). They participated in some exercises to test and demonstrate new IC technologies. One of the coolest things I saw was an omni-directional treadmill. It was really cool. It even had some mechanical devices attached to the user's torso that could simulate the forces involved in going up or down a hill. You can find more about it here.

    1. Re:Some related work by abramsh · · Score: 3

      The omni-directional treadmill was used in some experiments a few years back by the NPSNET research group. Here is a link to the paper published at UIST'97.

  17. Re:SQUID Sensors - does your brain like it? by Misanthrop · · Score: 2

    hmm, nice idea, but what if you spend hours and hours of walking but are not walking. just remember those cases of people losing the ability to write because they used this strange input mechanism of the PDAs. the brain is very sensitive and fragile. i am not sure if i would do such a thing.

    alex

  18. The Jimeroqui' Video by Money__ · · Score: 2
    One idea comes from a video I saw on MTV. The video shows a person walking on a moving floor. This was done my going into a warehouse with a big floor, building a room with no floor (4 walls and a roof) light enough to push around by hand. The camera was attached to the moving room, so that the camera perspective was locked with that of the room, and did not appear to move.

    Ok, now change the room to a 1/2 sphere (dome) that the user is inside of. One small camera/sensor, above the user in the sphere, looks down at the user and sends signals to 4 wheels on the outside of the 1/2 sphere. When the user walks right, the dome moves with him. The user can now walk normaly all over the warhouse, and the sphere covers him at all times.

    The dome (acually made of a rear-projection material) would be covered with DLP projectors (Like people use in buisness presentations). Each projector would be responsible for a small area of the total picture (1280x1027). The projected image would be proccessed digitaly before being projected, so that the final result aprears flat to the user inside the dome.

    The noise made my the servo motors moving the dome would be cancled by the 3D sound system positioned around the outside of the dome and noise canceling technology.

    1. Re:The Jimeroqui' Video by Pilchie · · Score: 2

      The problem with this is that it is essentially the same as the large room, visual tricks slolution above. What do you do when the half sphere reaches the edge of the warehouse. Note that a very similar (but much simpler and cheaper) solution is to just use an immersive head mounted display and self contained gyroscopic tracking system.
      >~~~~~~~~~~~~~~~~

      --
      >~~~~~~~~~~~~~~~~
      Pilchie
  19. Forget the Input Device! Track the Feet! by fixion · · Score: 4

    Pilchie--

    You're approaching it from the wrong direction. "Think outside the box", dude.

    Don't build a user input device. Model the leg's motion directly!

    Here's what I see you trying to do now: through a "natural" walking process, you want to have the walker's feet create a change in some input device (much as a hand creates a change in a mouse or keyboard or joystic). Then you measure the change in the input device and convert that to a model of motion.

    But you're sticking in an extra (and probably unnecessary) layer of abstraction: Information about legs' movement CONVERTED TO Information about input device movement CONVERTED TO virtual model of legs' movement.

    Input devices makes sense in modeling vehicular movement. For example, in modeling driving a car, you don't really care about the motion of the hand itself; you care about the motion of the steering wheel. (The fact that a human hand is guiding the steering wheel is irrelevant. From the standpoint of modeling the vehicular motion, it could just as easily be a monkey or a computer moving the steering wheel for all you care.)

    A steering wheel becomes the input device to control a virtual vehicle, because in the real world it ALREADY IS the input device used to control the vehicle.

    Input devices don't make sense in modeling movement of the human body. In modeling walking, you care about the motion of the feet -- their direction, the length of the gait, etc.

    In the real world, what is the input device that is used to control the motion of the feet? THE LEGS!!!

    Try this: Information about legs' movement CONVERTED TO virtual model of legs' movement.

    Gets rid of the input device.

    My idea: Instead of trying to measure the rotation of hundreds of spinning bearings, measure the motion of the legs in relation to a fixed point. (Actually, all you probably really need to know is the location of the footfall, so you probably only need to measure a couple points on the foot in relation to that fixed point.)

    How? The simplest example I can think of off the top of my head is kinesthetic analysis of athletes, like for golf swing analysis. The golfer and the club are covered with a series of white dots, typically at each joint, and is digitally filmed swinging the club. A computer analyzes the digital video, recognizes the dots, and uses them to construct a wire frame figure in a virtual space that can then be analyzed. It's used not just in golf, but in many sports as well. It's also used in dance!! I've heard of several dance projects out there that are attempting to use the human body itself as the interface to control or model a virtual dancer.

    If you want to model the motion of a jet fighter, who do you go talk to? An aeronautical engineer! Why? Because that's the person who is going to know most about how a jet fighter moves and how that motion is controlled.

    If you want to model the motion of a human body, who do you go talk to? Either an athlete or a dancer (or someone who studies athletes or dancers)! Why? Because they're the people who are going to know the most about how a human body moves and how that motion is controlled.

    So maybe you have dots on their feet and a camera trained on their feet, a computer watching the dots, crunching the numbers to model the motion of the feet, and passing their location onto the VR modeling system. Maybe you have a super accurate GPS system (or just some kind of very localized version of a positioning system) that sends back the specific location of the feet to the VR modeling system. If you know the how the feet's movement is changing in relation to a fixed point, then you know the direction of the walk, the speed of the gait, etc.

    You still need to decide how the subject (the real subject, not the VR avatar) interacts with the real environment while immersed. Probably the *easiest* way is to give the subject plenty of room to maneuver. Put them in an airplane hanger with a VR headset on. Let them move around to their heart's content in the hanger, model the motion of their feet (and hands! and head direction!) and build the virtual world around them as appropriate, based on that motion. It would look funny from the outside -- a couple of soldiers in headsets wandering around a hanger. But from inside the VR they might be in downtown Beirut or wherever.

    With that plan, you *will* run into physical limits -- it would be possible for them to bump into the hanger wall. If you don't need an infinite virtual space, then big deal. If you do . . . eh, you might be able to do the "large room/visual tricks" option to make them change direction. Howver, that's probably too complex, I think.

    If you need an infinite VR space, you might need to put them on that 2D mesh of ball bearings, so they can "walk without going anyplace." (You just have to make sure there's enough ball bearing friction that the subject doesn't fall on his ass!)

    You don't need to measure the motion of any of the ball bearings. Just ignore the ball bearings. The ball bearings are just the foot's medium of motion. If you want to model head motion or hand motion, you don't try to track the displacement of all the air molecules around the head or hand! No, you track the motion of the head and hand itself in relation to a fixed point. Then why try to track the displacement of the "ground" (aka ball bearings) beneath the feet!!?? If you know how the feet are moving in relation to a fixed point, then you can use that information to model the virtual motion.

    TRACK THE FEET, DUDE!

    --
    fixion
    fixion@yahoo.com


    P.S. Two minutes searching the Web netted me these links:

    Peak Performance, Inc. (http://www.peakperform.com/) has a product called Mocap that captures 2D & 3D motion coordinates with real-time optical sensors. Found via a search for sports technology and biomechanics.

    VNSIII (http://www.interlog.com/~drokeby/vnsII.html) allows you to respond and analyze motion information captured in real-time. Found via a search for "dance and technology."

    and the kicker:

    Whole Body Kinesthetic Displays (http://www.cybernet.com/rnd/contracts/contractbri efsp59.html) Cybernet, Inc., under contract for the U.S. Army, has developed a "foot-haptic" (i.e. tactilely responsive to the feet) system for modeling locomotion in VR. Patent pending. Found via a search on "kinesthetic analysis".

    Translation: US Army beat ya Canadians to it, dude!

    And that's just, like, the first three promising ones I came across!




    1. Re:Forget the Input Device! Track the Feet! by Pilchie · · Score: 2

      Yes I agree that it would probably be easier (and more accurate) to track the feet. The whole thing about the bearings, etc, is to solve the infinite virtual space problem, otherwise you could do the large room thing. I guess the goal wasn't stated clearly enough. You want to provide a way of walking which feels natural AND allows infinite virtual space.
      >~~~~~~~~~~~~~~~~

      --
      >~~~~~~~~~~~~~~~~
      Pilchie
  20. Hmmm by Signal+11 · · Score: 2

    Sphere idea - friction may not be an issue. Simply put something along the floor that's piezo-electric (generates electricity when pressure is applied) in small segments, and then use hydraulics to move it. You can coat it with a thin layer of something to prevent people from damaging it as well, such as teflon.
    --

  21. Re:Why ? by oneiros27 · · Score: 2

    For common activities, I can the benefits of this, however, there are a few tasks where it may not be in the best interest to actually practice something of this sort...

    (I mean, if you can get virtual walking-- who's to say we can't do virtual rock climbing?)

    For the most part, I would think that this would have to be limited to practicing dangerous activities that aren't dependant upon dynmaics due to the human body -- I mean, the speed at which you rotate in gymnastics/diving/ski jumping/etc is based on the size of each limb in an individual's body, and it's relative position to your center of mass.

    Of course, as with anything else, you'd have to decide if the cost to develop the program would offset the risks associated with performing the actions in real life.

    Otherwise, you'd have to have it developed by someone who knows they'll be able to recoup their money within an acceptable period of time, which would most likely be the entertainment industry.
    (Although, I'm guessing that any 'interactive movies' would have to be more like Dragon's Lair than anything else, which wouldn't really depend on this topic, as you're forced to take one of a limited choice of paths)

    --
    Build it, and they will come^Hplain.
  22. Looking at it from the wrong angle. by The_Morlock · · Score: 2

    Maybe we're all looking for too difficult a solution. What I was thinking is this:

    the user is suspended an inch or two off the ground by an immobile waist or hip harness (the user doesn't change direction. the VR world rotates around him. like in doom, quake, etc) with positional sensors on shoulders, elbows, hands, head, knees for directional orientation of the various body parts. I mean the kind of sensors they use to input dance steps, martial arts motions.

    the feet would be in a sort of stirrup which is connected to positional sensors AS WELL AS a force feedback system. this would be used to translate the user's natural walking motions into directional input, and the force feedback would simulate the additional force required to climb, etc.

    Of course, this has the drawback of the vertical orientation problem (you would always know that your body is vertical, even if you "fall" in the sim, unless we find a way to monkey around with your inner-ear), but as far as arbitrary movement in a 2d-with-incline world or climbing rocks and ladders, it should solve the problem nicely. or at least adequately. There still would be no sense of inertia.

    Nice setup for a Mortal Kombat type game, right? hehe :) Or add a gun and you have doom.

    What do you guys think?
    I guess I'll have to break the news
    that I've got no mind to lose...
    I guess I'm gonna have to tell 'em
    That I've got no cerebellum

    --
    So you say life sucks? Well, life is what you make of it.
    so if your life sucks, YOU suck.
    -----BR
    1. Re:Looking at it from the wrong angle. by Eric+S.+Smith · · Score: 2
      the feet would be in a sort of stirrup which is connected to positional sensors AS WELL AS a force feedback system.

      This is exactly what I was thinking. If flight simulators can mount a cabin on movable struts, why not just attach the struts right to the arms and legs of your player?

      I'm thinking more of robotic arms than hydraulic pistons, of course. The person stands in boots that are attached to the end of two robotic arms, and then slips his/her arms into something similar (gotta simulate gun recoil and crawling...). Maybe have a fifth arm on a tight backpack that the players wear, just so you can mess with (and read motions of, not to mention support) the torso, and then maybe mount the whole thing on powered gimbals, so you can have 'em do a simulated faceplant, hanging there three feet off the floor, should the simulation demand it.

      So, now you've basically strapped a person into this multi-armed contraption, with their various extremities individually supported and controlled. "All" that's left is to work out the force-feedback system so that they can walk realistically -- as the foot "hits the ground," the arm stops moving. As the foot is raised, the system freely allows that movement (possibly assisting it, to hide the fact that there's this giant steel arm strapped to your foot). As you walk forward, the arm lets you slide your foot back on a flat plane, to simulate the ground moving beneath your foot. Irregularities in the surface can be simulated by changing the height of the "ground" and angle at which the foot "rests". You can do stairs; mud can be simulated with "mushy" feedback.

      Some problems:

      • acceleration, as others have pointed out. Maybe tilting the body on the gimbals would be reasonably convincing, but it'd never truly fool the player.
      • getting the feedback to work right.
        • the programming will be tricky, though maybe motion capture would help
        • getting the robot arms to work without lag will be another doozy
    2. Re:Looking at it from the wrong angle. by ken_i_m · · Score: 2
      With all those arms attached to the player it could get ugly if one of them moves the wrong way.

      "Please do not spindle or mutilate."

      ken_i_m

  23. Re:use the ring thing and add a trackball platform by moonboy · · Score: 2

    Yeah and if the surface of the "track ball" were large enough, you wouldn't notice the slope very much at all. It would seem as though you were walking on "flat ground." Good idea.

    ----------------

    "Great spirits have always encountered violent opposition from mediocre minds." - Albert Einstein

    --

    Co-founder and designer at Music Nearby: http://musicnearby.com
  24. Re:hmmm by Eccles · · Score: 2

    The Walkthrough project at UNC-Chapel Hill uses (used? It's been a few years) a treadmill with bicycle-style handlebars to steer. It's certainly not a perfect simulation, but the primary goal of the project is to give people a reasonable sense of virtual spaces. The treadmill is much more effective for this than (for example) joysticks or other hand-manipulated input devices. One thing that makes life a little easier in this goal is that the brain will often imagine some of the sensory phenomenon even when they aren't there. Georgia Tech has used VR simulations to help people overcome fear of heights, despite problems with parallax (even with headmounted displays, lag means that the visual feedback is out of sync with the head motion) and the absence of focal depth.

    --
    Ooh, a sarcasm detector. Oh, that's a real useful invention.
  25. Motion sickness? by DragonHawk · · Score: 2

    Contrary to popular beleif, the inner ear problem is not much of a problem at all. If you provide an immersive enough visual environment, the human brain will compensate for not receiving the acceleration inputs for moderate accelerations, such as those involved in walking, etc.

    Very interesting. Would that mean that prolonged use might cause motion sickness? That's what causes motion sickness, after all - different sensory inputs not agreeing with each other. The brain gets confused, and you feel sick.

    --

    dragonhawk@iname.microsoft.com
    I do not like Microsoft. Remove them from my email address.
    1. Re:Motion sickness? by Pilchie · · Score: 2

      I never had a problem being on the ship which was moving maybe 6 meters up and down visually, but not all physically, although I never was in it for more than about 20 mins. I suppose it could, but I have been told that the primary cause of "motion" sickness in simulators is the refresh rate, and lag of the display. The mind compensates for not receiving the motion cues, but it has more problems if you turn your head and the visuals don't change at the same time. This is a total guess, but I think that the longer you were "under the hood", the better your brain would get at compensating, and the less you would notice the lack of motion cues. But I could be waaaay off.
      >~~~~~~~~~~~~~~~~

      --
      >~~~~~~~~~~~~~~~~
      Pilchie
  26. Re:hmmm by Pilchie · · Score: 2

    That walkthrough project sounds like a good idea, simulate what you can, and provide a cheap solution for the rest. And I agree that joysticks, etc, don't give a good perception of virtual space.
    >~~~~~~~~~~~~~~~~

    --
    >~~~~~~~~~~~~~~~~
    Pilchie
  27. Re:hmmm by Pilchie · · Score: 2

    I beleive the 2D treadmill was the original idea behind the bearing mesh solution, so that the bearings are free to turn on either axis.
    >~~~~~~~~~~~~~~~~

    --
    >~~~~~~~~~~~~~~~~
    Pilchie
  28. Hybrid bearings/ringed/tensor solution... by drama · · Score: 2

    I immediately thought "ball bearings" when I saw this. Bearings mounted on mouse-like x,y motion detectors would be ideal - as a start.

    After this, it would be nice to simulate slipping or even difficulty in running. Therefore the rollers used to detect motion in the bearings could have some motors attached to them to provide graduated tension to the ball bearings (climbing up an icy landscape vs. climbing up a rocky hill).

    Now, we need to tackle 3D issues. First of all, the structure will likely need to be larger than 5 feet in diameter to provide a decent "feel" for realism - taking into account the length of stride for a long person (or someone with unusually long legs). Then, the entire bed of bearings (BoB) should be rotatable to 55 degrees (at least) for mountainous climbing simulations, etc...

    Taking this into account, how do we simulate steps, curbs, etc? Well, we could seperate the ball bearings into seperate rings around the player. the rings could be raised individually (or set to rotate individually) using foot high pillars on hydraulics designed to move up and down, and sense the weight (and respond stiffly - as in conrete steps; or softly - as in a muddy marsh) to the person's weight.

    What's left? player positioning without using wires. Well, they're now selling little gyroscopic options for N64 (?) controllers to allow you to play without having to use the control sticks or pads. So rotation isn't an issue... Magnets could do the rest for positioning. A few devices like this attached to key points on the body will allow you to tell exactly where the various body parts are (one on each elbow, knee, each end of the waist, etc).

    $300k for one of these things maybe?

    Next week: I solve world hunger.

  29. Re:hmmm by Strango · · Score: 2

    Get a tread mill that can detect the amount of pressure being placed on it and in how many spots. If it finds 2 spots (2 feet) then the user is most likley standing still. If only 1 foot than you know the person is walking. You can use time to determine the person's stride. I'm not sure about climbing, but you could still do jumping as long as you told the user to just jump straight up. When the treadmill detects 0 pressure being placed on it have it go into jump mode. At the last time it detected have it store the horizontal velocity and use this as the velocity while no pressure is placed on the pad. As for the height of the jump, i'm not really sure since you would have to know how the long the jump would be overall. For arbituary direction maybe you can have the user wear a certain type of shoe so that the sensors on the treadmill can determine which direction the user is facing and adjust within the game.

  30. Mesh + sensors by sjames · · Score: 2

    How a mesh and harness simply to allow the feet to move and keep the person from falling. The actual movement is computed based on LEDs on their shoes + a switch in the soles to signal when they are actually applying pressure.

  31. Baby Walker by bjk4 · · Score: 3

    Does anyone remember the baby-walkers?

    Perhaps you could have the person supported on an item that could be dragged back to the center of the room without their knowledge. That way they can walk in any direction and never hit a wall. When they come near a real wall, the user is visually prompted to raise up his/her feet so that the mechanism can return to the base.

    There are a few disadvantages though, like having to build the thing to hide the acceleration back to base. Also, the lifting of the feet could be troublesome.

    -B

  32. here's my $0.02 by leddhead · · Score: 2

    what if the user is standing on two hydraulic platforms(one foot on each platform) feet buckled to the platforms, kinda like skis. and as the user walks a computer regulates the tension in the hydraulic pumps below the platforms to simulate lets say walking up a hill. the platforms could also slope towards a particular direction, so that the user has to lean forward; exactly what you do when you are walking up a hill. the next refinement could be platforms that bend; to simulate small changes in terrain, like putting your foot on a small rock etc..

    --
    Writing a new OS only for the 386 in 1991 gets you your second F for this term. - Prof. A.S. Tanenbaum, author of Minix,
  33. nausea by TheCarp · · Score: 2

    Well yea it would feel weird walking without
    feeling like walking"

    The brain would get confused fast. It is giving
    signal to walk, it sees the motion of walking
    but...the inner ear isn't detecting motion.

    When the inner ear and brain loose sync...the
    natural reaction is to begin "Stomac pump
    procedue".

    I had an interesting discussion about this with
    someone onece. It apears that this may be an
    evolutionary "Defence mechanism". Loss of
    proper signal from the inner ear can often be
    caused by certain chemicals, say the chemicals
    in an amniata muscaria mushroom (big red caps)
    or something in the nightshade fammily (other
    than tomatoes)

    Basically the idea goes that the brain stops
    getting the right signals from the inner ear
    so as a defence it induces vomitting so that if
    this was caused by some poison, it may stop
    absorbtion of the rest of it.
    (which is why nausea is very common with alot of
    drugs)

    anyway...I would think that ALOT of people would
    react very badly to any such setup.

    --
    "I opened my eyes, and everything went dark again"
  34. I have... by Graymalkin · · Score: 2

    thought on this subject before and have come up with some of the same answers as in the question. One idea that might work is to have a base with an array of IR motion capture sensors like on the new Microsoft mouse with the funky red lights on the bottom. The user would have a harness that suspended them above the base so they could walk on it without moving, the user's shoes having a real smooth padding to let them slide on the base; the harness would be ables to move up and down (crouching and jumping) but not laterally. The base would capture the movements of the person's feet and translate that information into speed and direction. The base could also have IR sensors to find the position of key parts of their body and the position of the barrel of their rifle. This is rather ill suited for the home but could work very well for tactical simulates for police and military.

    --
    I'm a loner Dottie, a Rebel.
  35. Damn... by Greyfox · · Score: 2

    Why not just build some of those rotating thingies from the Lawnmower Man? I've actually seen those for sale outside the context of the movie. Then it just becomes a matter of providing some tactile feedback when appropriate.

    --

    I'm trying to teach myself to set people on fire with my mind... Is it hot in here?

    1. Re:Damn... by coaxial · · Score: 2

      Why not just build some of those rotating thingies from the Lawnmower Man?

      Because they don't do anything. Your static while the the "sphere" rotates you along two axes simultaneously. Since your rotating in two directions you don't get disoriented. (I know I went to Space Camp. The idea is just to ride it, not to stop it like they say in that sucky movie.)

  36. Re:hmmm by lovelace · · Score: 2

    Actually, the UNC Walkthrough group hasn't used that for several years. Right now we have what is probably the only "wide area tracker" in existence. It uses cameras to reference know spots on the ceiling (in our case, infra-red LEDs) and is accurate to 0.2mm with less than 1ms of latency. You can find more information about it here. Or, you can read about an interesting experiment we did with it last year where we studied what works better: flying (as with a joystick), walking in place (to simulate walking), or real walking.

  37. Gyroscopes by Luke+B.+Bishop · · Score: 2
    A totally different idea. Concentric positional gyroscopes with the person held in the center. They never actually have their feet on the actual ground, but they wear a tactile/force feedback suit while suspended. The gyros can rotate to simulate the effects of moving-mass and gravity. The force-feedback can give the impression of actually walking. The positional readback sensors can determine the actual change in position vectors.

    I work with the software for VR systems. So far, this idea seems to have the most promise, although I have not yet personally had a chance to work with it. But short of full neural interface, this is as close to real as we will get.

    --
    -- For large values of one, one equals two, for small values of two.
  38. The best solution is to combine stilts and wires by JoeShmoe · · Score: 2

    I have often played with this idea in my mind and have figured out that the most realistic solutions is as follows:

    Have the person wear a suit that is suspended by wires. The wires would both provide lift (to keep the person upright) and register joint motion (bending and turning).

    The foot would be placed in a cradle that has a stilt attached to both the ball and heel of the foot. The tips of the stilts would have ball and socket connections to allow for free rotation. The stilts will be connected by cross-braces to allow for tilt and pitch.

    Servos would either apply resistance to provide lift and simulate contact. When the program calculated the foot had hit ground, the servo would apply enough force to prevent the foot from moving any lower. When the person lifted his or her foot (detected by weight sensors in the foot cradle) the serve would feed enough force to counter the weight of the device.

    With enough tuning, this system could be configured to feel just like a pair of combat boots. It would also have an extra advantage: terrain. You can't simulate changes in terrain with bearing meshes and spheres. This means you've gone from a situation where you can only simulate combat vehicles to a situation where you can only simulate building combat.

    With the stilt system, each foot have have separate heights. Even if it were pitch black, the user could feel his way around with just his feet. If the user were at the edge of a cliff, he would put his foot in front of him and find that the servos give no support whatsoever.

    Anyway, I doubt this would be practical enough that Logitech could come out with a home version, but in cases where money is no object, it is very doable. Still, I think it is rather obvious. If it isn't already patented well...oops. I guess now it will be. =P

    - JoeShmoe

    -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= -=-=-=-=-=-=-=-

    --
    -- I wonder which will go down in history as the bigger failure: the War on Drugs or the War on Filesharing
  39. Unnecesary Corrollary by Crutcher · · Score: 2

    Okay, my suggestion with the sensors was for the creation of a *NEW* task, which is not that difficult. This *NEW* task would not compete with prior tasks (as in walking) at all, and would develop it's own nerve patterns.

    If you are skeptical of what I just said, try this experiment for 1 month: Once a day, for 5 minutes (while wating for the TV commercials to end) *PUSH* up through the center of your palm. Nothing will happen that is visible to you, but the nerve association/muscle feedback will build this activity up as a task in your mind, and you will be able to actualy feel the difference between doing and not doing this task. That difference is detectable, and that is how squid inputs are designed.

    So imagine an input system that allowed you to direct your movement by pointing in an arbitrary direction, with speed/direction rendered analog by a squid glove. (THink about the way you move in your dreams) Or even using a cranial ring (try the above experiment with pushing forward/back/left/right with the top of your head.

    Talk about intuitive, but it does take about 15 minutes before you can get good input from such a system (the first time) but the long term (a day or two) finnese reachable beats hell out all other input devices.

    -Crutcher
    -Crutcher

    --

    -- Crutcher --
    #include <disclaimer.h>
  40. Re:Radial Moving Sidewalk by PurpleBob · · Score: 2

    Tricky doesn't even begin to describe it... a radial treadmill would have even more problems than the omnidirectional treadmill described elsewhere. The material would have to stretch to surround the entire surface, and yet compress into a hole in the center small enough that you can't feel it under your feet, with no noticeable differences in texture or friction. I'd think this would be close to impossible with known materials.
    --

    --
    Win dain a lotica, en vai tu ri silota
  41. Re:SQUID Sensors - does your brain like it? by dillon_rinker · · Score: 2

    just remember those cases of people losing the ability to write because they used this strange input mechanism of the PDAs

    I know exactly what you mean - there was an article on slashdot that talked about it. Perhaps you missed the bit that explained that it was satire. NO ONE actually lost their ability to write. The article's author made it up. It was humor.

    OK, maybe you're being funny, too, but I just got to work on Monday morning and am still humor-impaired...

  42. Re:slippery semispherical arena by plunge · · Score: 2

    The ultimate solution, of course, is to develop a truly frictionless surface. That way, whenever you tried to walk on it, you'd immmediately fall flat on your ass and slide right off the frictionless platform. That would gradually teach people to just learn to stay put!