Microsoft Plans VR Simulation of Everything?

Ian Lamont writes "Microsoft recently updated ESP, a virtual reality modeling platform that until now has primarily been used to model aircraft and flight simulations. Microsoft has plans to expand it to other industries such as real estate and urban planning, but one of the most interesting possibilities could be what one observer refers to as a 'simulation of everything,' based on Virtual Earth and perhaps even user-generated content. Indeed, Microsoft's research chief has been promoting the idea of commerce applications and other tools built on top of what he calls the 'Spatial Web', a blend of 3D, video, and location-aware technologies. He gave an example of a shopkeeper creating 3D models of his store's interior and goods with Photosynth and then uploading the results into a large 3D model of local shopping district. Customers could 'visit' the area, browse products, and order them for real-world delivery."

Comments

I've never understood this sort of thing

[fiordhraoi]

Is it really easier or more desirable to "virtually browse" store shelves than to browse a web page? It seems to me to be a clunky, uninspired way to interact in a digital environment.

Re:I've never understood this sort of thing

[lxs]

Did you really think that Microsoft Bob could be killed that easily?

Re:I've never understood this sort of thing

[TheRealMindChild]

It is, but lets be honest. The point of these "virtual world" interfaces is so that an unfamiliar person can use skills they already know to use the computer. When a 65 year old lady tries to use a computer for the first time, it can be strange and overwhelming. If she was able to see and interact with the system in a paradigm that she is already knows how to deal with, the anxiety and reluctance will come way down.

Re:I've never understood this sort of thing

[Onaga]

Maybe. For people with arachnophobia, researchers were able to help them by having them virtually approach a large spider and eventually "touch" it. link

Another example I can think of is shopping for curtains or rugs. It would be nice to be able to jump into a virtual room with whatever curtains you are thinking of to see how it would look.

Re:I've never understood this sort of thing

[hey!]

Well, yeah. A lot of early attempts to make user interfaces better were based on modeling a literal desktop with things like file cabinets underneath and rolodexes and phones on top. They had limited short term success. In the long term the one that was successful was a "desktop" on which you put "windows" (stretchy ones at that) through which you viewed infinitely expandable sheets of paper. So go figure. You want computers to make things that are hard to do in the real world easy, and usually this means changing the rules.

However, one BIG application of VR that has only really begun to get developed is enhancing users' interactions with the real world. GPS units are a good example of this. You take them out into the world to see where you are. The ability to call up a Google street view on an Android phone is more than a curiosity. It tells you other things, like what street number you are at; it makes it possible to see around the corner without actually going around the corner.

I think form factor has been the limiting factor in this obvious application of VR. For example, really good heads up projection glasses would make it possible to superimpose information and models on real world objects. You could outline the parts of a copy machine and animate the process of clearing a paper jam at "C".

Re:I've never understood this sort of thing

[Anonymous Coward]

So you want to make sure the carpet matches the curtains?

Re:I've never understood this sort of thing

[fiordhraoi]

Input isn't necessarily a factor though.

Imagine, for a moment, we had true direct neural interface control of a full-3D "camera" and say a keyboard's worth of virtual "keys." (I'm using this as an example, because it is theoretically within our short-term technological capabilities, as opposed to having to make a program that understands everything we "think" and then interprets those commands).

I could use this technology to create a duplicate of the real world where I walk down an aisle and browse goods, see them on shelves, etc. Or I could use it to create a more streamlined version, where I display options and selections in a way that is impossible in physical space, allowing me to sort, select, filter, and save selections in a way I cannot in real life. Certainly, there are niche applications where being able to look at an item is valuable - someone a couple comments over mentioned curtains. But that doesn't necessarily have to be integrated into a "walk around the store" environment. Those features should appear integrated into a more streamlined process. There are quite a few websites that give you a "3D view" of products nowadays. Assuming VR technology becomes commonplace, I'm sure such things will exist in any digital marketplace.

I think the thing to remember is that simulating reality is useful in instances where we want to train people to interact with reality. VR for better flight sims, combat training, medical procedures, and so on could be incredibly valuable once we have the I/O and realism to reasonably simulate them. But shopping is an area that benefits more from shucking the trappings of reality than it does from emulating them. Why spend thirty seconds scanning the shelves for the item you want, when you can instead just input the name or category and have the available options presented to you?

Re:I've never understood this sort of thing

[BornAgainSlakr]

Not at all. Metaphors only go so far and then you start running into the limitations of the hardware used to interface with the visuals.

For instance, making a virtual store sounds good and might seem familiar to your hypothetical 65 year-old. However, this person did not grow up using a keyboard and mouse to walk through stores. So, the metaphor breaks down very quickly when you have to start implementing controls for navigation through this world. Ultimately, the experience will be more inefficient and frustrating than if the person just went to a real store.

Further, you are only modeling one aspect of the experience: walking through the store. You are not using a "walk out front door, get in car, put key in ignition, drive to store" metaphor to model how to get to that virtual store.

If this hypothetical person is able to start his/her computer, start a web browser, and navigate to the virtual store page, then it is very likely that he/she is also able to understand how to use a web page like Amazon to find products to buy. Therefore, the effort to model the virtual world is moot unless all you want is eye candy that makes the whole experience incredibly inefficient.

Re:I've never understood this sort of thing

[brunes69]

I for one would love to "superimpose models on real world objects.".

I can think of many times I would rather be looking at a model than what I am actually looking at...

Re:I've never understood this sort of thing

[jmyers]

Computers are not reality and people to not relate to them that way. New users will relate a computer to a book where you have a table of contents and an index more that they will relate to some VR interface that tries to look like a real store.

A huge benefit of shopping one line is the ability to search and find exactly what you want. Store owners hate this, they want you to browse through tons of crap and impulsively buy stuff you see.

Re:I've never understood this sort of thing

[theJML]

Or the Upholstery.... http://www.xkcd.com/508/

Re:I've never understood this sort of thing

[interploy]

One can only hope it's old ladies shopping. When I read "simulations of everything" and "user-generated content" I immediately thought of the time I first discovered "Furry Island" in Second Life. The horror...

Re:I've never understood this sort of thing

[lxs]

The one thing I liked in the video was "Onward!" instead of the boring old "Continue" on one of the buttons of the letter writing wizard. I think that would be a definite interface improvement.

Re:I've never understood this sort of thing

[hairyfeet]

You know that damned search puppy in XP and that PITA Clippy in Office? Now imagine if you had to deal with them EVERYTIME you needed to do anything on your PC. Sure it might keep the little ones entertained for awhile and some of the old folks might think it is cute but pretty soon you'd be look for the button to "kill" it just like killing that damned search puppy. One of the first things folks ask when they bring an XP machine in to me is "Can you kill that damned puppy?"

While I think VR has its uses(science and medicine,possibly CAD if done right) I just don't think the interface has gotten to the point where it can be anything more than a novelty to Joe average. I can just imagine my dad trying to shop with VR: "God Dammit! Where the hell am I! I just wanted to buy the new Clint Eastwood DVD and now I can't find a damned thing! Son come fix this stupid thing!". As it is now 3d is good for FPS and MMORPGs, but I don't see how with current interfaces it could top the ease of use of a plain old fashioned webpage.

And please don't say that "minority report" style gesture system, because not only would using that for any length of time make your arms want to fall off, but I can just picture users like my mom and dad wiping stuff out all over the place doing gestures like drunken mimes trying to get the thing to work. It really wouldn't be pretty.

Re:I've never understood this sort of thing

[lysergic.acid]

absolutely. i think this is another idea sorta like Blu-Ray and HD-DVD. basically, it's a gimmick that was likely thought up by marketing execs during a board meeting rather than a useful technology born of real innovation by engineers or developers.

you can usually tell when this is the case because the product/service will be designed primarily around serving the commercial interests of the businesses selling/offering it rather than the consumers who are supposed to purchase/use it. so with BD/HD-DVD you have a new media format that provides only marginal improvements in image/sound quality to consumers, but is crammed full of "features" (like DRM, region codes, and online ads) that are only there to serve movie studios and other content producers. HD media really just seems like it was created mainly to get people to purchase the same titles again rather than to confer any real benefit to the consumer.

likewise, a 3D shopping mall doesn't seem to serve any practical purpose to consumers. it might make sense to businesses as a way to attract consumers using a "Virtual Reality" store, but aside from the novelty (which will wear off eventually) there's really no reason for consumers to use a virtual store rather than a normal e-commerce site. this is the kind of technology we can use less of. what we can use more of are places like the old Bell Labs, where true research takes precedence over immediate profits and marketing gimmicks.

Hasn't Google already done this?

[elrous0]

Between Google's Street View and their failed Lively, it would seem like MS is once again following the old "imitate, don't innovate" philosophy here. And even Google abandoned Lively when they realized that (like VRML and its many other predecessors) it wasn't of much use in practical application.

Wow!

[Anonymous Coward]

Think of all the por... uhh, possibilities!

And the codeword for the project is...

[Lorens]

the Matrix

Re:And the codeword for the project is...

[TypoNAM]

Neo: If you're blue screened in the matrix, you die here?
Morpheus: The body cannot live without the mind

As someone who has never used Second Life

[Anonymous Coward]

I guess I'm not the target market, but this seems stupid beyond belief.

Some shopkeeper is going to use photosynth instead of simply setting up a catalog for online commerce?

People are having so much trouble shopping they have to have the real world modeled?

Things are laid out in isles and shelfs because that is a good way to use space in the physical world, not because people need to shop that way.
   

Re:As someone who has never used Second Life

[fprintf]

How many novel ideas that finally took hold were initially bandied about as "unnecessary" and a "solution in search of a problem". As with any forward thinking company, you create ideas, test market them, further develop the ones that seem to gain interest, do it some more, and if you are lucky 1 in 5 will be successful. Fortunately for Microsoft, they seem to have deeper pockets than most and can take this ideation further along, beyond the simple 'let's try it out with a few of my friends' that casual business development takes, so we end up with articles (and responses) like this suggesting the uselessness of an idea.

If I think about all the ideas that I have dismissed as "not possible", I'd be Bill Gates or Sergey Brin kind of money. Unfortunately I am not so persistent to pursue my ideas, nor creative to find just the right kind of way to implement the ideas, and I'd think most people aren't.

Anyway, it might be an idea whose time has come. Or perhaps someone else will come along with a minor tweak that makes it take off when the idea comes up yet again in a few years. I can't blame Microsoft, Google or any other company pursuing VR for trying though. It is how they are going to make money in the future I'd bet... though I won't bet because I know I won't pick the right one and don't have enough money to spread across all the different potential VR solutions. :-)

Simulations of simulations?

[kvezach]

In order to be a simulation of everything, it has to contain a simulation of itself, too; and then it has to contain a simulation of a simulation of itself... infinite descent! Only Microsoft would think they could manage that in finite space :-)

Systems managers rejoice!

[GMonkeyLouie]

Everyone else seems to be pretty skeptical of the usefulness of "Virtual World" technology, but I think it could revolutionize consulting.

I could show people competing alternatives for recommendations on how to restructure their physical operations, like "in scenario one we have your checkout lanes over here, just past the cheeses... contrast that with scenario two, where we have them flanked by bakery counters...".

Also, has anyone considered how excellent this could be for porn?

A blast from the past.

[mbone]

He gave an example of a shopkeeper creating 3D models of his store's interior and goods with Photosynth and then uploading the results into a large 3D model of local shopping district. Customers could 'visit' the area, browse products, and order them for real-world delivery."

With all due respect, this sounds very 1996. Why on Earth would anyone want to shop that way ?

Not a useful interface

[Isvara]

I thought we learned in the 90s that virtual representations of physicals things, be they stores, libraries -- whatever, are simply not the most useful way to access information. I don't want to go wandering around virtual stores to find the things I want to buy. What I want is something that lets me specify the thing I want, and tells me the cheapest place to buy it -- Google Products already does that quite well.

Re:hmm

[sveard]

Blue Syringe of Death

That's easy

[Shakrai]

How many atoms would it require to construct silicon chips to simulate every atom on the earth? In the universe?

42

I've been dreaming about this for decades

[Anonymous Coward]

I seriously can't wait to play GTA:8 - my neighborhood. Without getting in trouble this time.

Yes,

[sveard]

But does it simulate linux?

(these posts almost write themselves, so easy!)

1980s mistakes

[Tom]

Welcome to the 21st century. Right now, we are doing an "80s revival". No, that doesn't mean the clothes or music, for this one, we are reviving 1980s faults, errors and misconceptions.

Today: The misguided idea that 3D, VR or other "close to reality" interfaces are by default good interfaces. Let's ignore the past 20 years of research! Be happy! There is no uncanny valley. We don't have other options that might offer better interfaces than a simulation of reality does. No, let's assume that rocket cars, 50s music and VR are what we want.

Seriously, this is so stupid, it hurts. When I'm online I don't want to "browse". That was 20 years ago. Even "searching" is on its way out. I don't want a cheap 3D copy of your shop, I want something adapted to the medium I'm using. I want search, overviews and recommendations. I want to narrow down my view and sort according to arbitrary criteria of my own, not browse through the collection in whatever order you put it up in your shop.

It appears TFA misses completly why people do online shopping at all. Newsflash: It is very rarely because you don't want to walk or drive to the shop. In fact, I've been in a physical shop multiple times and went online there in order to research and sometimes even buy the article I was holding in my hands online. More information, price comparisons, and many more things.

gah.

[apodyopsis]

gaah. this is the sort of drivel that managers beat over and looks good in slideshows.

this smacks of first life... http://www.getafirstlife.com/

at the end of the day I can see the appeal of virtual models of real events, for example
1. police officers using stereoscopic cameras to build a very, very detailed model of the crime scene that can be explored later on or shown to a jury
2. virtual walk through of museums, natural wonders or education exhibits

but shopping and other mundane aspects of life? the obvious comment is that it will waste power, take longer and never be as satisfying as the real thing.

that said, if you can build me a holodeck then I am pretty sure I will never leave it, nothing could be as important as the simulated Monica Belluci and her simulated identical sisters.

The new twist: Photosynth

[dazedNconfuzed]

Yes, all the snide comments about VR being so '90s, been there done that didn't work, and rehashing of all the gung-ho fanboy rhetoric aside, there IS something new to this.

Photosynth.

A major problem with VR was having to construct everything manually. You want a shelf full of products? start drawing lots of polygons by hand - and that's a lot of polygons. Yes, there were some tools to help, but it still came down to a largely handcrafted virtual world - most of which turned out pretty lame.

Enter Photosynth.

Now said shopkeeper can spend 10 minutes wandering thru his store with a video camera running, take a gazillion frames of lots of angles of view, and let Photosynth stitch it all together into a fully-formed, fully-illustrated 3D model. Behold: a detailed, realistic 3D walk-thru rendering of the entire store in about an hour, mostly generated automatically.

And before anyone complains that it's slow, hard to use, etc. - it's little different from "first person shooters", which provide a familiar 3D interactive walkthru experience. Difference is, this one is the real world - without all that tedious hand-measuring hand-coding of agonizing detail of reality.

'bout time someone did this. Made sense to me long before I saw Photosynth turn pictures into 3D models, M$ just did it before I got to it (funny how deep pockets helps that...).

does not scale

[Speare]

After discussing the implementation of Virtual Reality systems with many, many non-programmers, I have come to the conclusion that the fastest way to truly grok the difference between a million and a billion is to watch a computer try to render something complicated. Because we've been living in a 2D world, and 2D graphics performance has been making steady gains over everyone's computer-using lifetime, just don't understand how problems can scale or fail to scale. Put another way, the advances in 2D have tracked with Moore's Law, but 3D is a completely different exponent. Until you really give a computer a problem that scales faster than 2D, many people just assume computers will handle any level of complexity. Watching a computer choke on something their own mind comprehends easily is the humbling moment.

Bill and Jerry...

[ethicalBob]

So THAT'S what those Microsoft ads with Gates and Jerry Seinfeld were about; A Show about Nothing in a Simulation of Everything...

No wonder they made no sense...

Re:I've just had a nightmare

[Coraon]

awww, test subject T4417-C, you were doing so well you bought into the simulation so well...ok guys time to terminate this test subject.

Is it?

[Moraelin]

Yes, but does it actually work that way? All attempts so far to make a computer have the aspect of real world, only ended up introducing more limitations and no advantages. And didn't really ease up anyone's anxiety either.

All the way back to MS Bob, which was already mentioned.

All the metaphors and interfaces that actually worked are actually the abstract ones. E.g., the mouse. It's the most useful and easily comprehended way to use a computer (I even got my 80 year old grandma using one pretty quickly), but it has no RL equivalent. E.g., the menus. They look nothing like a restaurant's menus or anything RL, but it's the one way to give commands to a computer that worked best so far.

Heck, don't even look at just the computer. Cars use a steering wheels and pedals, _not_ trying to simulate the experience of an old horse- or ox-drawn cart. Nobody had a problem adjusting to that. Radios had knobs, not trying to simulate paying the local minstrel to sing something for you. Modern telephones don't try to simulate the disk dial of old ones, nor the asking an operator to connect you to John Doe in Smallville.

If an interface is good for the device at hand, there is no need to gimp it by imitating some RL equivalent badly.

What does imitating RL bring there anyway? Let's say Amazon was organized like a book store and I wanted to find a SF book. How many books do they carry? Tens of thousands? Do you want me to walk _miles_ in a virtual store, reading the spines, until I find the one I want? What if it's larger goods, like, say, their electronics section. They take more space individually. Let's say... a mile worth of TV aisles alone?

The whole point of virtual stores is that they can carry a lot more choices -- including the stuff they don't actually have in stock at the moment, but can order for you -- than a local bricks-and-mortar store. Whereas a local bookstore would have some thousands of books, and a local computer shop might have dozens of mainboards, a virtual one can easily carry 10 to 100 times more. It's not like they pay rent by the square ft for it. Arrange that in 3D in a replica of a real shop, and you now have whole squares of kilometres for that person to virtually walk through. Why? How does it make it less intimidating to suddenly be lost in a store the size of Washington DC?

What if I don't know where their SF aisles are? Do I have to hunt down a virtual employee and ask him for directions? Then actually walk according to those directions? Or will he just beam me there, and now I'm lost and don't know which way to the cashier? What happened to just clicking on a menu?

It seems to me that the secret of Amazon was precisely that it _didn't_ try to copy RL. They tried to make it as easy and quick as possible to blow your money on something. They'll even offer some (hare brained) recommendations, so you can just click them and buy them quickly. You know, so you don't even have to do the 2-3 clicks to the section where those normally are.

Trying to make the user navigate a virtual maze of aisles seems to be a step in the exact opposite direction.