3D Windowing System Developed Using Wayland, Oculus Rift

An anonymous reader writes Developed as part of a university master thesis is this "truly 3D" windowing system environment. The 3D desktop was developed as a Qt Wayland compositor and output to an Oculus Rift display and then controlled using a high-precision Razer mouse. Overall, it's interesting research for bringing 2D windows into a 3D workspace using Wayland and the Oculus Rift. The code is hosted as the Motorcar Compositor. A video demonstration is on YouTube.

Comments

great

[Anonymous Coward]

Now give me kinect interaction with it.

Re:great

[Anonymous Coward]

Sorry, you'll have to wait until I'm finished with mine ; )

What I predict is

[Rosco P. Coltrane]

Less productiviy.

Visually appealing stuff is nice, but the best way to use a word processor or a spreadsheet is still the good old flat desktop with no bells and whistles.

Re:What I predict is

[RyuuzakiTetsuya]

I don't think this is meant to be used in a real world environment. Not yet at least.

It's a nice proof of concept.

What I've really wanted was a virtual desktop where if i turned my head left, I got a palette or brushes or debugging inspector or something and looking forward gave me my main workspace. Looking up gives me mail or something.

I don't know if it'll work in reality, but...

Re:What I predict is

[Anonymous Coward]

Just wait until you have write your first 3D document, with bells and all!

The whole UI and the document presentation will have to be rethought to mach the possibilities and limitations of 3D UI. Windows are artifacts of a 2D UI and have little use in a 3D system.

Re:What I predict is

[deek]

I'm not sure you're entirely right. The 2D UI has evolved, in a way, as a metaphor to interactions with real world items. A 3D UI will do the same. Windows will still be needed in a 3D interface, much like how a book or notepad or projector screen is needed in our 3D reality. It's not an artifact of the older interface, but rather a natural way in which we parcel and digest information.

Re:What I predict is

[Anonymous Coward]

There is the question of ergonomics. The current VR experience lacks the input to the inner ear's vestibular system, that is to the balance organ. Then there are the neck pains from long static posture from reading and writing. Paper document in physical reality requires a surface due to gravity and other external forces. A document in 3D virtual, or augmented reality only requires a constrained distance, position and presentation in relation to the eye, which are not necessarily constant, to ease the reading and writing.
Perhaps a 3D interface could make new ways possible?

Mechanism: check. Now policy

[tepples]

Now that we have the mechanism, the next step is policy. I'd suggest rules like these: windows aren't spawned too close to another window, menus and window-modal dialogs float at x distance above the parent window, the cursor leaves a faint glow on windows, windows and the cursor leave a shadow x pixels thick on a desktop plane, a window can't be "rolled" (positioned such that the local X axis isn't parallel to the ground plane) more than momentarily, etc.

Re:Mechanism: check. Now policy

[Immerman]

Indeed. In fact while I can see value in occasionally having the ability to arbitrarily orient windows in 3-space, I suspect having them default to the surface of a head-centered sphere at roughly arms length, oriented parallel to lines of latitude, would probably be the optimal solution. Movement would default to polar translation, with the window plane automatically rotating to remain remain tangent to the sphere, with the addition of being able to adjust depth as well. There's likely not many reasons to have the window planes diverge much from tangent unless dealing with a truly massive number of windows, in which case being able to "teleport" between spheres is likely more useful (analogous to having multiple desktops)

I could see how roll could be useful though, especially when working on graphics that will be seen from multiple orientations (floor decorations for example)

Re:Mechanism: check. Now policy

[nullchar]

Working from the inside of a [virtual] sphere would be pretty sweet. Once you start sphere hopping though, you'll need a metaverse to navigate between them.

What's old is new again

[Dynedain]

Everything cycles around.

This tech existed 10-15 years ago. There were "popular" options for IRIX, and common in CAVE setups.

I attended SIGGRAPH in 2000 or so on an exhibitor pass for a company that was producing a 3D window manager to do exactly this.

Re:What's old is new again

[nurb432]

But in theory each time it cycles its less expensive, more advanced, and more available.

Re:What's old is new again

[i kan reed]

What's this? A lack of cynicism in a post?

Crucify him! Crucify him!

Re:What's old is new again

[nurb432]

It happens. Not often these days it seems, but it happens.

Re:What's old is new again

[H0p313ss]

But in theory each time it cycles its less expensive, more advanced, and more available.

And just as useless.

Re:What's old is new again

[nurb432]

I will hold of judgement until i see the real thing. However, even if it is useless this cycle, every time something comes "back around" it gets more and more useful as it is refined. Plus all the collateral improvements in other related technology that happens is nothing to discount either.

Re:What's old is new again

[Anonymous Coward]

every time something comes "back around" it gets more and more useful

Some ideas are just bad. A 3D windowing system doesn't provide anything useful for the vast majority of the population.

Re:What's old is new again

[nurb432]

That is what is said about a lot of technology in the beginning. Time and time again, that defeatist attitude is proven wrong, and those people who are short sighted, are just that and there is not basis on reality.

Re:What's old is new again

[Anonymous Coward]

In the vast majority of cases the defeatist attitude is right. I live in SV and hear about 100 of these dumb ideas everyday.

Re:What's old is new again

[exomondo]

It would be defeatist if it were being dismissed on the failings of the implementation - like low resolution, low refresh rate or inaccurate head tracking - but it isn't defeatist to point out that even if perfectly implemented this idea still doesn't provide anything useful. With the low price of monitors and projectors these days it's not particularly difficult or expensive to build a physical implementation of this where you surround yourself with displays but it still isn't useful or beneficial for a desktop. CAVE setups were fantastic for things like CAVEquake but were useless for desktop applications/processes, not for any technological reason, just that the concept itself is awkward.

Re:What's old is new again

[nurb432]

even if perfectly implemented this idea still doesn't provide anything useful

Again, i have heard that time and time again, from people with limited minds that cant see outside their own little box. If people like you had their way, we would still be living in caves "i dont see any need to go outside, its just fine in this here cave"

Idiot.

Re:What's old is new again

[exomondo]

Again, i have heard that time and time again, from people with limited minds that cant see outside their own little box.

Wrong, this whole concept and usable implementations have been around for decades and people don't use them because they aren't useful, not because there is anything wrong with the implementation just that some people can't understand that the square peg doesn't fill the round hole no matter how many times you re-implement the same square peg.

If people like you had their way, we would still be living in caves "i dont see any need to go outside, its just fine in this here cave"

I'm perfectly open to new ideas and innovation, but this isn't new or innovative, this is just flogging the dead horse.

Re:What's old is new again

[H0p313ss]

Exactly my point, if one of these UI "innovations" was actually useful it would have had some measure of adoption by now.

The absolute zero adoption of this concept in the 20+ years it's been around proves how pointless it is.

Turn your head to switch documents

[tepples]

the best way to use a word processor or a spreadsheet is still the good old flat desktop with no bells and whistles

Unless you want to quickly view more documents than your desk has monitors. This could let you have 180 degrees of documents surrounding you. A securities day trader would climax over this.

Re:Turn your head to switch documents

[fuzzyfuzzyfungus]

Does the warranty on a Bloomberg terminal cover...'fluid damage'?

Re:Turn your head to switch documents

[Anonymous Coward]

"180 degrees of low-rez, headache-inducing documents might just make that hypothetical trader of yours shoot himself in the face"

So, what you're saying is that this should be implemented immediately then?

Re:Turn your head to switch documents

[drinkypoo]

The inherent problem here is that you really need more resolution than any of the current options are offering if you're going to do Real Work(tm) in 3d.

Re:Turn your head to switch documents

[Anonymous Coward]

Well then let's work on getting him and his colleagues those monitors right away!

Re:Turn your head to switch documents

[sexconker]

the best way to use a word processor or a spreadsheet is still the good old flat desktop with no bells and whistles

Unless you want to quickly view more documents than your desk has monitors. This could let you have 180 degrees of documents surrounding you. A securities day trader would climax over this.

CTRL+TAB is faster than turning and focusing on a different monitor.
Next challenger, please.

Re:Turn your head to switch documents

[Ralph Wiggam]

History indicates that the resolution will increase rather quickly.

Re:Turn your head to switch documents

[Anonymous Coward]

https://en.wikipedia.org/wiki/Spatial_memory

This is why webOS's 1-dimensional line of cards in a stable order was a good task-switcher.

Re:Turn your head to switch documents

[Anonymous Coward]

I see your O(n) task switching and raise you O(1).
MOD+[0-9]
Tiling window managers, for people who can remember what they were doing.

Message composed in ed(I)

Re:Turn your head to switch documents

[strikethree]

CTRL+TAB is faster than turning and focusing on a different monitor.
Next challenger, please.

Not really. Yes, if you are turning 180 degrees and trying to find something, then maybe. If I am just cocking my head 30 degrees or so in any direction, my eyes and neck are considerably faster. Anything in the generally forward area is going to MUCH faster than alt/ctrl tab.

Tabbing requires me to refocus my attention and try to understand the new things being displayed to me. Moving my gaze about does not require rediscovery.

Re:Turn your head to switch documents

[fuzzywig]

"A securities day trader would climax over this."

Funny you should say that.

Re:Turn your head to switch documents

[Anonymous Coward]

What?

When I tab away, I instantly forget everything, and start over.

Developing without multiple monitors... No. I'll just program on paper them, those at least can placed spatially. (On every surface, that I can plaster it to!)

Re:Turn your head to switch documents

[tepples]

Your link led to a "not found" page. What did you intend to paste?

Re:The 90s called...

[Anonymous Coward]

Yes but the kid has to write his thesis on something, and let's face it, the gold has been mined out of EE and CS for a long time now.

Apple patent

[Anonymous Coward]

I think Apple was granted a patent on something similar to this recently (apple wins patents on 3d technology in desktop user interfaces).

http://www.cnet.com/news/apple-wins-patents-on-3d-technology-in-desktop-user-interfaces/

Re:Apple patent

[armanox]

And I think SGI might have prior art (I'd have to read Apple's patent to be sure).

So preoccupied with whether or not they could...

[Anonymous Coward]

...that they didn't stop to think if they should.

.

nauseating

[kharchenko]

I was looking forward to using rift at work, but this looks like a guy is desperately trying to type in a "deploy parachute" command into the terminal while tumbling through the air at horrible speeds.

Re:nauseating

[Anonymous Coward]

This. What this would need is some noise cancellation with regards to a persons head. Stabilization would work wonders for this.
I'd legit use this if it got tidied up. I'd use a 2D version of it in fact, not even required to be 3D.

If I could get to the stage of being able to replicate those cool glasses from Heavy Rain, I would be very happy indeed.
Who WOULDN'T want to have their office under water? Or at a beach with some waterfalls?
Certainly beats having to look at horrible offices or whatever else. And cheaper than having to make your place look good! Relationships? What's that? Can you toast it?

Re:nauseating

[Anonymous Coward]

It looks bumpy to you because your head isn't moving in exactly the same way his head was. His head was moving in exact sync with the video, so it didn't like bumpy to him. That's the point of head-tracking.

Re:nauseating

[Anonymous Coward]

This. What this would need is some noise cancellation with regards to a persons head. Stabilization would work wonders for this.

Adding stabilization would make it wobble exactly like the video.

The guy is wearing an oculus rift (a helmet on his head), the reason the camera is bouncing around is because the guy using it has a pulse and is typing on a keyboard that cause vibrations in his neck. The picture is perfectly smooth and not moving at all relative to the person using it, only third party observers perceive any movement occurring (look up Special Relativity).

BUT NETWORK TRANSPARENCY?????

[Anonymous Coward]

OMGWTFBBQ kill it with fire now!!!!!1

(just getting that one out of the way early)

Reboot

[Neruocomp]

Its the 90s all over again. This reminds me of Reboot, but worse.

Re:Reboot

[UnknownSoldier]

/sarcasm: What, you don't like the constant reboot / remake / re-imaging / regurgitation of Hollywood recycling old movies? ;)

mouse+occulus=poor combination

[Anonymous Coward]

This could be interesting with a custom mouse that added a depth control, such as trading mouse wheel click-ability for pressure sensitive depth (z-axis) motion, although one open issue would be how to hold a specific level of depth navigation without requiring constant pressure. On second thought the Razer mouse thumb controls are probably better for this.

Re:The 90s called...

[rogoshen1]

Your comment brought to mind the dopest accessory ever for the NES, as immortalized in the epic film "The Wizard". Yes, I'm referring to the Power Glove.

Those little shits won't stand a chance at Super Mario Bros. 3 now.

pointless

[Anonymous Coward]

Three dimensional UI's aren't any sort of advancement unless you can interact in 3-dimensions.
Ultimately we need to innovate in interaction beforehand. This is cart-before-horse.

No, God damn it!

[Overzeetop]

There's a reason I don't have 13 desks in my office, and a reason I have a three-wide monitor configuration. I want to see everything at once, not have to sift or "wander" through some 3D space to find what I'm looking for.

Re:No, God damn it!

[Anonymous Coward]

It occurred to me after watching the video that they are doing this wrong. Why place individual windows in the 3D space, why not whole workspaces that behave like normal screens?

You could use the single 3D viewing device to view the multiple viewing devices in the virtual world. That'd be exactly like using multiple monitors, assuming the viewing device supports head angle and position tracking, along with a sufficient field of view.

Re:No, God damn it!

[strikethree]

There's a reason I don't have 13 desks in my office, and a reason I have a three-wide monitor configuration. I want to see everything at once, not have to sift or "wander" through some 3D space to find what I'm looking for.

That is the interesting thing about the Rift. You can have 3 monitors side by side by side in virtual space. You could surround yourself in an entire sphere of giant monitor. You do not have to walk down the hallway to another virtual office to access another virtual monitor. Just because someone else likes it that way, that does not mean it has to be that way for you.

Needs a lot of work.

[MAXOMENOS]

I'm sure this technology has a lot of potential. Right now it drives me nuts just looking at it. The windows jitter too much and make me want to take my glasses off and rub my eyes 5 seconds into it.

Re:Needs a lot of work.

[Tumalu]

Most of the jittering in the video is due to the head movements of the user and is not actually noticable to the person wearing the goggles. In fact, if the jittering weren't present it would probably be a very uncomfortable experience for most people.

It's kind of like when you're looking at a physical monitor: As your head moves around slightly, the position of the monitor's projection onto your retina would also jitter, but it doesn't feel like the monitor is moving because your brain subconciously compensates for these movements based on data from your vestibular system. At the same time, if you had a camera strapped to your forehead, the position of your monitor would look jittery to someone observing the video stream unless the camera or the playback device was compensating for it.

Watching this video on YouTube is the equivilint of watching someone work at a computer from the perspective of a camera strapped to their forehead.

But does it help the user understand faster?

[tepples]

Faster is not always better for human-computer interfaces if faster misses cues that the user uses to relate one piece of information to another. Ctrl+Tab, despite being technically faster, fails to take advantage of the brain's hardwired spatial relationship processing. It also fails to fill the peripheral vision.

Re:But does it help the user understand faster?

[sexconker]

Peripheral vision for document reading? LOL!
Spatial relationship processing? LOL! LOL!

Re:But does it help the user understand faster?

[tepples]

It's faster for a machine to update a tiny 8-character display like the one on a Speak & Spell. But how is it faster for people to read from such a display?

Re:But does it help the user understand faster?

[Anonymous Coward]

Faster is not always better for human-computer interfaces if faster misses cues that the user uses to relate one piece of information to another.

If it is perceptively faster then the relationship is more easily maintained than a slow transition between 2 pieces of information, the relationship and context gets lost in the transition.

Re:But does it help the user understand faster?

[tepples]

Faster isn't helpful when the brain is thinking "Where'd it go?". Otherwise, we'd be reading through lightning-fast terminals that display one word at once.

I know this!

[ilsaloving]

It's UNIX!

Re:I know this!

[tepples]

When SGI does it, it's called fsn. But when Microsoft does the same thing, it's called Bob.

Re:I know this!

[ilsaloving]

Did Jurrassic Park come out so long ago that nobody gets the reference anymore?

I'm old! ;_;

Virtual machines

[jones_supa]

I've always wanted a silly interface for virtual machines, where I can walk in a 3D game and there's computers on the table which are showing the picture of the actual virtual machines running on my computer.

Cool.

[Anonymous Coward]

*sips coffee*

Try that with this desktop.

Missing the point

[Anonymous Coward]

I think many of the comment here are missing a major feature. On the description at the github page he mentions this very important component of his research " also by providing a mechanism for applications to request a 3D interface context in the 3D workspace in the same way that a traditional display server allows applications to request a 2D interface context in the 2D workspace". This could be a very interesting development for those working with 3d animation and modeling, as well as providing the possibility of showing information in new ways. Think of a file browser that would be a 3-dimensional tree structure, it might seem a bit outlandish now, but with all of the times that interesting 3D interfaces have been shown in film its about time someone look at defining some kind of structured interface to allow for people to explore the possibility of several 3D interfaces working together.

dumbass

[Cammi]

so a 2d object is required to navigate a "3d" environment? Someone turned stupid.

Warning: incoming game.

[tepples]

Nope, just the all-CGI cartoon that Nintendo stole the name of its 2001 game console from. "When the user loads a game, a GameCube drops on a random location in Mainframe."

Interesting

[Warbothong]

But if I had a headset strapped on, I'd rather be in an immersive world like OpenCroquet/Cobalt/Qwak[1] (which support VNC for accessing "legacy" applications) than a white space surrounded by floating rectangles.

[1] https://code.google.com/p/open... https://virtual.wf/ http://en.wikipedia.org/wiki/C...

Not "truly" 3D

[Anonymous Coward]

The glasses still have only one focus point. It would probably be better if the objects were at different focal distances, then it wouldn't even be necessary to "alt-tab" (just like when using a laptop outside, you can either focus on the picture on the screen or the reflections of the scenerey, and both appear quite clear). Anyway there's no tech to provide holographic 3D, and this is a possibly useful intermediate, but I just object to saying it's "truly 3D"