Experimental Virtual Graphics Port Support For Linux

With his first accepted submission, billakay writes "A recently open-sourced experimental Linux infrastructure created by Bell Labs researchers allows 3D rendering to be performed on a GPU and displayed on other devices, including DisplayLink dongles. The system accomplishes this by essentially creating 'Virtual CRTCs', or virtual display output controllers, and allowing arbitrary devices to appear as extra ports on a graphics card." The code and instructions are at GitHub. This may also be the beginning of good news for people with MUX-less dual-GPU laptops that are currently unsupported.

Comments

Video Streams?

[WeblionX]

Sounds like we're going to be able to start passing around video in the way PulseAudio lets you connect various devices together, and make it easier to handle miniature external/secondary displays. Though the biggest benefit seems to be of an easy way to pass rendered 3D content directly to a web stream or over some remote desktop connection.

Does anyone know if this would this provide a performance boost over something like VNC for similar things? Or how about the possibility to pass rendered output as a fake video capture card input to a virtual machine? I think I get what this does, but I'm kind of wondering how exactly it's better than current solutions to these problems.

Re:Video Streams?

[justforgetme]

no,no "work" is not the word you are looking for when describing pulse audio.

I had a nightmare last night that PA was keeping ALSA captive demanding the
release of 1000000 CPU cycles the system was keeping for thread scheduling.
In the end we used an SCSI driver to nuke the damn thing to /dev/null using
an NPTL.
Unfortunately when we stormed the desolated daemon we found out the cruel
things it had been doing to ALSA all along, leaving it a mutated and deformed
carcass. /dev/rand spoke a few words about it's former beauty.
I'm telling you it was rough times.

Re:Video Streams?

[Gaygirlie]

Does anyone know if this would this provide a performance boost over something like VNC for similar things?

The slowest part of VNC and similar is the actual transmit of image data over network, and this is obviously not a new, fancy image compression algorithm or anything like. So no, it might require a teeny tiny amount less CPU time on the VNC server, but on the client end it'll have absolutely no effect.

Re:Video Streams?

[dokc]

Sometime I am somewhat impressed that Lennart never even did try to sue Ubuntu for all the badmouthing and ill-will and extra time he had to handle because of how Ubuntu did not know how to use pulseaudio, broke it in more ways than one thought possible, and then ship it in a "stable" release...

I feel pain in your words and that sounds like the pain I had after last Ubuntu upgrade. I ditched pulseaudio because of all the problems I had and now they installed it per default. I can't believe how many things they brake every time. After so many years being faithful to Ubuntu I will switch to Debian unstable to get more stable system!

Re:Video Streams?

[Gaygirlie]

On the other hand, if you have spare CPU cycles, you could take that output video and compress it to MP4, which VNC doesn't yet support. Still, far less efficient than sending the 3D commands over the wire for the device on the other end to render.

(Assuming you mean MP4 == H.264)

Real-time encoding of the stream to H.264 would be... total waste of cycles that could be used for running actually useful stuff on the server. If we assume the average VNC session is 800x600 pixels in size, high-profile in order for the picture to retain clarity (ie. small text must be readable, for example. Otherwise it'd just be absolutely useless for anything even remotely productive.), and say, 25 fps, it'd simply be impossible for old single-core systems to handle at all, and dual-core systems would still be pegging at 60%-70% CPU time used. On something like an 8-core box it would obviously not be as bad, but even then would you really want to sacrifice one core just for video compression when you're likely serving many, many other users, too?

Now, we could also drop fps to something around 10 which would likely be possible even for a single-core system to do, though it'd still likely use something around 90% CPU. But now, 10 fps stream is even less responsive than what VNC is now.

The problem is that high-profile H.264 simply does require quite a lot of CPU to do in real-time. One could in theory relegate the compression to GPU, but that would require the server to have a rather powerful GPU. And most servers don't have that. Implementing the actual code for doing the compression on GPU isn't that difficult, plenty of software do that already. Though I've only seen them doing baseline- and normal-profile, I haven't been able to find a single one that is able to do high-profile on GPU. And the quality is always a lot lower than when using software solution.

Long story short: H.264 would be a poor choice. There are better ones for this type of stuff.

Re:Need some help here

[AHuxley]

From the read me at https://github.com/ihadzic/vcrtcm-doc/blob/master/HOWTO.txt :
"In a nutshell, a GPU driver can create (almost) arbitrary number of virtual CRTCs and register them with the Direct Rendering Manager (DRM) module. These virtual CRTCs can then be attached to devices (real hardware or software modules emulating devices) that are external to the GPU. These external devices become display units for the frame buffer associated with the attached virtual CRTC. It is also possible to attach external devices to real (physical) CRTC and allow the pixels to be displayed on both the video connector of the GPU and the external device."

GPU accelerated sound?

[Adriax]

Wonder if this could be used to create a GPU accelerated sound system?
Take the scene modeling, texture objects based on their acoustic properties, create light sources for every sound source, and output the scene to a sound device that translates the visual frame into a soundscape for output.

Or am I just not up to date with audio acceleration technologies (since I've never upgraded beyond a cheap headset).

Aureal3D

[Chirs]

That's basically what the old Aureal technology did a decade ago--took the 3D scene data and passed it to the audio card for processing. It was awesome--Half-Life with four speakers was eerily realistic.

Re:Pff, nothing new

[adolf]

Indeed -- not new, at all.

Similar tricks were used a dozen or so years ago by Mesa 3D to get standalone 3dfx Voodoo cards to output accelerated OpenGL in a window on the X desktop. The 3D stuff rendered on a dedicated 3D card, and its output framebuffer was eventually displayed by a second, 2D-oriented card that actually had the monitor connected.

PA works

[Anonymous Coward]

PA works just fine as long as the one who sets it up more or less knows what he's doing. Ubuntu and most user friendly distros had packagers who didn't, hence massive problems. Of course, there are other real problems like Skype borks which mostly come from Skype using ALSA in an arguably incorrect way that when used with PA shows why directly accessing guestimated hw: stuff is a bad idea. But things people almost always complain about were caused by inept Ubuntu devs not real problems with PA.

Re:Need some help here

[prefect42]

I've used DMX with Chromium to give 3D accelerated X over 28 monitors on 7 machines. Works, but the performance can be terrible if you don't have the interconnect to deal with what you're rendering. With gigabit basic X applications could cope, but firefox with google maps would take seconds per redraw. Depending on the 3D app you /can/ get decent performance though.

Re:Pff, nothing new

[adolf]

Perhaps, depending on how hardware-agnostic the APIs in question were/are.

Then again VirtualGL has been around for a bit, too, which brings network transparency thrown into the mix. I don't know how much more hardware-agnostic such a thing could be...

In the Kernel please

[sgt scrub]

David Airlie's HotPlug video work is really cool. I'm not surprised something bigger is coming out of it. What I really like are Elija's thoughts on putting it in the kernel so support is for more than X. Below is from the DRI-Dev thread. http://lists.freedesktop.org/archives/dri-devel/2011-November/015985.html

On Thu, 3 Nov 2011, David Airlie wrote:

>
> Well the current plan I had for this was to do it in userspace, I don't think the kernel
> has any business doing it and I think for the simple USB case its fine but will fallover
> when you get to the non-trivial cases where some sort of acceleration is required to move
> pixels around. But in saying that its good you've done what something, and I'll try and spend
> some time reviewing it.
>

The reason I opted for doing this in kernel is that I wanted to confine
all the changes to a relatively small set of modules. At first this was a
pragmatic approach, because I live out of the mainstream development tree
and I didn't want to turn my life into an ethernal
merging/conflict-resolution activity.

However, a more fundamental reason for it is that I didn't want to be tied
to X. I deal with some userland applications (that unfortunately I can't
provide much detail of .... yet) that live directly on the top of libdrm.

So I set myself a goal of "full application transparency". Whatever is
thrown at me, I wanted to be able to handle without having to touch any
piece of application or library that the application relies on.

I think I have achieved this goal and really everything I tried just
worked out of the box (with an exception of two bug fixes to ATI DDX
and Xorg, that are bugs with or without my work).

-- Ilija

Re:Pff, nothing new

[zefrer]

You neglect to mention that said standalone 3D cards were physically connected to the 2D card via a pass-through cable which was what sent the video signal from one card to the other, allowing it to appear on your monitor.

This is a software solution of the same effect that will work on any card, even remote cards on different machines. Hardly the same thing.