Hardware-Based Video Acceleration Coming To Linux

sammydee writes "Phoronix reports that GPU based video decoding acceleration will be implemented in Gallium3d sometime this year. Drivers currently using Gallium3d include the open source nouveau driver for NVIDIA cards and experimental Intel GMA drivers. This is definitely good news for anybody who has ever tried to play high-definition 1080p content on any CPU older than about a year."

Comments

Extremely stupid

[Bombula]

I suppose I'm both ignorant and stupid, having been out of the build-your-own-box scene for more than five years now, because whenever I stroll past the video card section at best buy I swear I read things like, "LIGHTNING FAST DVD PLAYBACK AND VIDEO DECODING!" I had no idea video decoding was still CPU dependent. Give the governor harumph, I guess.

Re:Extremely stupid

[gEvil (beta)]

Basically history is repeating itself. When DVDs first came out, MPEG-2 decoding was done on the CPU. Then video cards started having hardware supported MPEG-2 decoding. Now that we have the HD video codecs (AVC, VC-1, hires MPEG-2), we went back to square one (especially with AVC and VC-1, which require some pretty heavy lifting). In the past couple of years, video cards have added hardware support for those tasks, which otherwise would max out a fairly decent processor (~2GHz dualcore) trying to decode a high bitrate 1080p stream.

Re:Extremely stupid

[somersault]

I think the point is that it is CPU dependent if you don't have any assistance from video hardware acceleration, as is the case if your video drivers are incomplete.

Re:Extremely stupid

[LarsG]

That the hardware supports it does not mean that the driver supports it.

Re:Extremely stupid

[jonwil]

Part of the problem with hardware accelerated video decoding on Linux is that because Windows uses the accelerated video decoding to play back DRM protected media, the hardware companies cannot reveal how the video decoding part works (since it would presumably allow someone to grab the unencrypted-but-compressed video for various DRM protected video files by writing a windows driver or something)

Re:Extremely stupid

[MBGMorden]

Actually, there's an extra flip-flop step in there. When DVD's FIRST came out, hardware decoders were quite common. I remember DVD drives commonly coming with PCI hardware decoders (the RealMagic Hollywood+ was common) because at the time, many home computers simply didn't have the horsepower to pull off real-time MPEG2 decoding. My computer system when I got my first DVD drive was an AMD K6-2 450Mhz and software encoders couldn't keep up and I'd get occasional video stuttering. My hardware DVD decoder card kept up with the video just fine though.

After I upgraded to a Celeron which was overclocked to 550mhz, I was able to then switch to a software encoder (preferable because the hardware solution had very finicky drivers).

So we went from hardware, to software, back to hardware, back to software, and now back to hardware yet again :).

Re:Extremely stupid

[MilesAttacca]

And those hardware encoders are still godsends. I picked up a 500MHz Pentium III ThinkPad and a Margi PCMCIA decoder card for about $100, which gives me two hours' battery life and a 13.3" screen -- which respectively equal and thrash your average portable DVD player. When I'm not playing movies I can use the same device to check my e-mail, and when I am watching movies, I can even use a dongle to hook it up to a bigger screen and use it as my only DVD player. The downside is that I have to dual-boot because you can't find Linux Margi drivers anymore. (Can anyone hook me up?)

Re:Extremely stupid

[serviscope_minor]

Actually, there's an extra flip-flop step in there. When DVD's FIRST came out, hardware decoders were quite common. I remember DVD drives commonly coming with PCI hardware decoders (the RealMagic Hollywood+ was common) because at the time, many home computers simply didn't have the horsepower to pull off real-time MPEG2 decoding. My computer system when I got my first DVD drive was an AMD K6-2 450Mhz and software encoders couldn't keep up and I'd get occasional video stuttering. My hardware DVD decoder card kept up with the video just fine though.

There's another factor: better software. When I first started regularly watching TV and DVDs on my computer (Dual 866MHz P3), dvd playback was quite expensive and real-time encoding from the TV was right out. After a few years ffmpeg improved to such a point that playback used little CPU and real-time encoding from the TV was a reasonable proposition. The ffmpeg software codecs have been getting consistently faster and more efficient for quite a few years. These days, with improved scheduling and better software, I expect you could play DVDs smoothly on that old K6-2@450 with no problem.

Don't know what to say ...

[Swizec]

... you mean we can do all the fancy stuff windows can, and better. But playing videos efficiently was the one thing we couldn't do? We had fancy GUI effects long before windows, we had efficient RAM usage, great file systems, but we had trouble playing a fucking video?

Wow, wish I'd known.

Re:Don't know what to say ...

[Fweeky]

If it's any consolation, GPU accelerated playback on Windows doesn't work all that often, and open source codecs/players tend to be the better ones there speed and support wise, acceleration or no.

Re:Don't know what to say ...

[Anonymous Coward]

Playing a video is no problem.

The problem is that a high-bitrate 1080p video stream requires a lot of CPU time to decode, and then you have to transfer the whole uncompressed frame out to the video card.

Unless you have a really high-end CPU, there are no Windows-based video players capable of doing this with just software. They all require some kind of hardware acceleration. The specs for all this are nicely closed up, and are known only to Microsoft, video card manufacturers, and the few companies that implement video decoders (which is probably Microsoft again).

Video card manufacturers like nVidia have refused to implement any kind of video decoding in their Linux drivers until there's an appropriate spec. They won't tell anyone what they're hardware is actually capable of, so nobody else can write a spec. We can't even reference Microsoft's basic design, because it's all closed off and secret.

Oh, and most of it doesn't even work properly in Windows either. I've never managed to get it to work at all, and benchmarks I've seen seem to suggest that it's really only offloading 10% or so of the workload. Just enough to make the difference between working and not working, but not as much as it could be.

The Gallium guys are planning to implement the entire thing in their video drivers, using only the 3D capabilities that video cards are known to have. That neatly bypasses the whole thing, but required that we have a single base driver to work with (which Gallium provides), and one or more video drivers actually using it (which, again, Gallium provides).

I kind of hope that Gallium implements something similar to nVidia's CUDA - a programming model for running stuff on graphics cards that doesn't rely on graphics-related stuff like textures or polygons. That way, we'd have a way to implement different kinds of video decoders, encoders, or even things like physics simulations. Bonus points if it can be made compatible with something available on Windows...

That said... Linux video players tend to be a hell of a lot quicker than Windows video players. I've played videos in Linux that were impossible to play in Linux.

Re:Don't know what to say ...

[dotancohen]

I've played videos in Linux that were impossible to play in Linux.

You should try Linux next time.

Re:Don't know what to say ...

[nategoose]

That said... Linux video players tend to be a hell of a lot quicker than Windows video players. I've played videos in Linux that were impossible to play in Linux.

Quite an accomplishment!

Re:Don't know what to say ...

[hairyfeet]

Your post,with the allegedly,and supposed to,kinda makes me think of a good point: How are we supposed to know whether or not these things actually do what they say they do? What I mean is,tomorrow they could say this thing can decode format X,Y,Z,but when you actually TRY to find a way of getting to decode X-Z,you get this maze of "allegedly" and "supposed to" and this endless hunt through forums where we are told to buy this or that programs that "should" do it,maybe.

These things are such black box hardware that short of someone actually getting it to work through CUDA I have no way of knowing without shelling out a bunch of cash and hoping my configuration works with their voodoo that the thing actually works. They can claim it'll render every codec known to man but if the only way to do it is some special driver that they don't offer to the public,how can that count? We can test for framerate,render quality,etc,but how do we accurately test for this? All Nvidia offers is a single Mpeg 2 codec for $20. I could not find anywhere where they would sell me an .mp4 codec for the card. And why are they allowed to tout this "feature" if there is no way to actually get the feature without shelling out more money to some third party that they don't even guarantee will work?

Everyone else has to put disclaimers like "batteries not included",but while I can go to the corner store for batteries,I am supposed to...what.write my own driver? Shell out a bunch of cash playing "guess which codec might work"? It seems like a total scam to me,because they are touting a "feature" that 90% of their audience will never be able to use for lack of a freakin' driver.But as always this is my 02c,YMMV

1080p?

[willie3204]

first 1080p post?

Re:1080p?

[Kugrian]

What does this message say? It doesn't display on my computer.

Already there

[bconway]

nVidia's binary drivers and X.org's Intel drivers have had XvMC support for well over a year. I've been using both card successfully with Xine and accelerated 1080p video. I think the news here is that the nouveau project is catching up, but that's hardly clear from the article.

Re:Already there

[krgallagher]

"I think the news here is that the nouveau project is catching up, but that's hardly clear from the article."

I copied this from Wikipedia:

Gallium 3D will provide a unified API exposing standard hardware functions such as shader units found on modern hardware. Thus, 3D APIs such as OpenGL 1.x/2.x, OpenGL 3.x, OpenVG, GPGPU infrastructure or even Direct3D (as found in the Wine compatibility layer) will need only a single back-end, called state tracker, targeting Gallium 3D API. By contrast Mesa 3D requires a different backend for each hardware platform, and several other APIs need translation to OpenGL at the expense of further overhead.[1][2][3] In addition, using the modular structure of Gallium 3D, there are works underway to leverage the LLVM compiler suite and create a module to optimize shader code on the fly

I'm no expert, in fact this is a little out of my league, but I think it means that hardware drivers can expose a common implementation of system calls for 3d API's to use. It provides more standardization and allows more of the processing to be passed off to the GPU instead of handled by the CPU. If I missed the concept here, someone with better understanding please explain it to us "non-techies."

Re:Already there

[samkass]

The LLVM approach is interesting. They're basically following Apple's lead here, whose drivers use LLVM intermediate bytecode to compile shaders to either a GPU or CPU depending on hardware availability and heuristics. It basically makes it easier to support new hardware and provide relatively high-performance fallbacks in the case specific hardware capabilities are not present. All using a common architecture instead of one-off development.

Re:Already there

[hummassa]

1080p with XvMC and which codec? I thought XvMC didn't do x264, for instance.

Re:Already there

[roystgnr]

nVidia's binary drivers and X.org's Intel drivers have had XvMC support for well over a year.

I'm confused - what happened a year ago? nVidia's binary drivers have had XvMC support for their older cards for many years, whereas for the 8xxx series of cards their drivers lack XvMC support *still* (at least as of the version 173.14.05 I installed a couple months ago).

Google summer of code...

[MMC Monster]

This is apparently a google summer of code project.

While I am hopeful, let's not write this one on stone until it's released.

Not really

[JamesP]

This is definitely good news for anybody who has ever tried to play high definition 1080p content on any CPU older than about a year.

Actually, one of the most preeminent examples of HW decoding of video nowadays is the Intel Atom processor, not really old processors.

Video accel. is inside the chipset for this one.

And yes, it is available in Linux, you will probably be able to watch h264 movies in your new EEEPC

Re:Not really

[WaroDaBeast]

And yes, it is available in Linux, you will probably be able to watch h264 movies in your new EEEPC

Yay, 1080p movies on my seven inch screen! Plus I can even hook it up to my newly acquired 35:000 contrast ratio, 24p capable, 1080p fifty-two inch flat TV through a VGA connexion! HURRAY!


Relax, I'm just poking fun. ;)

Speaking of Xine

[DrYak]

Xine has supported hardware accelerated DVD video (MPEG1/2) decoding using EM8300 based cards like Sigma Design's Holywood+ and Creative's DXR3, since about 2000.
(But that was done using an ad-hoc module inside Xine, not using generic APIs like XvMC or the future Gallium3d video)

The Gallium3D Video API is a good news, because it'll probably be able to address shortcommings that XvMC has.

Re:Wow

[Anonymous Coward]

The got the headline wrong. It should have said:

Hardware Accelerated High Definition Pr0n vids Coming To Linux

Re:Release documentation

[dave420]

But they're video card manufacturers, not OS providers. It's not anti-competitive in the slightest. Sour eggs, perhaps? :)

Re:Windows games

[somersault]

The point isn't that Linux doesn't have codecs to play play hi-def content, the point is that there apparently are no Linux drivers out there that make use of the HD video acceleration hardware that is currently available.

As for having yet another API to go through, I fail to see how it would be a 'crushing blow to all gaming for Windows', since even if it did have perfect Direct3D compatibility, it would simply make it easier to port Windows games to Linux. We already have Direct3D support through WINE, and even then it isn't 100%, so what makes you think that starting again and trying to route it through another API is going to make it any better?

What would be more of a problem for Windows gaming, is if games manufacturers actually started making games for other operating systems. Even then it is not going to be a 'blow' unless this actually causes less games to be sold for Windows.

I don't mean this post in any hostile or patronising way btw, I simply am trying to point out what I think is the case. I've often wished over the years that more games were made specifically for Mac OS and Linux. Recently I just gave up and got a PS3. Consoles haven't been a very attractive option compared to PC gaming since the controls aren't very suitable for some types of game (RTS/FPS), but at least these days they can download patches and extra content. I still wish all FPS games on the PS3 would support the keyboard and mouse arrangement, but I'm getting okay with the joysticks. And at least for online play, everyone else is limited by the same type of control system. I mostly just miss the mouse when using sniper rifles now, as I know from Counter-Strike and Operation Flashpoint that I am capable of far, far better speed and precision with a mouse.

Re:XVMC?

[hummassa]

IIRC XvMC is only MPEG2 and not AVC (=x264?), VC-1, more common HD codecs.

Was there, nearly gone

[Jo Deisenhofer]

XvMC does only accelerate MPEG-2, and not many use it, because the de-interlacing you can get sucks. And for MPEG-2, you usually don't need hardware acceleration anyway.
nVidia, like ATI, are moving away from video overlays and have some pretty nice GPU-based video decoding built into their cards using the GPU (most of it probably implemented as shader programs).
But you can't use any of that with linux, you are basically down to hardware video scaling, which the newest cards no longer really support.
It is high time for a new, industry standard API for video. I hope that's what they are doing.

Re:Windows games

[WaroDaBeast]

I think you meant h.264 -- you were probably thinking about RS-232 for the number.

And yes, VLC Linux does support h.264 playback, but 1080p videos are likely to display stuttering if not GPU-accelerated: I tested it on my machine, CPU is an Athlon X2 5600+ and GPU is a Geforce 8800GTS; anything up to 720p was fine, but 1080p was unwatchable prior to installing nVidia Purevideo.


P.S.: Seems 1080i is fine too if not GPU-accelerated.

What about software?

[ThePhilips]

Last time I was trying to play HD video on my Ubuntu - with both Xine and Mplayer - I hadn't noticed that there was performance problem related to lack of HW acceleration. (I didn't tried VLC - it can't even playback smoothly HD video on Windows where such acceleration is already available.)

While CPU load was remaining low (~25% on dual core CPU), 720p video still was playing with terrible jitter. In Mplayer few minutes later A/V sync (as usually) went south. Xine started dropping frames. All that while nor CPU load, nor kernel times where displaying any anomaly.

I'd say that problem lies elsewhere and HW accel (though welcome) might not solve the video playback problems.

P.S. At least when there would be HW accel, it would be easier to bash the server/hpc/oracle folks who now monopolize completely LKML. Probably then they would start paying attention to desktop Linux needs. Quoter of the attention they spent discussion fresh Oracle benchmarks would be more than enough.

P.P.S. Tests (actually I was just trying to watch my anime on Linux) where done on AMD 4200+ X2 + nVidia gf7800gt (evil proprietary drivers are installed) + RAM 2GB DDR CL2.

Re:What about software?

[puto]

VLC works great for me on the windows boxes I use for my home media system. Plays HD contect fine.

The one in the living room is a Core2 1.8 with 1.5 of ram with an ATI 2400 HD card and no problems with HD acceleration in XP.

I do not game much, so this 59 dollar HD card has been great.

Workstation is same setup, with two gigs of ram and vista, no problems with HD video.

Server side is a linux box which runs headless. /all rooms have a pc attached to the tv /my guests can surf the web, watch movies, videos /or grab a wireless rumblepad and play any mame game of their choosing.

Re:What about software?

[PitaBred]

It may be with your filesystem, drives or possibly just NVidia. An Athlon X2 4600+ drives 1080p video for me fine, but I use an RS690 embedded chip (AMD x1250).

Re:Not 1 year

[sricetx]

If you are running Windows and using the manufacturer's drivers for a relatively recent (last few years) video card, then yeah, everything should be peachy. But if you are using open source video card drivers under Linux then good luck. Even with the proprietary Nvidia driver, highdef video playback can max out a fairly new CPU. http://www.mythtv.org/wiki/index.php/HD_Playback_Reports will give you an indication about the type of setup you need to get HD video playback to work.

PS3 and future possibilites

[tchiseen]

I'm not sure, but I've heard that a lack of hardware video acceleration is one of the factors which currently limits the capabilities of the PS3 as a linux machine (along with memory support and lack of emulators for the cpu architecture). This article gives me a bit of hope that we might see advances in the capabilities of the PS3 under Linux. ( http://ubuntuforums.org/showthread.php?t=624865 )

Re:PS3 and future possibilites

[Slashcrap]

PS3 Linux has slow video because Sony blocks access to 2D and 3D acceleration on the Nvidia chipset. This project uses 3D shaders to accelerate video decoding. So I'm struggling to see how using advanced 3D functionality that PS3 Linux doesn't have to replace basic 2D functionality that PS3 Linux doesn't have is going to help.

Re:Not 1 year

[neumayr]

Hm. My 2x2.6ghz AMD64 CPU can't do it. On Linux, with Nvidia's GPU driver.
Don't know if I've missed something, but it seems your video decoding is GPU accelerated.

Re:Not 1 year

[Hektor_Troy]

Uhm ... just read through that wiki entry a few times, and it gave me absolutely no information about the quality of the playback. Two of the comments in the "report" are usefull - the rest are quite literally useless when it comes to judging the playback ability. I.e

CPU: Intel Core2 Quad CPU Q6600 @ 2.40GHz
Speed: 2400.000 MHz
Memory total: 4050584kB
Kernel: 2.6.22-14-generic
Video codec/container: h264 / m2ts
Resolution: 1920x1080
Bit rate: 32469 kb/s
Comment: Blu-ray rip of "Casino Royale."
Media player: mplayer

Wauw. How about ... video frame rate vs playback frame rate? The above comment just tells me that the guy ripped Casino Royale. However, if the video frame rate is 29.97 fps, but only plays back at 19.87, the CPU isn't usefull for that particular application.

Re:Not 1 year

[DirkGently]

Dual-core 2.5Ghz AMD. In the same boat.

720p x264 works, though. I'm told there are MAJOR improvements in the new versions of ffmpeg that allow multi-threaded playback. I've been to chicken to upgrade my MythTV box to find out.

Gallium running out

[bugs2squash]

I though it was only a few days ago that /. reported that gallium is in short supply. Now we're blowing our precious reserves on frivolous video decoding.

Have some respect for Mother Earth...

Re:Not 1 year

[PitaBred]

MythTV uses mplayer to play the videos. So really, your solution is exactly what MythTV uses, except MythTV gives you a nice interface for a TV set.

And 1080i is about half the bandwidth of 1080p. There's a big difference between decoding the two.

Re:Not 1 year

[Anonymous Coward]

No, MythTV uses it's own built-in player (which I think is based on ffmpeg) for playing videos which definitely isn't mplayer. Now, you can set it up to use external players (such as mplayer) if you want to, but that isn't the default option.

Re:Not 1 year

[Orphaze]

You may need to improve your reading comprehension.

"Find the file that is most taxing to play while still playing smoothly and run the script on this page with it."

So, what the page allows you to do is say this file, of X resolution, of Y fps, with a bitrate of Z plays perfectly on my system $SPECS.