Slashdot Mirror

Now where exactly do we see FireFox's VP8 implementation 5 years from now?

Working fine, because it's already working fine. Open, royalty-free video will be dominant on the Web.

Does anybody believe that Mozilla will spend lots of money developing a hardware accelerated implementation for any platform and shove that into the source tree? Would they even accept such a thing if someone else developed it for them? They probably wouldn't do that either, as then they would have to maintain two VP8 codecs... So basically FireFox will never have hardware accelerated VP8, right? They wont use that nice system codec, after all.

Firefox already uses the GPU for video colour space conversion:

http://www.basschouten.com/blog1.php/2010/04/07/firefox-video-goes-up-to-11

Mozilla already spent money on the development of a hardware accelerated Theora implementation for mobile devices:

http://hacks.mozilla.org/2010/04/theora-on-n900/
http://www.schleef.org/blog/2009/11/11/theora-on-ti-c64x-dsp-and-omap3/

And of course the WebM project is looking to develop GPGPU acceleration of VP8 so Mozilla may not, in fact, need to do anything:

http://www.webmproject.org/code/roadmap/

I understand the appeal of hate, but please at least try to bring some informed hate to the discussion.

Not to mention Google and everyone else seems to be missing the gigantic elephant standing over by the potted plants: Hardware acceleration.

This is a specious argument. By this line of reasoning there would never be any change in video codecs. Let's not forget that mobile web traffic is at most 5% of all web traffic. StatCounter has it at 4.1%:

http://gs.statcounter.com/#mobile_vs_desktop-ww-monthly-200912-201012

Let's also not forget that many devices use a general purpose DSP which can be used for WebM acceleration just as easily as it is used for H.264 acceleration. Even Theora can be accelerated on such devices:

http://www.schleef.org/blog/2009/11/11/theora-on-ti-c64x-dsp-and-omap3/

Many devices are therefore software upgradable to support WebM acceleration. Combine this knowledge with the likes of Nvidia and TI introducing hardware with WebM acceleration:

http://www.youtube.com/watch?v=YcsfOMbfix8
http://www.nvidia.com/object/tegra-2.html

And the conclusion is that this is less a gigantic elephant, more a pot-bellied pig.

WebM/VP8 will force a non-accelerated CPU-only rendering path on ALL existing hardware. This eats power compared to hardware acceleration. (Look at how well most Android devices handle H.264 thanks to hardware accelerated decoding.)

My phone has a general purpose DSP which can be used to accelerate different video codecs. Accelerated WebM can be had on my phone with a software upgrade. Theora already has software available:

http://www.schleef.org/blog/2009/11/11/theora-on-ti-c64x-dsp-and-omap3/

Pretty trivial really. In many cases the capability for hardware acceleration of WebM is already there in existing devices. No hardware upgrade required.

Well, look at the whole H.264 vs. Ogg Theora debacle.

Both sides COULD just say, "we'll support any video CODEC installed on our system", which is the right answer for both open source and closed source. The simple fact is, the web browser has no more business implementing its own video CODEC than it has implementing its own graphics driver or file system. All modern OSs have multimedia subsystems which support system-provided, hardware optimized CODECs. That's all any browser should support.

But Apple and Microsoft, at least, are not doing that... they're saying "H.264 only". They aren't the whole of the market, but given their weight, it's pretty likely that many video sites will make H.264 their primary video format.

Ok, now, since the "other side" is totally and completely playing into their hands on this, it's easy to see who's affected: small companies and open source. So, Opera and Mozilla, for example, have both drawn their line in the sand and decided to do this just as wrongly, only by building in Ogg Theora as the only HTML5 CODEC they'll support. Either one could use the OS-supplied CODECs without an iota of fear for the need to license H.264 themselves, but I guess they're trying to push the market the other way.

So, in the end, we have this: the big boys all support only H.264. Open source and small companies all support only Ogg Theora. When most web sites support only H.264, we now find that small browsers and Open Source are the new iPhone -- second class citizens of the multimedia web. Mission accomplished, if you're Microsoft and Apple. They don't care about the cost of H.264, now or in the future, but they do care about competition in other areas. They want you to use their web browsers and their non-PC internet devices. Making the free stuff work less well gives them a less powerful potential competitor.

And they're protecting their small devices. Apple in particular has two motives. One is to separate video from everything else Flash does, because they understand video is both the low-hanging fruit and the largest application for Flash that most consumers care about. They have guided the discussion very well, which is why nearly every discussion about Apple vs. Adobe is about video. Apple needs to ensure H.264 because it's what iPods and iPhones have always supported. And they need to kill Flash, because they don't want people to be able to write good programs, even online Flash-style programs, which might compete with iTunes program sales. So promoting H.264 and only H.264 for gets them video support on the devices, and makes Flash less important on the net.

Today's pocket computers (call 'em PDAs or Smartphones or whatever) are all 500MHz-1GHz ARM 11 or ARM Cortex 8 devices, for the most part. And they all have very nice H.264-optimized video engines that offload that one thing form the CPU. That saves a ton of battery power on the devices, if they could even decode Ogg Theora, much less VP8, without that accelerator. Someone did put a Theora decoder on the TI OMAP3 (the SOC family in the Nokia N900, the Palm Pre and Mototola Droid), and it did a low resolution decode, but it was running mostly on the DSP, not on the H.264 acceleration hardware (which is apparently undocumented): http://www.schleef.org/blog/2009/11/11/theora-on-ti-c64x-dsp-and-omap3. But even if it worked, it would have to be done on a device-by-device basis, since each bit of video acceleration hardware is different. Lotsa work, and no guarantee it runs well on every device.

You can decode other codecs than what the hardware was intended for:
http://www.schleef.org/blog/2009/11/11/theora-on-ti-c64x-dsp-and-omap3/

I expect there are some programable components, but adding whole a new codec to existing hardware decoders may be asking a bit much.

Not really. David Schleef achieved a lot for Theora hardware acceleration without much documentation:

http://www.schleef.org/blog/2009/11/11/theora-on-ti-c64x-dsp-and-omap3/

DSP acceleration:

http://www.schleef.org/blog/2009/11/11/theora-on-ti-c64x-dsp-and-omap3/

And of course Da Vinci is even faster.

So prove me wrong. Show me mobile devices that play Theora. Show me any little bit of a trend that firmware is magically being written for this hardware.

Okay. This is an implementation (in progress, but with good results so far) for the C64x series of DSPs used in TI's OMAP series, which are used in most popular smartphones. It currently doesn't use the any of the built-in video acceleration at all, because it's undocumented, but it wouldn't be too difficult to offload some parts of the CODEC to the VPU in the OMAP given the documentation.

Most mobile devices don't actually come with a hardware H.264 implementation, they come with hardware implementations of the most processor-intensive steps in H.264. A lot of these steps are also used in other video CODECs.

Right, because a software update can create a hardware Theora decoder *rolls his eyes*.

You must think you made a good argument... The reality is that the decoding doesn't happen in hardware, processor intensive parts of the decoding are accelerated in hardware, said operations are common across many codecs. Experimental hardware accelerated decoding for Theora has been done with existing hardware. It's not a roll-eyes-at problem, it's a manpower problem.

A lot of people just don't get the problem and constraints on mobile devices here - it isn't just that the codec is technically inferior, meaning that it requires more processing and decoding than h.264, it is the availability of hardware decoders.

Bullshit, you are just talking out of your ass. The often cited technical inferiority of Theora has nothing to do with higher processing requirements, its the lower complexity that hinders the encoding quality. Theora is considerably lighter then H.264, where, even with hardware acceleration, you have to fuck with low complexity profiles.

For most equipment, this isn't possible, it would require a new revision of the hardware, and all previous revisions wouldn't have access to the Theora content.

So which devices have H.264 decoding done in dedicated chips as opposed to using DSPs for acceleration?

>AIUI, my n900 has a DSP on the SOC which is used for MPEG4 stuff, but could just as well be used to accelerate other codecs

see also: http://www.schleef.org/blog/2009/11/11/theora-on-ti-c64x-dsp-and-omap3/

I agree but defining or limiting it to one codec is a fail.

Web browsers support a variety of open image formats. There's no particular reason why they can't support a variety of open video formats. Theora is ready for use now, Dirac will be ready in the future. If Google releases VP8 as an open format (as many expect they will) then at some point there will be at least three open video formats to choose from.

Theora I doubt will ever get hardware support. I could be wrong but I don't see it. Google's codec could if they created their own chip and made it part of Android but odds are they will not do that because they will have to support it on YouTube or look like fools if they don't. YouTube has already spent how much money support H.264 for the iPhone?

Accelerated playback of Theora is already available:

http://www.schleef.org/blog/2009/11/11/theora-on-ti-c64x-dsp-and-omap3/

Millions of devices already have the capability to accelerate Theora playback. They just need the software. Fennec (Firefox for mobile) supports Theora playback on the Nokia N810 and N900:

http://www.mozilla.com/en-US/mobile/

And Fennec is coming to Android:

http://arstechnica.com/open-source/reviews/2010/02/hands-on-and-under-the-hood-ars-tests-firefox-on-android.ars

Mozilla will happily support VP8 in Firefox if Google releases it as an open format.

Tnen you have Microsoft. They have not said they will support the video tag at all! My guess is that fantasize that SilverLight will be the new standard.

IE is always a problem. Fortunately you can work around it with Cortado, a Java based Theora player:

http://www.theora.org/cortado/

Or with a Silverlight based Theora player:

http://arstechnica.com/open-source/news/2010/02/nuanti-brings-html5-and-ogg-theora-video-to-silverlight.ars

Neither is ideal, but for the time being it's the best you can do in IE. I think IE will support HTML5 video eventually. It's more a question of "when" than "if".

The two issues that prevented YouTube from using the Ogg Theora codec still apply.

Many hardware devices already have H.264 decoding built into the chip, ranging from set-top boxes to the iPhone. Moving away would mean losing ability to run on these target devices (or run at an unacceptable frame rate).

The argument that there's no hardware support for Ogg Theora or any other open video format doesn't hold much water. In fact, many devices use general purpose DSPs to accelerate H.264 playback. You can make use of the same DSP to accelerate Theora playback. That's what's been done here:

http://www.schleef.org/blog/2009/11/11/theora-on-ti-c64x-dsp-and-omap3/

Millions of devices are capable of accelerated Theora playback today. All they need is the software.

Or maybe this one: http://www.schleef.org/blog/2009/11/11/theora-on-ti-c64x-dsp-and-omap3/ (your link failed for me).

Yeah, they're using the DSP, and that's something. But the H.264 acceleration is more than just DSP code on these TI parts. There are hardware units used to accelerate various parts of the video decode pipeline. This is clear from his demo: despite Theora being a lighter weight decode, he's limited to a somewhat jerkey 640×360/24p, which he pronounced "good enough". Perhaps on an iPod, but if you're using a DROID or another modern smartphone, you're going to want the full 848x480/30p... which plays perfectly on mine, from AVC.

The problem, of course, is supporting this all over the place. All modern phones have hardware acceleration for H.264, but they get it in different ways. The OMAP 3430 is probably your best bet for Theora acceleration, because of the DSP just sitting there. On the iPhone, they're presumably using the PowerVR H.264 acceleration hardware, which is much less flexible... maybe it maps to the needs of Theora (certainly, H.264, Theora, WMV, and MPEG1/2 are all DCT-based compression schemes), maybe not. But is Mozilla really taking on every platform? Won't they have the same problem with low-levels docs on these as they did on the OMAP?

Your links got chopped, the right posts are pretty easy to find right now, but just in case:

Theora on TI C64x+ DSP and OMAP3
This is really cool, and I hadn't seen it before

HTML5 video and H.264 – what history tells us and why we’re standing with the web

Yes, there's no reason you couldn't hardware decode Ogg/Theora on a mobile device, but all the existing deivices don't have the capability to do so. So if html5 adpot Theora as the standard, all existing smartphone users would be left out in the cold.

This is not the case. Many smartphones use a DSP to accelerate H.264 decoding. You can use the same DSP for accelerated Theora decoding:

http://www.schleef.org/blog/2009/11/11/theora-on-ti-c64x-dsp-and-omap3/

Millions of devices have the capability to accelerate Theora playback. All they need is the software.

Mozilla, for good reasons (IMHO), is not willing to support H.264, but that seems to be the direction YouTube is heading. But as good and open as Theora is, I think don't believe there is any hardware with a Theora accelerator (yet?).

You can make use of the DSP that's used for H.264 acceleration and use it for Theora acceleration or any other similar workload. That's what's been done here:

http://www.schleef.org/blog/20...-c64x-dsp-and-omap3/

As mentioned in the post, that work is broadly applicable to Nokia's N series of phones, the Motorola Droid, and the Palm Pre. There are millions of devices in the field today which are capable of accelerated Theora playback. All they need is the software.

See also Christopher Blizzard's post on the importance of open formats to the future of the web:

http://www.0xdeadbeef.com/webl...anding-with-the-web/

In the comments Christopher Montgomery from Xiph.org, the foundation behind Theora, says:

"As for the chicken/egg problem of hardware support, several big commercial groups are already scrambling to get over it, partly because full Theora support in hardware is so much simpler than full h264 support. It’s a tiny fraction of the complexity. You practically get that many transistors for free in the today’s average cardboard cereal box. Can’t say more– NDAs. But that’s OK, it will be reality or not soon enough."

As you say, Microsoft's lack of HTML5 support will probably be a problem for some time. Fortunately, it can be worked around with Cortado or Highgate media suite's Theora for Silverlight

There are at least two free software implementations of flash, one LGPL (http://www.schleef.org/swfdec/) and one GPL (http://swift-tools.net/Flash/).

The nice thing about reverse-engineering is that only one person has to do it: http://www.schleef.org/swfdec/

What kind of help? And more importantly, how are they making something compatable? Reverse engineering?

What the fuck? Flash format is open, why reverse engineering? Flash format is just as open as, say, PDF. If nobody in the opensource community has not written a opensource flash viewer is OUR fault not theirs.

Mind you, I've been viewing some simple flash files through gstreamer thanks to swfdec. Why all this noise now?

I wonder what their motivations are for working on this instead of helping out the gstreamer guys on swfdec. swfdec is licensed under the GPL and largely already works, including its Mozilla plugin. Even on non-x86 platforms.

http://www.schleef.org/swfdec/

I believe the GStreamer people are looking into using liboil. The license is two-clause BSD.

Another project trying to do something similar is liboil, the Library of Optimised Inner Loops.

However in the future I can see things changing for the structure of the stardard PC.

At the moment in a high end machine you have the CPU, which is a scalar processor, a GPU, which is in essence a glorified vector processor (not just useful for graphics, as projects like GpGPU are showing us), and SIMD extensions to the CPU to allow it to do small amounts of vector processing.

Scalar processors are good for some things (branchy code) and vector processors are good for other things (very predictable parallel code). Having both is very useful.

I would say in the next 5-10 years we will see the GPU join together with the SIMD extensions to provide a seperate general purpose vector processor.

PCs will ship with two processors - one scalar, one vector. And everyone will be happy.

Now, whether this will be transparent to the programmer depends on how automatic code optimisation progresses over the next few years. Is Intel's icc auto vectorisation already good enough? Don't know.

For thoses who want to see how programming is done in GNUstep, there's this short flash demo here

GNUstep is a free software implementation of the OpenStep API (like Cocoa), and it provides development tools as well. The demo steve do is doable in GNUstep as well..

(Yes, it's flash... a mpeg version will probably be available next week... in the meantime, it's a good idea to check either swift tools or swfdec , if you don't want or can't use the Macromedia Flash player..)