Hardware-Accelerated Ogg Theora For Firefox Mobile

An anonymous reader writes "Matthew Gregan is working on bringing David Schleef's DSP accelerated port of Theora to Firefox Mobile. He writes on his blog: 'The C64x+ DSP is often found in systems built upon TI's OMAP3 SoC, such as the Palm Pre, Motorola Droid, and Nokia N900. Last year, Mozilla funded a port, named Leonora, of Xiph's Theora video codec to the TI C64x+ DSP. David Schleef conducted the port impressively quickly and published his results. The intention of this project was to provide a high-quality set of royalty-free media codecs for a common mobile computing platform. The initial focus is Firefox Mobile on the N900, so I am working on integrating David's work into Firefox. To experiment with other facilities Firefox could use to accelerate video playback, and test integration, I've been hacking on a branch of a stand-alone Ogg Theora and Vorbis player originally written by Chris Double called plogg.'"

Comments

Hardware Accelerated?

[Anonymous Coward]

In my mind, hardware accelerated means using fixed-function, special purpose functional blocks in silicon. Saying that using a DSP is hardware acceleration for video codecs is like saying that software is hardware accelerated by the execution units of a CPU.

Re:Hardware Accelerated?

[X0563511]

Your mind is wrong.

Lets look at graphics cards. A DSP is the same "thing" as a GPU is, in that it's a special purpose processing unit (ie, Graphics Processing Unit, Digital Signal Processor...). Rendering with a GPU is a hell of a lot faster than with a general CPU. Likewise, processing digital signals is a hell of a lot faster in a DSP than with a general CPU.

So, to reiterate: you are wrong.

Re:Hardware Accelerated?

[chill]

FYI: NEON is ARM and AltiVec is PPC. They're the other processor equivalents to SSE.

Re:Hardware Accelerated?

[O'Nazareth]

DSP is hardware acceleration for signal processing (this is the name). A video frame is a 2D signal. So yes, this is hardware acceleration.

Battery life?

[TheRaven64]

It's great seeing the benchmarks showing the CPU usage dropping from 99% to 1%, but at the same time the DSP and GPU usage is going up by some unstated amount. It would be really great to see some comparison of how this effects the battery life. Playing MP3s on the C64x can be done in a bit under 15mW, but Theora is a lot more complex and doing the colourspace conversions and compositing on the GPU is going to add a bit too. I'd expect the power usage to be lower than doing it on the CPU, but maybe not by a huge amount.

Either way, it's great that they can free up the CPU to do other stuff. 0.4% of the ARM core isn't really enough to run the scripts that typically accompany a web page that uses the video tag, 99% almost certainly is.

Re:Battery life?

[X0563511]

Looking at the datasheets for a few TI C64 DSPs, we are looking at supply voltages under 2v and current draws measured in microamps (yes, micro, smaller than milli).

The power draw for these special-purpose devices is teeeny tiny compared to the other hardware.

http://www.ti.com/lit/gpn/tms320c6424 (PDF) - Page 127 lists electrical characteristics in the normal operating temperature range.

More models listed here:
http://focus.ti.com/paramsearch/docs/parametricsearch.tsp?family=dsp&sectionId=2&tabId=217&familyId=477

Re:Give it up, Mozilla :)

[TheRaven64]

Theora isn't bad. It's better than MPEG-1 and a MPEG-2, and similar quality to MPEG-4 part 2. It's not as good as H.264, VC-1, or Dirac, but that's not the same as being bad. It's perfectly acceptable for a lot of uses.

Re:Give it up, Mozilla :)

[imroy]

I'll admit that Theora isn't the greatest video codec. But it's been explained many times that Mozilla simply can't use MPEG-4 AVC/h.264 because of the patents involved.

Mozilla and wider Open Source world has three options: (not mutually exclusive)

• Work to kill software patents so they can use AVC/h.264 and other codecs that come after it.
• Make Theora better.
• Make something better than Theora, but free of patents.

But no amount of whinging will make them use AVC/h.264. It's simply not an option at the moment.

Open systems drive innovation

[human spam filter]

This is a good example how open systems drive innovation. Allowing people to tinker with the device (root access, access to the DSP) attracts hackers, which in this case lead to DSP accelerated Theora video decoding. It's quite fitting that Apple is resisting Theora in HTML 5, mainly because their devices don't have accelerated Theora decoding (this is what I assume). While at the same time the restrictions imposed by Apple make it impossible to develop something like this for their iCrap devices (apart from not having root access, no API for accessing the DSP, it would also violate the developer agreement.. since you need some DSP assembly, which is not an approved language).

Re:Firefox

[HeikkiK]

We need a backup in the case H.264 patent owners start collecting some money when H.264 is spread everywhere. If I understood right, the current permission to use H.264 freely in free web content is just temporary and ending December 31 2010! Developing and optimizing these codecs take time, so better start working now. In fact we should hurry!

Also the situation is very similar to using OpenOffice-threat to blackmail discounts for Microsoft Office. So even if you happen to like H.264 more than Theora, this development is a win for you (unless you happen to be H.264 patent owner).

One phone is a good start. If you carefully read the introduction, it says that C64x DSP is quite common. The optimization just have to be taken into use. N900 just happens to be the most open phone right now so it is a natural first target.

Also well working and hardware optimized codec brings some pressure to hardware optimize also H.264 for common platforms.

Yes you have no idea

[Nicolas MONNET]

They made their point quite clearly.

They can't embed the codec and remain truly Free software.

Second, while they could link to platform-provided codecs, it's a support nightmare.

Third, it would legitimize patented crap.

Re:Yes you have no idea

[someSnarkyBastard]

No, the name and icon are not closed-source, they are trademarked. There is a difference between the two. Linux is trademarked by Linus Torvalds, but to claim that makes it closed-source is patently ridiculous. Trademark only means that other parties cannot use your brand to advance their own products without permission, aka diluting your brand. The fact that you can point to Libre derivative forks of Firefox disproves your claim that Firefox is closed-source.

Re:Give it up, Mozilla :)

[imroy]

I sure don't want to go back to the Bad Old Days where FF on Linux couldn't view the same media as FF on Windows.

That's currently the situation with plugins. They're pieces of object code, so you're dependent on the plugin developer to provide you with a version for your OS/architecture. For example, Flash (something that HTML 5 and the video tag could replace) is (AFAIK) only available on three mainstream platforms - Win/x86, OSX/x86, and Linux/x86. There's a beta version for x86-64 and special versions (Lite?) for embedded devices, but that's pretty much it. As for the Quicktime plugin, there's no Linux or *BSD support at all (mainly because there's no QT on those platforms), although there's plugins that use VLC or totem, etc instead.

Now, lets look at Firefox. The Debian iceweasel package (Firefox without the branding) is listed as being available for 14 architectures (not counting the unofficial 68k port that languishes at 2.0). That includes x86-64 for your latest AMD64/Intel64 machines, armel for your portable ARM devices, powerpc for your old G3/G4/G5 PowerMacs, and a bunch of other architectures too. None of those are supported by Adobe or the many other companies that have, over the years, tried to lock us into their proprietary software with binary-only plugins. So I'm siding with Mozilla for some very practical reasons. Theora's not the best out there, but I'd prefer the web to be open to all.

h.264 use in free web content ends 2015

[tk77]

Please check your facts on the licensing dates, they changed in February.

http://www.mpegla.com/main/Pages/Media.aspx

(direct PDF of the press release is here: http://www.mpegla.com/Lists/MPEG%20LA%20News%20List/Attachments/226/n-10-02-02.pdf )

Re:h.264 use in free web content ends 2015

[gmuslera]

If all internet use h.264 now, will still be using, and in far more ways, in 5. You now can move away with not so big effort, in 5 years will be impossible, and most of the internet content will be tied to the will of a single company wanting that you and everyone else in the planet pays them for every device and app that potentially connects to internet. Support it "by now",and will never be pressure to change till will be too late.

And open source somewhat follows the "shoulders of giants" idea, all have the source, so the experience could be used with another future open source codec if necessary.

documenting Theora on http://en.swpat.org

[ciaran_o_riordan]

I've been documenting the software patents situation for Theora here:

• Ogg Theora
• Audio-video patents
• Campaigns to avoid certain patented ideas

Adding information about On2's v8 codec would be very welcome.

Re:Give it up, Mozilla :)

[arose]

And Microsoft and Apple CAN implement Theora, then we can have a nice baseline and H.264 fans can still use it if they wish to.

Re:Firefox

[Svartalf]

Heh... Damnshock, have you ever worked with a TI SoC?

The codec was implemented against the C64x+ on a Linux based target on OMAP3.

Each phone isn't magically different in that respect. In fact, TI provides a platform SDK to use the DSP and everybody using Linux as their base OS is using the same SDK and pathways to get the data in/out of the engine and merely specifying their own get/on get/off points, which are actually submitted to the DSP via the API TI provides to each and every customer of their hardware.

What does this mean?

It means that the DSP work done now is available, largely ready for use on:

Motorola Droid
Palm Pre
Palm Pixi
Nokia N900
BeagleBoard and derivatives...
Gumstix Overo Water & Fire
Open Pandora Handheld
iPhone GS... ...and anything else using an OMAP3 SoC and Linux.

Seriously. Done once, able to be used elsewhere as long as you provide consistent interface rules. You might need to adjust the place you get and put the data for the operation, but the algorithm and the vast bulk of the DSP code doesn't change from device to device.

More to the point, you're going to find that with a bit of tweakage, the work can probably be implemented on OMAP2 platforms such as the Nokia N800/N810 as there's more than enough headroom on the C64x+ to lead me to believe that there's a possibility of doing it on a C55x class DSP- not that someone would go to that trouble unless there was a business case for it.

Now, this doesn't mean everybody gets "hardware" Theora yet- you still need to implement for at least Blackfin and Snapdragon as well for the bulk of the mobile device space. The main big deal here is that it has now been shown that it CAN be done and done fairly easily.

Good thing I still have it in my basement

[paradigm82]

I knew it would some day be possible to have full-HD on my old C64 :-)