Slashdot Mirror
Hardware-Accelerated Ogg Theora For Firefox Mobile
An anonymous reader writes "Matthew Gregan is working on bringing David Schleef's DSP accelerated port of Theora to Firefox Mobile. He writes on his blog: 'The C64x+ DSP is often found in systems built upon TI's OMAP3 SoC, such as the Palm Pre, Motorola Droid, and Nokia N900. Last year, Mozilla funded a port, named Leonora, of Xiph's Theora video codec to the TI C64x+ DSP. David Schleef conducted the port impressively quickly and published his results. The intention of this project was to provide a high-quality set of royalty-free media codecs for a common mobile computing platform. The initial focus is Firefox Mobile on the N900, so I am working on integrating David's work into Firefox. To experiment with other facilities Firefox could use to accelerate video playback, and test integration, I've been hacking on a branch of a stand-alone Ogg Theora and Vorbis player originally written by Chris Double called plogg.'"
I have no idea why Firefox still fights against the giant and loses money and time on it. Great, they got hw-accelerated Theora to one single mobile phone. What about all the televisions, other mobile phones, computers, airplanes, PS3, 360, and everything else under the sun that has H.264 hw supported? It's a lost battle.
If I were them, I would seriously start concentrating on the next generation of video codecs. It might be closed H.264 for now, but if you want to get an open source product out there, you have to make it technically better, make sure (and contact!) companies to support it in their products, and just do marketing and PR.
Open source has some advantages, but if it's technically lesser and doesn't work with companies, it's not going to win.
In my mind, hardware accelerated means using fixed-function, special purpose functional blocks in silicon. Saying that using a DSP is hardware acceleration for video codecs is like saying that software is hardware accelerated by the execution units of a CPU.
It's great seeing the benchmarks showing the CPU usage dropping from 99% to 1%, but at the same time the DSP and GPU usage is going up by some unstated amount. It would be really great to see some comparison of how this effects the battery life. Playing MP3s on the C64x can be done in a bit under 15mW, but Theora is a lot more complex and doing the colourspace conversions and compositing on the GPU is going to add a bit too. I'd expect the power usage to be lower than doing it on the CPU, but maybe not by a huge amount.
Either way, it's great that they can free up the CPU to do other stuff. 0.4% of the ARM core isn't really enough to run the scripts that typically accompany a web page that uses the video tag, 99% almost certainly is.
I am TheRaven on Soylent News
Theora isn't bad. It's better than MPEG-1 and a MPEG-2, and similar quality to MPEG-4 part 2. It's not as good as H.264, VC-1, or Dirac, but that's not the same as being bad. It's perfectly acceptable for a lot of uses.
I am TheRaven on Soylent News
I'll admit that Theora isn't the greatest video codec. But it's been explained many times that Mozilla simply can't use MPEG-4 AVC/h.264 because of the patents involved.
Mozilla and wider Open Source world has three options: (not mutually exclusive)
But no amount of whinging will make them use AVC/h.264. It's simply not an option at the moment.
This is a good example how open systems drive innovation. Allowing people to tinker with the device (root access, access to the DSP) attracts hackers, which in this case lead to DSP accelerated Theora video decoding. It's quite fitting that Apple is resisting Theora in HTML 5, mainly because their devices don't have accelerated Theora decoding (this is what I assume). While at the same time the restrictions imposed by Apple make it impossible to develop something like this for their iCrap devices (apart from not having root access, no API for accessing the DSP, it would also violate the developer agreement.. since you need some DSP assembly, which is not an approved language).
I still don't understand his comment, because if we do what he suggest, and look towards Google:
http://mobile.slashdot.org/story/10/04/10/1147254/Google-Funds-Ogg-Theora-For-Mobile?art_pos=8
Seems Google's view opposes his.
Change is certain; progress is not obligatory.
We need a backup in the case H.264 patent owners start collecting some money when H.264 is spread everywhere. If I understood right, the current permission to use H.264 freely in free web content is just temporary and ending December 31 2010! Developing and optimizing these codecs take time, so better start working now. In fact we should hurry!
Also the situation is very similar to using OpenOffice-threat to blackmail discounts for Microsoft Office. So even if you happen to like H.264 more than Theora, this development is a win for you (unless you happen to be H.264 patent owner).
One phone is a good start. If you carefully read the introduction, it says that C64x DSP is quite common. The optimization just have to be taken into use. N900 just happens to be the most open phone right now so it is a natural first target.
Also well working and hardware optimized codec brings some pressure to hardware optimize also H.264 for common platforms.
Mozilla can use AVC/H.264. They just need to do it via the OS, like I think IE and Safari will do, and Opera already does. H.264 is already directly supported on Windows 7 and Mac OSX. Linux users need to do their tricks, like with MP3.
They only way they can do as you suggest is by severely compromising Firefox. This has been answered exhaustively many times. See Mike Shaver's blog post about it. To quote:
People have raised questions about using existing support for H.264 (or other formats) that may already be installed on the user’s computer. There are issues there related to principle (fragmentation of format under the guise of standardized HTML), to effectiveness (about 60% of our users are on Windows XP, which provides no H.264 codec), to security (exposure of arbitrary codecs to hostile content), and to user experience (mapping the full and growing capabilities of to the system APIs provided); I’ll post next week about those in more detail, if others don’t beat me to it.
Security is not to be underestimated. It's worth noting that IE9 will not support all media formats supported on the host operating system, purely to limit the attack vector surface area available. See this comment by Microsoft's Frank Olivier on the IE blog. He's talking about image formats but the same applies to video formats:
We've not heard many requests for additional image types - to limit the attack surface in a web browser, it is a good idea to not expose more decoders.
See also Chris Blizzard's blog post about the decision to only support open video in Firefox. And most importantly of all, don't worry about. Open video is the way forward for video on the web. Theora is here now, VP8 may be joining it next month, and Dirac may join both in the future.
They made their point quite clearly.
They can't embed the codec and remain truly Free software.
Second, while they could link to platform-provided codecs, it's a support nightmare.
Third, it would legitimize patented crap.
That's currently the situation with plugins. They're pieces of object code, so you're dependent on the plugin developer to provide you with a version for your OS/architecture. For example, Flash (something that HTML 5 and the video tag could replace) is (AFAIK) only available on three mainstream platforms - Win/x86, OSX/x86, and Linux/x86. There's a beta version for x86-64 and special versions (Lite?) for embedded devices, but that's pretty much it. As for the Quicktime plugin, there's no Linux or *BSD support at all (mainly because there's no QT on those platforms), although there's plugins that use VLC or totem, etc instead.
Now, lets look at Firefox. The Debian iceweasel package (Firefox without the branding) is listed as being available for 14 architectures (not counting the unofficial 68k port that languishes at 2.0). That includes x86-64 for your latest AMD64/Intel64 machines, armel for your portable ARM devices, powerpc for your old G3/G4/G5 PowerMacs, and a bunch of other architectures too. None of those are supported by Adobe or the many other companies that have, over the years, tried to lock us into their proprietary software with binary-only plugins. So I'm siding with Mozilla for some very practical reasons. Theora's not the best out there, but I'd prefer the web to be open to all.
the rest of the world plays grande ball with the H.264 format and the superior encoders available for it
Up until Dec 31, 2010 when the patent holders have stated that they're ending the royalty-free period and it becomes the GIF of the video world.
You can bet that with all the money youtube loses now, Google ain't going to pay for millions of H.264 videos and H.264 will literally disappear overnight. I'm sure Google is working overtime on getting that shiny new codec they just bought into chrome, if not firefox. Perhaps they'll write a flash player for it too.
If I have been able to see further than others, it is because I bought a pair of binoculars.
Please check your facts on the licensing dates, they changed in February.
http://www.mpegla.com/main/Pages/Media.aspx
(direct PDF of the press release is here: http://www.mpegla.com/Lists/MPEG%20LA%20News%20List/Attachments/226/n-10-02-02.pdf )
This is /. so how dare you try to use an argument that may or may not use facts and or logic in it. Plus I ain't clicking on any fancy links just because you want to blow steam off on the internet. Remember winning an argument on the internet is like winning the special olympics, your still retarded after you get that medal.
I've been documenting the software patents situation for Theora here:
Adding information about On2's v8 codec would be very welcome.
Expert in software patents or patent law? Contribute to the ESP wiki!
VP8 is RUMORED to be made open source sometime in the near future.
We do not know for sure.
Theora isn't very good, but that doesn't mean that it is utter crap.
At youtube video quality it uses somewhere around 10% more bandwidth than h.264. It gets less efficient with HD content.
That is not correct.
http://www.mpegla.com/Lists/MPEG%20LA%20News%20List/Attachments/226/n-10-02-02.pdf
(DENVER, CO, US – 2 February 2010) – MPEG LA announced today that its AVC Patent Portfolio License will continue not to charge royalties for Internet Video that is free to end users (known as Internet Broadcast AVC Video) during the next License term from January 1, 2011 to December 31, 2015. Products and services other than Internet Broadcast AVC Video continue to be royalty-bearing, and royalties to apply during the next term will be announced before the end of 2010.
Morpheus, God of Dreams.
And Microsoft and Apple CAN implement Theora, then we can have a nice baseline and H.264 fans can still use it if they wish to.
Analogies don't equal equalities, they are merely somewhat analogous.
No, that's a stupid idea.
Not all systems have an h.264 codec installed. Mac OS X does (via QuickTime). Windows 7 has an included h.264 codec, but neither Windows Vista nor Windows XP do. The only available h.264 codecs for Windows Vista and Windows XP are either commercial (which nearly nobody will have installed) or unlicensed. On Linux, there is exactly one licensed h.264 codec (Fluendo's GStreamer plugin), which virtually nobody has installed.
So this doesn't solve the problem at all. It also has some rather bad side-effects.
First, it makes the browser behave differently depending on which OS you're using. Mac OS X can basically only play h.264 and MPEG-4 ASP out of the box, and only in the MPEG-4 / QuickTime container format. Windows XP and Vista can play a couple of useless, outdated codecs in the AVI container, or one of several versions of WMV. Oh, and Windows XP didn't ship with a WMV9 codec. On Linux... anything can happen - GStreamer may or may not be available, and what codecs it actually contains varies widely (although it almost always includes Theora). What about embedded systems that don't even have their own codecs? What about embedded systems which have their own codecs, but provide no way to install new ones?
It also increases the attack surface, which will cause more security vulnerabilities. Say you just let DirectShow handle any media type. You've not only exposed Windows' h.264 decoder, but also every other decoder and parser available through DirectShow. Some of those are from the mid to late 90's. More likely than not, they're full of security vulnerabilities.
To put it another way:
IE9 ships with it's own h.264 decoder. It's the only way to support h.264 on Vista, and it limits the attack surface. Silverlight ships with it's own h.264 decoder, for exactly the same reasons. So does Flash. Safari on Windows effectively ships with it's own h.264 decoder (via QuickTime). It only uses the OS supplied codecs on a Mac, where Apple control the entire platform anyway. Chrome includes it's own h.264 and Theora decoders on all platforms. Opera includes it's own Theora decoder, via an included version of GStreamer. Firefox includes it's own Theora decoder.
They only way they can do as you suggest is by severely compromising Firefox.
Utter garbage. Firefox is *already* subject to any bugs in the underlying OS on an incredibly wide range of issues. Hell, just a few years back, Firefox was compromised thanks to an image rendering bug in GDI on Windows. So, what, *now* they've suddenly decided they're going to reimplement OS-supplied services for fear of security issues? Please.
On the bright side, at least the Mozilla devs *tried* to invent a technical excuse for not supporting H.264, now that people have realized there are no legal barriers to doing so. Of course, it's a stupid, irrelevant, indefensible technical excuse. But, hey, kudos for trying, guys.
Excuse me... Video in HTML5 is supposed to get rid of that plugin thinking in the first place .
It's not that you can do that- HELL, you can do that NOW.
The real problem is that while it's implemented and "legitimized", each and every use requires a payment and a sign off against a license you may/may not agree with. Even to provide the content on the Internet will eventually require royalty payments with it. You won't be able to legally implement ANYTHING with video without their permission- and they can withhold it- just witness what's happening with Nokia and Apple right at the moment.
If you've not caught all of that, perhaps you need to pay closer attention.
I am not merely a "consumer" or a "taxpayer". I am a Citizen of the State of Texas
Heh... Damnshock, have you ever worked with a TI SoC?
The codec was implemented against the C64x+ on a Linux based target on OMAP3.
Each phone isn't magically different in that respect. In fact, TI provides a platform SDK to use the DSP and everybody using Linux as their base OS is using the same SDK and pathways to get the data in/out of the engine and merely specifying their own get/on get/off points, which are actually submitted to the DSP via the API TI provides to each and every customer of their hardware.
What does this mean?
It means that the DSP work done now is available, largely ready for use on:
Motorola Droid ...and anything else using an OMAP3 SoC and Linux.
Palm Pre
Palm Pixi
Nokia N900
BeagleBoard and derivatives...
Gumstix Overo Water & Fire
Open Pandora Handheld
iPhone GS...
Seriously. Done once, able to be used elsewhere as long as you provide consistent interface rules. You might need to adjust the place you get and put the data for the operation, but the algorithm and the vast bulk of the DSP code doesn't change from device to device.
More to the point, you're going to find that with a bit of tweakage, the work can probably be implemented on OMAP2 platforms such as the Nokia N800/N810 as there's more than enough headroom on the C64x+ to lead me to believe that there's a possibility of doing it on a C55x class DSP- not that someone would go to that trouble unless there was a business case for it.
Now, this doesn't mean everybody gets "hardware" Theora yet- you still need to implement for at least Blackfin and Snapdragon as well for the bulk of the mobile device space. The main big deal here is that it has now been shown that it CAN be done and done fairly easily.
I am not merely a "consumer" or a "taxpayer". I am a Citizen of the State of Texas
I knew it would some day be possible to have full-HD on my old C64 :-)
You could probably do that, but Chrome is not open source.
You say that as if people care about "open source".
They don't. At all.
All normal people see (and say) is "the Cool New Video Site is broken in firefox, which obviously must be firefox's fault. So I'll just use Chrome/IE/whatever".
What all the anti-H264 people are missing is that not supporting H264 will drive people away from Firefox. I fail to see how driving people to the non-free browser is a "win for open source".
Ce n'est pas une signature automatique.