YouTube, HTML5, and Comparing H.264 With Theora

David Gerard writes "Google Chrome includes Ogg support for the <video> element. It also includes support for the hideously encumbered H.264 format. Nice as an extra, but ... they're also testing HTML5 YouTube only for H.264 — meaning the largest video provider on the Net will make H.264 the primary codec and relegate the equally good open format Ogg/Theora firmly to the sidelines. Mike Shaver from Mozilla has fairly unambiguously asked Chris DiBona from Google what the heck Google thinks it's doing." DiBona responded with concerns that switching to Theora while maintaining quality would take up an incredible amount of bandwidth for a site like YouTube, though he made clear his support for the continued improvement of the project. Greg Maxwell jumped into the debate by comparing the quality of Ogg/Theora+Vorbis with the current YouTube implementations using H.263+MP3 and H.264+AAC. At the lower bitrate, Theora seems to have the clear edge, while the higher bitrate may slightly favor H.264. He concludes that YouTube's adoption of "an open unencumbered format in addition to or instead of their current offerings would not cause problems on the basis of quality or bitrate."

Comments

Theora FAIL

[drinkypoo]

Understanding TFA linked from your "equally good" link to a slashdot story? YOU FAIL IT!!! From TFA:

Let me reiterate- and this is important- as folks have run way too far cherrypicking quotes from this update: Both before and after the correction, this graph shows only that Theora is improving. PSNR means very little when comparing Theora directly to x264. PSNR is an objective measure that does not represent perceived quality (though they correlate), and PSNR measurements have always been especially kind to Theora. None of these PSNR measurements, including clips where Thusnelda 'wins', mean that Thusnelda beats x264 in perceived quality, as it certainly does not (yet ;-), only that the gap is closing even before the task of detailed subjective tuning has begun in earnest.

So just to recap, you have suggested that Ogg Theora video provides quality comparable to H.264 based on a study using a specific development-version Ogg Theora video codec and a specific H.264 encoder (x264) which is NOT the best encoder around, when it in fact has inferior SnR (the only thing the study was meant to test) as compared to x264, which has inferior SnR as compared to other H.264 encoders?
I don't know who failed bigger, you, Soulskill, or the peoples of slashdot who actually use the firehose... but you have all failed miserably.

With all that said; is there any reason they can't add Theora support later?

Re:Theora FAIL

[Jurily]

With all that said; is there any reason they can't add Theora support later?

The codec Youtube uses will severely affect everything else on the net, if they come out first. You can't deny that.

How long will it take for IE to have support for another codec? They will have Youtube support in no time, I guarantee you that.

Re:Theora FAIL

[mariushm]

IE will probably render any video tag through Silverlight, forcing you to install it. That's how you make market share for your products in Microsoft land.
On the good side, Silverlight 3 has support for both WMV and h264 and can decode them in hardware using the video card.

Re:Theora FAIL

[IntlHarvester]

I don't see a problem with this approach. One of the silly things about HTML5 is that it looks like browser vendors are all going to run off and implement their own media stacks. Which just increases bloat and potential security issues. Why not just use WM, QT, or whatever comes with the OS?

Not to mention that if I'm RTFAing correctly, Firefox's <video> tag is already incompatible with Chrome's.

Re:Theora FAIL

[samkass]

YouTube may have some effect on the de-facto internet codecs, but Theora has been losing this battle for awhile now so this isn't an out-of-the-blue decision. Many desktop and embedded video chips can decode h.264 in hardware, it's the primary Blu-Ray codec, it's used in several video chat applications, and many cable and satellite providers are going from MPEG2 to h.264. In addition, YouTube has been using h.264/AAC for over a year for "high quality" videos and videos delivered to iPhones, so they already have an h.264 infrastructure.

And for consumers, it actually seems to work really well. The "encumbered" nature of the codec may affect some tiny number of people, but for most it appears to be a huge win.

Re:Theora FAIL

[hairyfeet]

Good point. My Radeon 4650 supports hardware decoding of Divx, WMV, and h264. Does Theora even have a hardware accelerated codec? With the rise of netbooks, green computing, and the Ion Netbook solution it is pretty obvious that hardware assisted video decoding is where the market is headed. So even if Theora gets "good enough" (which reading TFA may be awhile yet) if Theora doesn't come up with a good hardware assisted decoder and quick I'm guessing it will be a non starter.

Re:Theora FAIL

[Ralish]

Your argument subtly implies that Firefox's implementation is more secure, without providing any proof of your own assertion. Bluntly, Firefox's security record has been far from top-notch for quite some time now, and while their patch response times tend to be excellent, this doesn't change the fact that security vulnerabilities of varying severity are still frequently occuring; and we're all familiar with Microsoft's security record. I can't conclude which implementation is likely more secure.

Which is irrelevant anyway, as you've missed the point of the GP's post in the first place (did you listen?). His argument was that if the OS supports decoding the video format, which it will if it's a modern consumer OS, why should every browser then implement its own media stack to provide a service that the OS already provides? You just end up with a proliferation of software that all does exactly the same thing. Thus, you end up with more security issues (as each implementation will almost certainly have security flaws throughout its lifetime) and more bloat (code duplication, and increase in code size for each respective browser implementing its own media stack).

You can be surgical here and note that this doesn't necessarily translate to greater exploitation, just more security issues. Lots of different media stacks means different exploits, meaning different exploit code, and incompatibility is high. So, any given exploit might only be able to target a small subdomain of the overall browser market, but this is really just a security through obscurity argument, and good security practices (e.g. sandboxing) should mitigate such concerns, and all browsers should have either implemented such technologies or have it on their roadmap.

I understand the value in having a variety of different options, but implementing a solution for no express reason than to offer an alternative, is inherently pointless. It has to have an advantage (and no, being open-source isn't an advantage for most), so if the OS implementation is up to snuff, then the GP does have a valid point.

Re:Theora FAIL

[iluvcapra]

Question is how long it will take them to implement HTML5 video.

They'll get to it, just after they've managed to properly implement HTML.

Re:Theora FAIL

[roca]

Where by "some tiny number of people" you mean "everyone broadcasting H.264 on the Internet, next year when the moratorium on H.264 Internet broadcast fees runs out".

Re:Theora FAIL

[master5o1]

I am in partial agreement: The browser venders should be implementing HOOKS to the operating system's native multimedia libraries. In Windows, Internet Explorer, Firefox, Safari, Chrome and Opera should all be hooking into DirectShow, QuickTime if installed, ffmpeg if installed, VLC's libraries, if VLC is installed.

In Linux distributions, Firefox, etc should all hook into FFmpeg, Gstreamer, etc.
On MacOS X, Safari (etc) should hook into QuickTime.

They should be acting more like any other media player: Implement the native multimedia API, rather than 'create' your own. This way all browser should be able to support as many codecs as the operating system can support.

Re:Theora FAIL

[mabhatter654]

the point is that the codex part of Theora is pretty settled down. Sure it's slow, but it's FREE... really Free just like png or HTML. The HTML5 group isn't mandating that people HAVE to use Theora for commercial sites. What they're really after is that ALL web browsers will support Ogg & Theora as part of the basic specification. Then everybody will be able to have multimedia functions without paying anybody royalties. It's the companies with interest in their own pony that are causing the problems because they like having everybody have to pay "somebody" for multimedia.

Re:Theora FAIL

[mabhatter654]

the spec is designed to be open and to use whatever the vendor wants to include. That's good. Along the way the HTML5 folks are trying to throw Free Software a bone by using Ogg and Theora as the "preferred" spec partly as a matter of philosophy and partly as a matter of pragmatism .

The big problem is Apple and Noika. Both of which build hardware and both have significant browser interests now... webkit and Qt (covering Safari, Nokia phones, and Chrome].Both also have no problem being buddy with the media companies and other patent holders. Unlike Firefox and Opera, Apple and Nokia are part of the patent club and see no need to "rock the boat" for "moral principal" reasons. Hence people keep berating Ogg & Theora simply so that they look "right" by not playing along simply because they don't want to and it conflicts with their other interests.

Re:Theora FAIL

[hairyfeet]

I personally don't give a flying crap if they do it by bashing pixies over the head to squeeze pixie dust into the hardware, all i care about (and I'm guessing 99.995% of the public agrees) is that my CPU isn't being pounded when I'm running 1080p. And since I happen to have the box in front of me I'll be happy to quote from it. Quote-

Multi Code Hardware Acceleration. Enables inloop deblocking, Motion Compensation acceleration for the latest codecs including H.264, WMV9, DivX. programmable video engine with enhanced post processing capabilities.

Now again me and the bazillion other folks buying these things don't give a crap HOW they are doing it, all we care about is we can watch 1080p while doing half a dozen other things because our CPUs aren't being pounded into next week. And my earlier point still stands. with the rise of Netbooks, Nettops, and Ion based platforms more and more of the video decoding is being done on the GPU instead of the CPU. Even on a dual core desktop like mine offloading to the GPU gives the video a nice fast cache of GPU RAM and a processor more designed for video decoding than a general purpose CPU, which lets my CPU do other tasks, as well as cut down on heat as today's GPUs are a lot more heat efficient than CPUs.

If Theora wants to be taken seriously they need a GPU based hardware decoder than works on the big three, Intel, ATI, Nvidia, and they need it yesterday, and they need to start offering it to the GPU manufacturers so they can bundle support like they do for WMV9, Divx, and H.264. Because out of the box with the default drivers my Radeon decodes all of the above as well as Mpeg2. The Nvidia does the same. Folks don't want to go play "hunt the decoder" they just want it to work. So while I applaud the Theora guys for trying to come with a free high def codec I'd say they still got a ways to go for mainstream use. Hardware decoders for the big three should be right at the top of the list IMHO.

Decoding Chips

[chonglibloodsport]

Superior in objective PSNR Quality. OK.

How about CPU utilization? Are there any ultra-low-power decoding chips that play Theora?

H.264 already has a large install base of devices that play it. Is there enough of an advantage to Theora to warrant dumping all of those for new ones?

Re:Decoding Chips

[benwaggoner]

No, the problem is decoding too. Software decode is fine on the desktop, but a non-starter for phones. Good phone video requires and uses ASIC or GPU acceleration. Theora, as a much older and simpler codec will probably decode faster in software than a maxed-out H.264 bitstream, but even if it could get to full-screen on a handset, it'd require a lot more bandwidth, and would run the battery down very quickly.

"Bandwidth is the problem" is also very much Theora's problem. The rather...odd example linked to aside, for any interesting bitrate or quality, Theora will need at least 2x as many bits to hit the same quality level as H.264 High Profile.

The example page is a little confusing. While they compare Theora to H.264 (and admit it wins), their "money" compare is to H.263, which is a VP3 era codec in its own right. If they compared a good H.264 encode to Theora at the 327 Kbps bitrate, H.264 would turn Theora into a thin red paste.

Re:Decoding Chips

[benwaggoner]

Where's your evidence? Why is Greg's example odd? Have you done a comparable experiment with a different video clip to justify your 2x claim, or more importantly, show that at the same bitrate Theora looks much worse on your clip?

Xiph's own metrics show a 2x advantage on even a very easy short clip:
http://web.mit.edu/xiphmont/Public/theora/demo7.html

And yes, I've done plenty of my own tests as well on Big Buck Bunny and many other clips.

Why would it, since it didn't at 499Kbps? Or are you claiming that Youtube uses a bad H.264 encoder? Or do you think that example is rigged?

YouTube doesn't use High Profile (no 8x8 blocks or adaptive quantization matricies) or CABAC entropy coding. So they're going to be at least 20% less efficient than the best encodes could be.

Also, they trimmed the clip before the really interesting high motion parts. Most of the shots in the section they did use were static camera, and as animation is noise-free. Toss a nice grainy movie trailer in there and Theora shows basis pattern left and right.

I wish he'd reported the Theora settings used in the encode.

repeat of ogg?

[jd142]

I remember when ogg first came out. I read slashdot regularly, saw all the information about how great it was, how since it was free it would be easily adopted by hardware makers who didn't need to pay for the privilege. I bought into the hype. I ripped my cd's to ogg files, paid extra money for a neuros player because it was one of the few players that handled ogg files.

Now, 5 years later I have a large collection of ogg files that are essentially useless. No one in the mainstream uses ogg, despite the superiority and price. Whenever I get a new player, I have to carefully read the specs to see if it will play my oggs. Few do. Luckily I have the cds and I can simply re-rip them to mp3s as I find the time/care too.

My guess is that the same thing will happen with theora. It may be superior. It may be cheaper. But I just don't think it will catch on. It's another example of the slashdot echo chamber.

Re:repeat of ogg?

[vivaelamor]

Any chance we can blame Slashdot for VHS too?

Re:repeat of ogg?

[vadim_t]

Rip to FLAC, then encode that into whatever fits the device best.

In my experience, finding a player that does .ogg isn't that hard. Look at the players made by Cowon for instance, they're very nice.

Re:repeat of ogg?

[nxtw]

Ogg Vorbis was better in quality than MP3 - back then (and even today) the most popular compression for music. However, AAC and WMA are also better than MP3 - and people actually sold music in AAC and WMA formats as well as MP3.

Theroa is not better than h264 (the new popular standard for video on the Internet, many Blu-ray discs, HD satellite, and HD broadcast in some parts of the world), so it's not a repeat of Vorbis at all. Theora just scores higher on a scoring algorithm when compared ot a single h264 encoder, the open-source x264.

Re:repeat of ogg?

[Bagels]

Not to defend ogg vorbis too much, but it has actually achieved success in a few realms - it's the audio format of choice on Wikipedia, which is one of the web's most popular sites, and it's used in tons of video games (precisely because it doesn't need to be licensed, I think). The things that made it successful in those areas (matching ideology and price/compression performance, respectively) don't really mean much to the average MP3 player user, though.

Re:repeat of ogg?

[jd142]

I wasn't blaming so much as pointing out that like many blogs, slashdot can be an echo chamber. The same opinions are repeated over and over and treated as if they are held by the majority of people. I was younger then and still thought slashdot had a finger on the pulse of technology. It doesn't. It's really great as a news aggregator and the comments are often a hoot, but it isn't what I thought it was.

Re:repeat of ogg?

[julian67]

There are an awful lot of players which support ogg. Almost anything from Cowon, iRiver or Sansa does. And almost all the Chinese brand/no-name/shop brand players support ogg even though they fail to explicitly state this (preferring to emblazon their players and packaging with mp3 and wma logos). I used to import and sell mp3 and mp4 players and generally it's only the very cheapest mp4 video players which don't support ogg, that's the ones which claim asf container support is something to brag about.....usually these use an old rockchip video processor.

I have 5 personal players. 2 are old iRivers, H140 and H340, 2 are tiny no name Chinese mp3 players and one is a Chinese mp4 video player. Only the iRivers claim to support ogg audio but the cheap mp3 players handle it fine as well. I lived in SE Asia and every cheap mp3 player I ever checked played ogg audio too. Not a single one made mention of it in the instructions, the specs, on the box or on the player.

Re:repeat of ogg?

[Daniel_Staal]

Bingo. Theora may be equally as good, but it's trying to supplant an already-established format. 'Equally as good' isn't good enough for that: You have to be noticeably better. And Theora isn't. It offers no major advantages, and would just give YouTube headaches, as it either tried to re-encode into a choice of formats, or had to explain to people how to play the videos.

The first of those costs money, the second costs viewers. I'd bet very few people would choose the Theora choice, making the money just wasted money. And YouTube lives on it's viewers: making their site any more complicated than 'click play' is just not acceptable.

It's not worth it. Theora doesn't have enough of a benefit.

Re:repeat of ogg?

[Dutch Gun]

Ogg Vorbis is also used in video games because it has some other advantages: it supports 6-channel audio, and has support for bit-accurate decoding, allowing seamless looping of audio, and it sounds better at lower bitrates. I know MP3s can be kludged to do some of these, but it's easier just to use Vorbis in these cases.

Our upcoming game will actually be shipping with both MP3 and Ogg Vorbis audio. The MP3 decoder we're using is significantly more efficient than the reference Vorbis libraries, and allows us to play more simultaneously decoded channels. However, if a piece of audio needs to loop, to use multi-channel, or if we're encoding a LOT of it (music, voice-overs, etc), we use Ogg Vorbis.

Honestly, the cost of the license isn't really an issue at all. It's all about what does the job best for us, and MP3 and Vorbis each have strengths and weaknesses.

Re:repeat of ogg?

[iMacGuy]

> Theora just scores higher on a scoring algorithm when compared ot a single h264 encoder, the open-source x264. It doesn't even do that; it only scored higher when using Xiph's PSNR tool, because it respected a buggy colorspace header written by ffmpeg that didn't match the video. x264 won rather heavily when that was fixed, but /. never retracted the story.

VHS was better

[tjstork]

Everyone has made a mythology about VHS somehow losing to Sony Beta despite being inferior. If you lived in that day, and walked into a store, there was really no significant difference between picture quality between VHS and Beta on the average TV of the day. There just wasn't. And, everyone forgets that the superiority of Beta was achieved by making the tapes only an hour long. VHS vs Beta was a silly argument. Beta claimed superior picture quality on TV's nobody had, but, VHS could store entire movies. To most people, Beta's claims sounded a lot like BS, while VHS was clearly better.

Re:Somebody explain to me why HTML5 != evil

[moogord]

html was never really designed to do much more than have a single "document" that can link to other "documents" on the internet. over time dynamic ideas were tacked on such as javascript but it still has never been designed in such a way that 'app-y' ideas can be created without hacking up the 'document' model.

Thus html 5 attempts to correct this by modifying the original 'document' model so that it now supports 'documents' and 'app-y' ideas. its not evil, its progress.

Re:Stupid stupid stupid...

[sakdoctor]

Are those sarcasm tags part of the HTML5 standard?

Re:Google and Firefox

[selven]

Stop misquoting the motto! It's "don't be evil", not "do no evil". Google is just saying that they don't intend to screw over their own customers, not that they intend to become the moral custodian of justice for the entire world.

Re:Stupid stupid stupid...

[loufoque]

An open-source browser cannot legally read h264 video, that is the real issue that people seem to have trouble to understand. That is why the HTML standard only mandates a format that is not impaired by any legal restrictions: Theora.

Not being able to legally play DVDs, Blurays, connect your ipod, etc. on linux are already big problems, we don't need another one.

That's a different situation

[Wesley Felter]

Pirates have the advantage that they don't have to pay for patent licenses, so H.264 and Theora are both "free". But for law-abiding companies like Mozilla and Google, Theora is free and H.264 isn't.

Re:That's a different situation

[benwaggoner]

But the cost of the H.264 licenses are vanishingly small compared to the extra bandwidth cost of using Theora for a company like Google or Apple.

Using H.264 is free as long as clips are under 12 minutes. A number which may sound familiar to YouTube users :).

Here's info on the MPEG-LA licensing terms: http://www.mpegla.com/avc/avc-agreement.cfm

Re:That's a different situation

[benwaggoner]

This?

http://people.xiph.org/~greg/video/ytcompare/comparison.html

I read it, and have commented on it at length throughout this thread.

Basically the article briefly says that YouTube's H.264 is better than Theora, and then goes on at length showin how Theroa is better than H.263.

Xiph's own data shows H.264 has a big bitrate advantage at the same quality level even in a test that should favor Theora.

http://web.mit.edu/xiphmont/Public/theora/demo7.html

in the Rate-Distorion graph, note they start the plot at 50 Kbps, so look at the actual numbers.

For example, at 40 dB, x264 needs 70 Kbps and Theora needs 120 Kbps.

The gap would be bigger with higher motion, more detail, longer content, and particular when there are buffer constraints. Also, x264 is (properly) tuned for perceptual quality more than strict PSNR accuracy.

Theora suffers from not being a very mature implementation (which Xiph is making great progress on addressing) and being a 90's era codec design (about which Xiph can't do anything without breaking compatibility). And other codecs are getting better as well; if Theroa is refined enough to be reasonably optimal in a year, it'll be competing against the improved H.264 and VC-1 codecs of a year from now, and H.265 not that far away.

There are all kinds of interesting things about Theora, but competitive low bitrate compression efficiency isn't going to be one of them.

Re:MS won't require Silverlight. Too easy.

[hplus]

Why would they install another browser when they could just click the "Click here to install silverlight and watch this video" button?

Re:Laziness or Ignorance? You decide

[benwaggoner]

Reread Shaver's methadology:

A keyframe interval of 250 frames was used for the Theora encoding.

10 seconds is absurdly short for any kind of codec test. That's almost as long as the buffer would be, and current Thusndela builds don't include full buffer management. Plus he picked a pretty low motion section of the clip. He should the full clip. Current Theora builds are plenty fast; it'd be faster than realtime on a laptop.

In a real codec compare, CBR is often the best way to see differences between codecs and implementations, since that's where rate distortion really shows its stuff. How well a codec can preserve quality with high motion in a fixed buffer is a key differentiatior.

That said, I believe that the Theora+Vorbis results are substantially better than the YouTube 327kbit/sec. Several other people have expressed the same view to me, and I expect you'll also reach the same conclusion. This is unsurprising since we've been telling people that Theora is better than H.263

His primary quality comparison is between Theora and H.263, not H.264. H.263 is even older than VP3 which Theora is based on. As to H.264 he says:

In the case of the 499kbit/sec H.264 I believe that under careful comparison many people would prefer the H.264 video.

Yep. And it would be a huge differential if he'd picked a more challenging section of the source.

And while it doesn't have any impact on the comparison, no compressionist would use those frame sizes. We always try to round to the nearest mod16 value, so that we have macroblock alignment.

Thus 480x272 and 400x224 would be more efficient choices in both cases. 400x226 is particulary egregious, as it means the codec is really encoding at 400x240 internally with 14 lines of padding.

Re:Stupid stupid stupid...

[Super_Z]

This document describes the terms of the H.264 license. The license seems to cover both encoding and decoding.

Re:Laziness or Ignorance? You decide

[Ben Hutchings]

10 seconds is absurdly short for any kind of codec test.

WTFV. The keyframe interval is 10 seconds but the clip is nearly 5 minutes long.