Slashdot Mirror
Theora Ahead of H.264 In Objective PSNR Quality
bigmammoth writes "Xiph hackers have been hard at work improving the Theora codec over the past year, with the latest versions gaining on and passing h.264 in objective PSNR quality measurements. From the update: 'Amusingly, it also shows test versions of Thusnelda pulling ahead of h.264 in terms of objective quality as bitrate increases. It's important to note that PSNR is an objective measure that does not exactly represent perceived quality, and PSNR measurements have always been especially kind to Theora. This is also data from a single clip. That said, it's clear that the gap in the fundamental infrastructure has closed substantially before the task of detailed subjective tuning has begun in earnest.'
Momentum is building with a major Open Video Conference in June, the impending launch of Firefox 3.5 and excitement about wider adoption in a top-4 web site. It's looking like free video codecs may pose a serious threat to the h.264 bait-and-switch plan to start charging millions for internet streaming of h.264 in 2010."
Sure, Theora is great, so is OGG Vorbis and FLAC... Unfortunately I can't really play any of those formats save for on my computer, and if I'm using something other than Linux, I most likely will have to install extra software in order to play them. So no, I don't think this will be some big improvement until I can play them on everything without extra software.
Taxation is legalized theft, no more, no less.
if your laying out cash on infrastructure i'd say it serves you right for not doing your homework first.
If you mod me down, I will become more powerful than you can imagine....
I don't mean to belittle Theora, I've really been rooting for them over the years. And this recent test does look fantastic.
But I can't help wonder what settings they are testing x264 with. x264 has recently been shown to be highly sensitive to clips like the Akiyo one tested here -- it also lost to some other H.264 encoders that it usually beats fairly consistently. The version and settings used to encode this one make a WORLD of difference.
my reading was $10,000 per year per local market service... assuming your internet services hits many thousands of local markets you would hit the maximum royalty for Participation ie millions. This may be an inaccurate reading. Your reading seems logical as well.
The PSNR graph is quite interesting. To get comparable PSNR values from a recent x264 for the given source, you will have to use ridiculously low settings. I got about 700fps, with the required (lowest) settings, which still give better PSNR at 250kbps (47.333db) and above (300kbps is 48.222db), than is marked on the graph. This is with the lowest possible x264 settings, one-pass ABR. Also note, how the PSNR graph for x264 looks like a perfect logarithmic curve. None of the other plots are as smooth. Now, if you were feeling paranoid, you might get the feeling that they didn't even test their source with x264 at all.
Then why haven't video game companies been hammered? Just about all video game consoles for the last decade or so have been sold at a loss for market share.
DEMETRIUS: Villain, what hast thou done?
AARON: Villain, I have done thy mother.
Shakespeare invents 'your mom'
The results are real:
x264-0.0.0-0.20.20080905.fc10.x86_64 was used.
PSNR computed with dump_psnr (tool that ships with Theora), so that the same tool could be used with multiple formats. I compared the decompressed lossless yuv4mpeg files. You can easily reproduce these results: Grab http://media.xiph.org/video/derf/y4m/akiyo_qcif.y4m and the current Theora Thusnelda SVN, the above mentioned x264 and go to town. Encode with defaults. Constant QI in both cases. (CRF and other common wisdom x264 knobs hurt PSNR in this case, though because of the nature of the test I would have stuck with defaults regardless)
This test wasn't intended to be a critical bake-off between formats. Thats something for a third party to do anyways. I feel somewhat dirty for having a part in something being spun this way.
A big concern for Theora is performing well enough that no one feels the need to regret using a freely licensed format. Being as good/better than some particular encumbered encoder would be great, but really it is just important to be in the ballpark. The videophiles are going to use whatever feels sexiest today (read: best marketed) regardless of licensing, CPU consumption, or even real quality.
While completely real this testing was not *at all rigorous*, you can think of the x264 example as something provided to give the graph scale and not something you can use to say that Theora is superior, only that its not laughably worse. I think this does show that some of the claims that "theora sucks" are over-hyped.
I initially created these graphs because someone published a paper with highly flawed and unreasonable results showing Theora doing >30dB worse than x264. So a lot of the testing parameters came from trying to mimic his particular test rig so I could understand his mistake. -- It just so happens that the graph makes a nice statement about Theora's improvement over time, so Monty made use of it in his latest report to his employer on Theora's progress.
No one involved with Theora is saying that this test says that Theora is generally better. It's only "Look, you can stop fretting about quality-- we're basically in the right ballpark now. It's time to get other issues like adoption, software support, etc fixed while the final polish is being put on the new encoder".
--Greg Maxwell
I've also commented on this reddit thread: http://www.reddit.com/r/programming/comments/8iphn/theora_encoder_improvments_comparable_to_h264/
It is because it was good enough, at the right time.
This is such an awesome, succinct way of explaining the sometimes-inexplicable success of so many things. I will be sure to use it again!
I actually don't think it's too late for Theora to have an impact though.
The big thing is the tag that is being considered for HTML5. If Firefox and Opera and Chrome all bust out solid support in release builds soon, we'll be converting our video library to support it (catalogue of video game trailers on ausgamers.com - I realise one site isn't enough, but if others feel the same...)
The big roadblock is IE - even with IE8's improved 'standards compliance' I can't see them wanting to encourage any other video streaming stuff when they're pushing silverlight so hard.
Interesting times though.
Yes, I was being sarcastic. I was actually showing that free solutions CAN catch on and dominate.
Image quality vs bitrate means very little without mentioning CPU/memory usage. H.264's greatest weakness is the heavy CPU load on playback, it's just not friendly to low-cost and/or mobile devices. If Theora can get within the ballpark in terms of quality, but beat H.264 in speed, that could be the edge it needs to hit the mainstream.
Right now it's little more than an academic experiment. Floating point everything can give you fantastic quality, but it will crawl so slowly that people will choose a lesser-quality alternative that runs faster.
-Billco, Fnarg.com
I looked at his links, and what I see backs him up. This optimization of the x264 codec is optimized for SSIM (Structural SIMilarity, a measurement of image comparison accuracy), which inherently decreases PSNR (Peak Signal-to-Noise Ratio). After all, an accurate, less densely compressed image will actually show textures and such better! I personally like to be able to pause a video I'm watching, and look at say... the whiskers on Hugh Jackman's face (I'm sure you guys might like the equivalent). While PSNR is important for determining compressibility, in lossy compression, you're talking about losing details no matter what. Some people want those smaller file sizes! Comparing apples with oranges is more of a taste comparison than an aesthetics debate, but it's still important to be informed about the real differences.
There are no perfect answers, only the right questions. More questions at http://foresightandhindsight.blogspot.com/
First off, most people don't care about lossless compression. It's a niche market. After all, even on extremely good sound gear, you are hard pressed to pick out 256k MP3 from uncompressed in blind tests.
And this matters because...? high definition video is also a niche market, as Blu-Ray vs DVD sales analysis would show. Yet obviously we're talking about popularity within its scope, otherwise not even the iPod would be popular, if we were to consider the entire human race.
Also, popular though it might be, it wasn't popular enough for the big boys to pick up. Both Apple and Microsoft did their own lossless formats.
Remember WMA? and AAC? no, the "big boys" ignored FLAC not because it wasn't popular enough, it was because both have *very* strong NIH sentiments against it, as they did with MP3.
No problem is insoluble in all conceivable circumstances.