Slashdot Mirror
ExtremeTech Wages War of the Codecs
prostoalex writes "ExtremeTech tested Windows Media, DivX, QuickTime/Sorenson and QuickTime/MPEG4 codecs. They encoded clips from Matrix Reloaded, Monsters, Inc., X2 and Spider-Man. QuickTime/Sorenson won the encoding speed contest, for the quality tests read the entire review, as each movie sample was encoded with 500KB and 1MB bitrates. Video samples provided on the site as well, so see for yourself."
They expect to have their opinion valued on Slashdot when they don't review the open source video codec? (It generally wins in other tests.)
How do they expect me to keep pirating Hollywood movies if they keep changing the damn codec?!
OK, the article blurb claims that QT/Sorenson had the fastest encoding times, but also had the third-worst quality (only QT/MPEG-4 was worse). DivX seems to have the best quality, which, in my opinion, should be the end goal.
Think about it, how many times are you going to encode a movie? How many times are you going to watch it? Typically, you are going to encode once and probably watch it multiple times. Therefore, I would happily accept a little longer processing time in the beginning if that means I will end up with a better quality production.
Here's their most recent codec shootout with 3ivx, Divx, ffvfw, Nero, Real, On2 and Xvid. Xvid wins.
Come on! Are we now suddenly supposed to actually read those frickin' articles? Just tell me who won. This is the internet and my attention span ... wait, what's that shiny thing? ...
Whether they hide the first pass or not, you'll need two if you really want quality. It's the only way for the codec to know for sure where it can spare bits and where it can't.
Unfortunately, due to the fact that the source was itself the result of mpeg encoding, it could unfairly impact the ability of the various codecs to handle the content. On the flip side, much of the content people are encoding is actually decoded content, i.e. from a digital camcorder, etc.
What would be interesting is taking the original raw film footage (that hasn't been digitally compressed with a lossy method) and encoding, then comparing the results.
Did you read the *whole* article? They state very plainly at the end that "DivX encoded clips tended to have a touch more detail, but also a few more compression artifacts, than the WMV9 video" and that DivX encodes much faster than WMV9. In brief, the only reason for choosing WMV9 over DivX is that it may be included in upcoming consumer devices.
Good people do not need laws to tell them to act responsibly, while bad people will find a way around the laws-Plato
Okay, I just encoded some DVD-size video at 1mbit and 500kbit, 1/4 size, in QuickTime MPEG-4 and can barely see any artifacts in either. This dude seriously got some settings wrong in his MPEG-4 encodings, although I don't quite see how that's possible as settings are quality, framerate, keyframes, and data rate (and he said quality was set at best). I'll post some screenies later if I get a chance.
Does it tell you which codec is best? Maybe but only for recompressing MPEG-2 footage. They *should* have tested against DV output as the standard consumer format, and uncompressed video. Plus looking at snapshots of compressed movies is of limited value, there is a big difference between what detail we can determine through a still image and a moving one. If you were to freeze a tv picture (or look at a captured frame) which includes something moving you would see a combing effect of the interlaced video. It doesn't look like that when you view it though.
Apple's MPEG-4 has both a mediocre encoder and decoder. 3ivx is a much better choice, it both decodes MPEG4 (and Divx/XviD) with better postfilters (the best, argurably), even scaled to how much CPU time it can grab. Its encoder is extremely nice, and very userfriendly, and it's extremely high quality. (And before you point Doom9's tests, Doom9 has NOT been configuring 3ivx correctly, so he's shooting himself in the foot). It has a 'trial' codec free (no real restrictions, but only for 'trial use') available for both Windows, Mac, and even BeOS (older versions are available for Linux and other platforms).
Referring about Sorenson, keep in mind this is the FREE codec that comes with Quicktime Pro. Professionals use a several hundred dollar 'Developers Edition' with Cleaner (two pass VBR encoding, which makes Sorenson rock). I know this is a for-user comparison, but in the professional scene, Sorenson can be even better then third/second place in quality.
If I considered that as their worst methodological flaw, I'd tend to agree with you completely.
However, for those who haven't bothered to read the article, two points (well, more than two, but as examples...) stuck out as completely invalidating their results:
and...
Anyone else spot the problem, here?
First of all, starting with lossy source material automatically injects artifacts into the video. A codec that looks for similar ways to trim bits as the original (MPEG2), ie, MPEG4, will natually have a distinct advantage in having fewer artifacts in the final result. Not that I can think of any means by which they could have obtained high-quality "raw" video, but any valid test of an encoder's capabilities would require them to do so.
And, as if that didn't introduce enough bias, they then reencoded in Indeo 5 format, before using each "real" codec under consideration. Again, that injects its own artifacts, and favors codecs that look for similar ways to trim bits. But, all four of the codecs tested can deal with MPEG2 as source material, so even the "to make it fair" argument falls flat here.
Overall, this so-called "comparison" has zero external validity, in the strict experimental sense. They managed to waste a few hours of CPU time, nothing more.
At the very least (if they couldn't get ahold of raw HQ video), going straight from MPEG2 would have given a meaningful comparison of "how it will look ripped from a DVD". But by the Indeo pass, they removed even that as a possible claim.
Hey, I just noticed, that's like diVX spelled backwards!
Now that's clever.
Or does it stand for something like:
X Vid Isn't Divx
"Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
I didn't read the article, but after looking at the chart, it is clear that Monsters, Inc. clearly beat out those other movies. It's over 4x faster than Spiderman.
One big factor which is neglected in this article is compatibility.
I don't think this was neglected at all; it clearly influenced the choice in what codecs to test in the first place. The four codecs in this article are the four most popular - wmv comes standard on PC's, QuickTime is Apple's standard and works across all major platforms, Divx is still the non-Linux geek's codec of choice (and it works with WMP) and MPEG-4 is now supported in the latest QuickTime.
The codecs chosen for review, then, are the ones that work with the players used by the greatest number of people. A lot of earlier posters complained about this or that codec not being included here, but they obviously missed this pretty critical point of the article. It doesn't matter to me, as someone working for a commercial enterprise that has to encode videos for our customers, whether Xvid or whatever offers slightly fewer artifacts. Because the fact is my customers probably don't have that codec and aren't going to bother downloading it just for me. Even Divx is probably barely at the saturation point where it's worth covering in an article like this, but for certain purposes and for a certain audience (PC gamers, for example), it's worth considering.
As others have pointed out, there are articles out there dealing with the lesser-used codecs if you just want to know who the absolute quality winner is. But in the real world and unless you're encoding video only for yourself, whatever codec wins in absolute quality is basically irrelevant. What matters is which codec offers the best quality among those in widest general use, and I thought this was a decent article on that basis (though in all honesty simply seeing the examples is probably good enough - I don't need an explanation of how blocky MPEG-4 is in an image, I can see it myself).
And it seems to me that what this article is saying is that if you want to use a cross-platform codec that everybody probably has (even on Linux), use plain old QuickTime. If you want to encode for the geek crowd, use Divx. If you want the best quality overall and you don't care about excluding a small percentage of the audience, use WM9. Whatever you do, avoid MPEG-4. Simple, and helpful to any professional whose job includes either encoding or contracting out encoding of videos for customers.
One thing the article and most people here seem to miss is DivX *IS* MPEG4. XviD is as well - that's why a MPEG4 decoder like ffdshow can play them both.
The article can really give people the wrong idea - it's not the MPEG4 codec, but maybe Apple's implementation that's to blame. Perhaps it just doesn't support all of MPEG4's features. Then again, perhaps the people doing the review just didn't know how to set up the encoder properly. Regardless of codec, there's quite an art to good encoding.
Okay, so I AM the world's leading expert on video compression codecs and formats (no, really, I am). I cover the same ground in my book, and in a series of articles for DV magazine over the last five years. So I'm pretty picky on this kind of things. But these guys couldn't compress themselves out of a wet paper bag.
Some fundamental errors:
They're using MPEG-2 sources, which are already highly compressed (this has been amply covered by other posters).
They talk about converting to an "uncompressed" AVI, but never specify which flavor of uncompressed. They should have used a lossless codec that uses the native Y'CbCr color space of video, like Huffyuv. They way they just said "uncompressed" suggests they used the AVI "None" codec, which is uncompressed RGB. This causes two lossly color space conversions - one from the Y'CbCr of the source to RGB, and then back to Y'CbCr in the delivery codec.
They used Indeo 5.1 as their intermediate codec. This is terrible. Indeo uses what's called YUV-9 sampling. There is only one measurement of color per 4x4 block of pixels. This throws away 75% of the color information from the DVD (which uses 4:2:0 sampling, with 2x2 blocks), before it even touches a codec. And this results in very ugly blocks whenever there are highly saturated regions with sharp contrast. So, all the output is going to look highly compressed when rendered from these intermediates, even if further compression is lossless. Look at the Spider Man test frame for an example. Notice the red blooming around the shoulders of the vocalist. And the color everywhere is very muddled. Indeo can also be slow to decode, unless it was encoded with all keyframes. And how slow it is to decode will vary with the tool, which probably added measurable error to their encoding time measurements.
They don't know the difference between Sorenson Video 3, which comes free with QuickTime, and Sorenson Video 3.3 Professional, which you have to pay for and is what Apple uses for their movie trailers. With the Pro version, critical features like B-frames and 2-pass VBR are available.
Apple's MPEG-4 encoder isn't very good - 1-pass only, tuned for speed more than quality. A file with the exact same compatibility could be made with Squeeze, Compression Master, Envivio, etcetera with MUCH better quality. And the Divx MPEG-4 codec is, of course, also MPEG-4.
They didn't use 2-pass encoding! No quality-concious encoder would ever put content on spinning disc without using 2-pass. And they didn't mention most of the other encoding settings they used, which by context I'd guess were basic defaults.
That's from an initial skim. If I spent more time with the article.
In summary, these guys spent hours and hours analyzing the results of tests, where they would have been WAY better off spending an hour asking someone who knew anything about video compression how to administer this kind of test.
Oddly enough, their results are vaguely like you'd expect - WMV9 and DivX do well, Sorenson less so, and Apple MPEG-4 at the rear. Done properly, I imagine WMV9 would have had a slight lead, and Sorenson 3 Pro would have been a lot closer to DivX. And no one uses Apple's MPEG-4 codec for content distribution. QuickTime's decoder is fine, so folks would use a professional-grade MPEG-4 encoder instead.
My video compression blog