Slashdot Mirror
ExtremeTech Wages War of the Codecs
prostoalex writes "ExtremeTech tested Windows Media, DivX, QuickTime/Sorenson and QuickTime/MPEG4 codecs. They encoded clips from Matrix Reloaded, Monsters, Inc., X2 and Spider-Man. QuickTime/Sorenson won the encoding speed contest, for the quality tests read the entire review, as each movie sample was encoded with 500KB and 1MB bitrates. Video samples provided on the site as well, so see for yourself."
OK, the article blurb claims that QT/Sorenson had the fastest encoding times, but also had the third-worst quality (only QT/MPEG-4 was worse). DivX seems to have the best quality, which, in my opinion, should be the end goal.
Think about it, how many times are you going to encode a movie? How many times are you going to watch it? Typically, you are going to encode once and probably watch it multiple times. Therefore, I would happily accept a little longer processing time in the beginning if that means I will end up with a better quality production.
Unfortunately, due to the fact that the source was itself the result of mpeg encoding, it could unfairly impact the ability of the various codecs to handle the content. On the flip side, much of the content people are encoding is actually decoded content, i.e. from a digital camcorder, etc.
What would be interesting is taking the original raw film footage (that hasn't been digitally compressed with a lossy method) and encoding, then comparing the results.
If I considered that as their worst methodological flaw, I'd tend to agree with you completely.
However, for those who haven't bothered to read the article, two points (well, more than two, but as examples...) stuck out as completely invalidating their results:
and...
Anyone else spot the problem, here?
First of all, starting with lossy source material automatically injects artifacts into the video. A codec that looks for similar ways to trim bits as the original (MPEG2), ie, MPEG4, will natually have a distinct advantage in having fewer artifacts in the final result. Not that I can think of any means by which they could have obtained high-quality "raw" video, but any valid test of an encoder's capabilities would require them to do so.
And, as if that didn't introduce enough bias, they then reencoded in Indeo 5 format, before using each "real" codec under consideration. Again, that injects its own artifacts, and favors codecs that look for similar ways to trim bits. But, all four of the codecs tested can deal with MPEG2 as source material, so even the "to make it fair" argument falls flat here.
Overall, this so-called "comparison" has zero external validity, in the strict experimental sense. They managed to waste a few hours of CPU time, nothing more.
At the very least (if they couldn't get ahold of raw HQ video), going straight from MPEG2 would have given a meaningful comparison of "how it will look ripped from a DVD". But by the Indeo pass, they removed even that as a possible claim.
One big factor which is neglected in this article is compatibility.
I don't think this was neglected at all; it clearly influenced the choice in what codecs to test in the first place. The four codecs in this article are the four most popular - wmv comes standard on PC's, QuickTime is Apple's standard and works across all major platforms, Divx is still the non-Linux geek's codec of choice (and it works with WMP) and MPEG-4 is now supported in the latest QuickTime.
The codecs chosen for review, then, are the ones that work with the players used by the greatest number of people. A lot of earlier posters complained about this or that codec not being included here, but they obviously missed this pretty critical point of the article. It doesn't matter to me, as someone working for a commercial enterprise that has to encode videos for our customers, whether Xvid or whatever offers slightly fewer artifacts. Because the fact is my customers probably don't have that codec and aren't going to bother downloading it just for me. Even Divx is probably barely at the saturation point where it's worth covering in an article like this, but for certain purposes and for a certain audience (PC gamers, for example), it's worth considering.
As others have pointed out, there are articles out there dealing with the lesser-used codecs if you just want to know who the absolute quality winner is. But in the real world and unless you're encoding video only for yourself, whatever codec wins in absolute quality is basically irrelevant. What matters is which codec offers the best quality among those in widest general use, and I thought this was a decent article on that basis (though in all honesty simply seeing the examples is probably good enough - I don't need an explanation of how blocky MPEG-4 is in an image, I can see it myself).
And it seems to me that what this article is saying is that if you want to use a cross-platform codec that everybody probably has (even on Linux), use plain old QuickTime. If you want to encode for the geek crowd, use Divx. If you want the best quality overall and you don't care about excluding a small percentage of the audience, use WM9. Whatever you do, avoid MPEG-4. Simple, and helpful to any professional whose job includes either encoding or contracting out encoding of videos for customers.
Taking into account that they bash the Apple/Quicktime MPEG-4 quality, and that the article is published by Ziff-Davis (Who I consider to be the least credible source for information in the industry), you might as well be reading an article published by Microsoft about how great WMV is.
OH NO!!! The article didn't say that Apple is 100% awesome and they didn't replace the 's' in Microsoft with a dollar sign. That must mean that the article was written by Bill 'Son of Satan' Gates himself!!! Or it could just mean that the videos generated by the Quicktime encoder didn't look as good as the ones generated by the WMV9 and the DivX encoder. Look at the comparisons for yourself and realise that in this case the Apple tool was a distant third.