Slashdot Mirror
ExtremeTech Wages War of the Codecs
prostoalex writes "ExtremeTech tested Windows Media, DivX, QuickTime/Sorenson and QuickTime/MPEG4 codecs. They encoded clips from Matrix Reloaded, Monsters, Inc., X2 and Spider-Man. QuickTime/Sorenson won the encoding speed contest, for the quality tests read the entire review, as each movie sample was encoded with 500KB and 1MB bitrates. Video samples provided on the site as well, so see for yourself."
They expect to have their opinion valued on Slashdot when they don't review the open source video codec? (It generally wins in other tests.)
a/.ing "video samples provided on the site as well, so see for yourself."
Crushing dreams at the speed of sarcasm
... pale in comparison to ASCII-mation.
Episode four in under a meg!
"Video samples provided on the site as well, so see for yourself."
/. Good one!
Yeah, as if there was any chance of THAT happening after you submitted that site to
-Erwos
Plausible conjecture should not be misrepresented as proof positive.
Bah, they should have used the nipple scene from Spider Man. The rest of the movie was a total wash.
"Have you ever thought about just turning off the TV, sitting down with your kids, and hitting them?"
I took a look at there sample images and found very little difference (other than MPEG4 obviously) in their quality.
It is very difficult for ME to decide between them. I have never actually seen any QT movies up for download as far as real movies go. Most movies are encoded with divx and seem to work just fine.
Do people really care about minor differences in quality when the file sizes are down to 710mb? I know I don't. Blurred motion is just something I deal with when I download something.
Encoding time is important only if you do this regularly. For those of us just watching a movie it doesn't matter. Whatever gives me the smallest file size with a decent picture is what I want to go w/.
Thanks to the hard work of the Mplayer team, I can play any video format I want. If you havent tried it, you should.
Get mplayer
found here
How do they expect me to keep pirating Hollywood movies if they keep changing the damn codec?!
Dont want to piss off the BSD crowd either!
I'm suprised XviD, an open source, MPEG-4 compliant codec wasn't tested. It's quickly becoming a standard for the transfer of large movies, and its open source nature has all of the usual benefits: alternatives, power and no constraints or adware. I suggest anyone planning on encoding video seriously considers it. XviD.org
I wonder how much better the QT/Sorenson and QT/MPEG4 (and maybe divX, dunno if there's an encoder) testing would have been if they were done on a Mac and the Velocity Engine could have been utilized?
Because Windows Media wins the quality shootout, they say "check the site". You have to know that if DivX won the quality tests, it would be in all caps in the headline! Ha!
Moderators, wake up!
If you do check the site you will see that Windows Media didn't win - it was a toss up.
OK, the article blurb claims that QT/Sorenson had the fastest encoding times, but also had the third-worst quality (only QT/MPEG-4 was worse). DivX seems to have the best quality, which, in my opinion, should be the end goal.
Think about it, how many times are you going to encode a movie? How many times are you going to watch it? Typically, you are going to encode once and probably watch it multiple times. Therefore, I would happily accept a little longer processing time in the beginning if that means I will end up with a better quality production.
Here's their most recent codec shootout with 3ivx, Divx, ffvfw, Nero, Real, On2 and Xvid. Xvid wins.
A useful site for all things high(ish) quality video encoding, aimed at dvd backups to cd, is Doom9 - see their last round of codec comparisons. (Frame based, so you'll need to click through from the beginning to get the menu frames etc.)
You know you've been IMing too long when you almost say 'lol' out loud to a non-geeky friend...
Posting still images isn't the best way to point out video artifacts due to compression. Post five seconds of compressed material (all of this qualifies under fair use) and let the users see the artifacts themselves. The human eye is much more likely to spot the artifacts in a movie because of our perception of motion.
Sorenson Pro (which has 2-pass and VBR encoding) isn't available in the $30 QTPro package. Use Sorenson Squeeze or MediaCleaner.
Also, QuickTime's MPEG-4 encoder is not the best MPEG-4 encoder out there. But there are better ones available, and of course MPEG-4 being a standard, the output of those other tools will be playable in QT Player.
So to make the comparison valid, both in terms of encoding speed an quality, some other tool should've been used.
What a wishy-washy article. To sum up and save you the 2 minutes of your life to read that article, all 4 techs are good, and they are all good for something, bad at others. I wonder if the author could have sat on the fence any MORE when comparing the codecs.
I for one, will continue to obey my DivX Masters, they have always been good to me. It seems that the author had a hard-on for QuickTime and M$, both of which annoy the ever living crap out of me... QuickTime, with it's little icon in the toolbar that just won't go away, and Microsoft because I just can't trust them to not tell MPAA which movies I have on my HDD.
$0.02 Flamebaiting, Trolling response concluded.
(and my Karma just got back to Neutral, dang it)
Check out the best P2P sharing website: MEDIACHEST.COM
Episode 4 in Ascii-mation
Come on! Are we now suddenly supposed to actually read those frickin' articles? Just tell me who won. This is the internet and my attention span ... wait, what's that shiny thing? ...
You only need to look at the scene to know what codec is in this month. It's Xvid by a long margin, most TV-shows (Stargate, Enterprise, whatever), DVD-rips, Anime etc. are encoded with it. MP3 and AC3 are predictably dominating the sound codec, with many TV-shows now also making the transition to full AC3 (well, the DVD rips of them at least).
Test of some scenes from copyrighted movies? What're the results for? What format is best downloaded off of Kazaa?
And the samples are all live action.. Test encoding some hand drawn animation (ie; an old bugs bunny), a computer generated animation, a anime style animation, a dialogue type scene, a live action scene with a lot of action, black and white vs color, etc, etc.
The types of images on screen greatly affect the performance of different algorithms.
Plus, each codec has about a million tweaks and optimizations for different types of footage.
I doubt highly that there's one clear "winner". It's really not that simple.
Which is why I hate sites like ExtremeTech that always have to boil it down to "this product is the best, the rest suck!".
Like the ATI vs nVidia flamewars. ATI may benchmark faster, yet nVidia has effects in games ATI lacks. There is no clear "this one is the best". Or Intel vs AMD or Linux vs Windows, etc, etc..
Nothing in the realm of computer science is that simple.
I don't need no instructions to know how to rock!!!!
Unfortunately, due to the fact that the source was itself the result of mpeg encoding, it could unfairly impact the ability of the various codecs to handle the content. On the flip side, much of the content people are encoding is actually decoded content, i.e. from a digital camcorder, etc.
What would be interesting is taking the original raw film footage (that hasn't been digitally compressed with a lossy method) and encoding, then comparing the results.
As sad as it sounds, I'm not surprised that XviD was left out. After all, alot of these reviewers pay attentions to what is being *marketted*. But I am surprised that On2 failed to get their VP4 Personal Codec noticed by this reviewer. I guess On2's marketting group dropped the ball when it came to make ExtremeTech aware of it's offerings.
Did you read the *whole* article? They state very plainly at the end that "DivX encoded clips tended to have a touch more detail, but also a few more compression artifacts, than the WMV9 video" and that DivX encodes much faster than WMV9. In brief, the only reason for choosing WMV9 over DivX is that it may be included in upcoming consumer devices.
Good people do not need laws to tell them to act responsibly, while bad people will find a way around the laws-Plato
Ok. encoding speed is somewhat important to a few people and bitrate is pretty important to most people but, quality is the most important to almost everyone. From a quality standpoint DivX is the clear winner. But, it still isn't broadcast quality let alone DVD quality.
Aren't there any 'Stock-footage' type DVD's out there that include a license to redistribute they could have used? Or couldn't they at least have tried to get permission for the clips they used?
While they might not have want to try to argue fair use through education or reviewing, they could have found at least one clip they could distribute. Hell, rent a high end digital camera and make one. Tape traffic on a highway, both daytime and nighttime, and you've got a motion video test, or a fountain, or anything.
R: That voice. Where have I heard that voice before? B: In about 365 other episodes. But I don't know who it is either.
Count me in as surprised at how poorly Apple's MPEG-4 implementation did. However, as a very new codec I expect it will improve in time. Or Apple will simply license someone else's codec.
Regardless, Apple has been one of the biggest supporters of MPEG-4, and I thank them for that.
I don't know what kind of crack I was on, but I suspect it was decaf.
...is which codec is best for encoding pr0n???
Quicktime is not a format, it is an architecture.
Extremetech REALLY blew it.... even in the apple world quicktime pro is known to be a poor ENCODER. The architecture is not the problem, it is the programs... Those beauitiful trailers that are highly compressed are Qicktime, but they are encode3d in Sorenson 3 using another program... It's called "Cleaner" by CreativeMac...
Extremetech REALLY REALLY blew it... I have never had such bad results when i used quicktime pro, (before i asked around how come I couldnt get the amazing detail of the trailers and was told that they're done in Cleaner)....
again, WMA is a codec, Quicktime is an architecture (thus, useing the Sorenson 3 codec)...man, I am firing off a letter to them for incompetence...
Well, seeing how bad ET's iTunes Bad, WMA Good article was, I figure Doom9's codec comparison is better than this.
And yes, Doom9's comparison includes XViD.
I'm sorry, but some hand-waving, subjective "Hey, this thing kinda looks better than that thing" is not a test. Calling it a "War of the Codecs" is even more ridiculous.
I like having all my movies and music and shows just a mouse click away. No fondling media, no DVD drives whooshing and movies stuttering halfway throgh because some tiny piece of schmutz got on the precious disc. In order to do this, I don't care at all what 500kbs or 1mbps files look like - The Twins effect alone occupies about 2GB on one of my drives, and I still haven't been able to produce a rip of Natural City that satisfies me even when the last one I tried was nearly 4GB (lots of film grain in that one and I don't care to lose it).
Yes... many of us care about quality. In fact, this is the very reason I rip DVDs - so the programs I enjoy play (more smoothly) from my hard drive.
STUPID! YOU'RE SO STUPID!!!
Karma: Excellent (fuck, even in the future moderation doesn't work!)
I've read -- on Apple's web site a year ago, incidentally -- that they don't use just one codec when producing the beautiful movie trailers on their web site.
Several codecs may be used to produce a single movie trailer, with different codecs being employed where their relative strengths are required: low motion versus action versus bright scenes versus dark scenes.
These guys are WAY more sophisticated in their technique than any home user will ever be.
Lesson: Admire Apple's movie trailers but don't think you're going to reproduce their quality.
--Richard
Okay, I just encoded some DVD-size video at 1mbit and 500kbit, 1/4 size, in QuickTime MPEG-4 and can barely see any artifacts in either. This dude seriously got some settings wrong in his MPEG-4 encodings, although I don't quite see how that's possible as settings are quality, framerate, keyframes, and data rate (and he said quality was set at best). I'll post some screenies later if I get a chance.
The reality of life is that WMV9 is one of the most, if not the most, used codecs around. Therefore it would have been poor testing and irresponsible reporting to have excluded it from the test.
What you suggest would be like a round up of office suites that tested Open Office, WordPerfect, Star Office and KDE Office but, didn't include Microsoft Office. You can't do that and expect to taken seriously.
On the other hand, their result was that WMV9 was the overall winner. My testing is based on what is most important to me. Quality. From a quality perspective I felt that DivX was the clear winner. Of the examples they gave, DivX was clearer and retained more detail than WMV9 in all but a very few cases.
I suspect that high quality wasn't enabled, which (IIRC) means that post-processing was disabled in the Apple MPEG4 decoder.
3ivx, Xvid and divx all postprocess, not unreasonably. The Apple codec makes itself look bad for no good reason.
Slashdot looked deep within my soul and assigned
me a number based on the order in which I joined
The article's not very great. First, they convert the MPEG-2 stream to INDEO, then from INDEO to whatever the target is. Fine, but the process isn't a transitive one. Some codecs will not produce as good an input for other codecs, thus biasing the results.
What's particularly suspicious is that Apple's MPEG-4 came out so poorly, though WMV9, and DivX are nothing more than early MPEG-4 codecs. Sorensen3 is the only substantially different algorithm used. And why use MPEG-4? It was originally designed for low-resolution low-bitrate applications (PDAs, cell-phones, etc.)
Why so slow? I do most of my video transcoding under Linux, but they aren't getting much better throughput than I do, and their machine's at least 4 times as fast as mine? I suppose it's got to do with using Indeo (my source is DV), so there's an extra decode step, but it's still quite slow.
I've distributed a number of my own videos in the MPEG-4 format, and don't see the sort of horrible results they demonstrated in their examples -- but then again, perhaps I do preprocessing (quantization, denoising, etc.) that they don't include in their process.
Regardless, my personal experience is that at high or low bitrates, most of the codecs are interchangable. Perhaps you need to fiddle with the encoding parameters, but you can almost always get results close enough to identical as not to matter. It becomes more difficult with mid-range bitrates (2-3Mbps@720x480x29.97) that some codecs show strengths over the others. In that department, I almost always go with MPEG-2 with custom quantization matrices...
I mean MPEG-4, despite being open, is NOT a free format. You are required to pay licenses for encoders and decoders. Has XviD payed this? If not, it's technically not legal. That could keep it out of being a serious contender for pro use. I mean I'm betting the MPEG-4 group isn't going to care if some hobbiests are using an unlicensed encoder, but they'll care if pros are.
Does it tell you which codec is best? Maybe but only for recompressing MPEG-2 footage. They *should* have tested against DV output as the standard consumer format, and uncompressed video. Plus looking at snapshots of compressed movies is of limited value, there is a big difference between what detail we can determine through a still image and a moving one. If you were to freeze a tv picture (or look at a captured frame) which includes something moving you would see a combing effect of the interlaced video. It doesn't look like that when you view it though.
Don't you know, EVERYTHING is better on a Mac. Heck, put an Apple logo on your glass, your drink will taste better :).
In all seriousness it comes form the fact that many Mac users toss around apple marketing terms (like Velocity Engine) without understanding what they mean (it's a floating point vector math unit, like 3dnow or SSE2). They just assume it makes things better since that's what the hype claims.
Wow.. that is upset.
I remember when legal used to mean lawful, now it means some kind of loophole. - Leo Kessler
Apologies, my reply was a little brief (preview, dammit). Quicktime tracks have a 'high-quality' flag, which I guess is supposed to hint the relevant decoder that, if possible, spend a little extra time making the track look good.
I've never seen it used in practice. The 3ivx codec, for example, adjusts it's post processing according to the available CPU which seems more sensible.
Slashdot looked deep within my soul and assigned
me a number based on the order in which I joined
Apple's MPEG-4 has both a mediocre encoder and decoder. 3ivx is a much better choice, it both decodes MPEG4 (and Divx/XviD) with better postfilters (the best, argurably), even scaled to how much CPU time it can grab. Its encoder is extremely nice, and very userfriendly, and it's extremely high quality. (And before you point Doom9's tests, Doom9 has NOT been configuring 3ivx correctly, so he's shooting himself in the foot). It has a 'trial' codec free (no real restrictions, but only for 'trial use') available for both Windows, Mac, and even BeOS (older versions are available for Linux and other platforms).
Referring about Sorenson, keep in mind this is the FREE codec that comes with Quicktime Pro. Professionals use a several hundred dollar 'Developers Edition' with Cleaner (two pass VBR encoding, which makes Sorenson rock). I know this is a for-user comparison, but in the professional scene, Sorenson can be even better then third/second place in quality.
What I find interesting is the lack of information on how much processing power is needed to decode the content. For instance, I have some QT-Sorrenson-3 video that looks incredible, but requires too much heft to decode on my lowly iBook/500MHz. The MPEG-4 version of the file does not look as good, but the video DOES at least play.
I think it is VERY important to understand the target platform where the content is to be decoded. If it's set-top box, PDA, or mobile phone, then I'd imagine MPEG-4 would be more appropriate (it's also great at streaming). QT-Sorrenson-3 is more targeted towards the desktop.
I have tried using DiVX and Apple's MPEG4 to encode a short video and there are just far too many options to play with. It's virtually impossible for the average person to use any of these and get great results. We need something that will produce excellent results at the click of a button. Until then I'll stick to showing my digital videos saved back to the camera, plugged into a TV, where quality is fantastic.
--- What?
It was pokies.
With nipple you get skin
With pokies you get shirt.
The other codec missing from this test is real media. Now I know I know, "It's the Devil" but I've seen some pretty good results and now with real alternative around it's not too bad.
I mean I can fit 5 seasons of Daria on a cd and it's watchable. And there is tonnes of old anime kicking around in it.
I didn't read the article, but after looking at the chart, it is clear that Monsters, Inc. clearly beat out those other movies. It's over 4x faster than Spiderman.
in spiderman.
Kirsten Dunst's wet t-shirt scene.
If you always encode to 650MB or 1300MB/Movie, then yes.
But if you want to use disk space efficiently, you get too big file sizes for easily encoded movies and too bad quality for hard encoded movies.
IMO, nothing beats quality based encoding, ie. you specify a quality setting and the movie will have whatever size is needed for that quality.
As soon as the CD dies as the major storage for movies (being replaced either by hard-disk or DVD), we will hopefully see more focus on quality-based encoding and less on bitrate-based, because it's pointless.
The bigger question is how taking a compressed format (MPEG2) compressing it further (Indeo) and using that as a source is a good test. Each different type of video compression create artifacts that are unique to that algorithm. When you re-encode with a different codec you can have distortion that is amplified by the *beating* of the algorithms. Different codec's will react in different ways to source that has a specific kind of distortion.
To make a comparison of codec's based on an MPEG2 compressed source is justifiable from the standpoint that we are likely to be ripping DVD's. However, I very much doubt that we will rip down to an interstitial Indeo format before doing the final compression. The fact that they didn't separate the compression time test's from the compression quality tests is suspect. They say they didn't want to contaminate the test with disk access, but disk access times would have been the same for all of the codecs and would have modeled real user usage.
I would not be surprised that SV3 and MPEG-4 have a bad interaction with Indeo compression, or at least Indeo compressed MPEG2. They should have used the original MPEG2 source at a minimum, and ideally uncompressed source.
XviD is not "just as good as DVD but at 700 megs". Be reasonable. XviD, DivX and QT/MPEG4 are actually close relatives, they all "speak" MPEG4 "dialects".
I played with a lot of different codecs, including MPEG4-like mutants such as DivX, XviD, ffmpeg, etc. If i limit myself just at comparing DivX and Xvid, then:
- XviD is slightly faster than DivX, all else being equal
- XviD has slightly better quality than DivX, all else being equal, but it's an extremely close call (and sometimes the opposite is true)
So, in the Extremetech benchmark, if you replace DivX with XviD, it would fare slightly better overall. But definitely nothing as ridiculous as "owning the competition".
Facts please, not emotional knee-jerk reactions. Thank you.
Thanks to the hard work of the Xine team, i can do the same, but using more front-ends and a more flexible architecture.
http://xinehq.de/
I am very suprised that the Sorenson ended up so bad. Generally when you watch Soreneson encoded stuff it is very very clean... IE most movie trailers. I remember the very first few of Sorenson on an apple 8.5 cd with a Bare Naked Ladies video, we blew that thing up and stop framed and found almost no artification. Generally I use sorenson, and end up with much better results.
---In a time of Chimpanzees I was a Monkey.
I know this was strictly about quality - but i think its very shortsighted to miss some of the key features WHY people like Quicktime... and its not always about the quality of watching ripped DVDs.
For example - I don't see anywhere where it points out that Quicktime and Divx are by far the most DRM-less codecs out there. WMP9 can stick you up the ass if you're not careful. Plus, there are plenty of times that WMP9 will refuse to operate properly with multi=monitor setups (my friend's brand new ThinkPad, for example, refuses to play over the external VGA port....)
i also don't see any mention of the ability to cut/copy/paste with the built-in default players as a comparison tool. How many times have you wanted just a sliver of a movie to playback - ro to have the ability to quickly convert it to DV to put onto a workflow with some other editing? Even the average goofball making iMovies wants to do that all the time - but is 100% prevented from doing that with WMP9
While the quality of QT is obviously lacking - i'll use it EVERY DAY OF THE WEEK because its far more powerful for everyday use, and much more free of DRM issues.
this would explain, of course - Hollywood's facination with it - its got great quality while sacrificing little things... like fair use.
guns kill people like spoons make Rosie O'Donnell fat.
Just to clarify, Quicktime is a media architecture, not a file format or a codec.
This misunderstanding doesn't invalidate your argument, although I would disagree with you about MPEG-4. I've gotten good results with it, sometimes even great results.
It's not offtopic, dumbass. It's orthogonal.
From the QuickTime summary:
Pros: decent image quality
Cons: Horrible image quality
The codec itself is neutral from any copy protection mechanism, or you just like to yell "DRM" for some cheap mod points.
Okay, so I AM the world's leading expert on video compression codecs and formats (no, really, I am). I cover the same ground in my book, and in a series of articles for DV magazine over the last five years. So I'm pretty picky on this kind of things. But these guys couldn't compress themselves out of a wet paper bag.
Some fundamental errors:
They're using MPEG-2 sources, which are already highly compressed (this has been amply covered by other posters).
They talk about converting to an "uncompressed" AVI, but never specify which flavor of uncompressed. They should have used a lossless codec that uses the native Y'CbCr color space of video, like Huffyuv. They way they just said "uncompressed" suggests they used the AVI "None" codec, which is uncompressed RGB. This causes two lossly color space conversions - one from the Y'CbCr of the source to RGB, and then back to Y'CbCr in the delivery codec.
They used Indeo 5.1 as their intermediate codec. This is terrible. Indeo uses what's called YUV-9 sampling. There is only one measurement of color per 4x4 block of pixels. This throws away 75% of the color information from the DVD (which uses 4:2:0 sampling, with 2x2 blocks), before it even touches a codec. And this results in very ugly blocks whenever there are highly saturated regions with sharp contrast. So, all the output is going to look highly compressed when rendered from these intermediates, even if further compression is lossless. Look at the Spider Man test frame for an example. Notice the red blooming around the shoulders of the vocalist. And the color everywhere is very muddled. Indeo can also be slow to decode, unless it was encoded with all keyframes. And how slow it is to decode will vary with the tool, which probably added measurable error to their encoding time measurements.
They don't know the difference between Sorenson Video 3, which comes free with QuickTime, and Sorenson Video 3.3 Professional, which you have to pay for and is what Apple uses for their movie trailers. With the Pro version, critical features like B-frames and 2-pass VBR are available.
Apple's MPEG-4 encoder isn't very good - 1-pass only, tuned for speed more than quality. A file with the exact same compatibility could be made with Squeeze, Compression Master, Envivio, etcetera with MUCH better quality. And the Divx MPEG-4 codec is, of course, also MPEG-4.
They didn't use 2-pass encoding! No quality-concious encoder would ever put content on spinning disc without using 2-pass. And they didn't mention most of the other encoding settings they used, which by context I'd guess were basic defaults.
That's from an initial skim. If I spent more time with the article.
In summary, these guys spent hours and hours analyzing the results of tests, where they would have been WAY better off spending an hour asking someone who knew anything about video compression how to administer this kind of test.
Oddly enough, their results are vaguely like you'd expect - WMV9 and DivX do well, Sorenson less so, and Apple MPEG-4 at the rear. Done properly, I imagine WMV9 would have had a slight lead, and Sorenson 3 Pro would have been a lot closer to DivX. And no one uses Apple's MPEG-4 codec for content distribution. QuickTime's decoder is fine, so folks would use a professional-grade MPEG-4 encoder instead.
My video compression blog
VP4? I don't believe that was ever released. I had a review copy of it, but they quickly superseded it with VP5.
VP3 was the one that was open-souces, and is used as the basis of Ogg Theora.
The current On2 codec is VP6, which is free for personal use.
My video compression blog
WMV9 isn't MPEG-4 derived. MS MPEG-4v3 was, but that forked into WMV7 years ago. WMV9 is quite different than MPEG-4 now.
QuickTime encodes and decodes Simple Profile MPEG-4
DivX did Simple in V4, and V5 added support for Advanced Simple.
Most of this will be moot soon, since the MPEG-4 Part 10/AVC/H.264 codec is way better than the old Simple or Advanced Simple, and will rapidly replace the old versions in the next couple of years.
My video compression blog
Xvid and 3ivx get mentioned many posts further down, but no one says anything about ogg!!! From my limited experience, it has amazing quality and the greatest advantage over xvid and that damnd WMV9 is that it is instantly seekable. No lag while xvid thinks and no buffering by windows. I personally use media player classic for my WMV's & get instant seeking, but his target audience doesn't.
All hail ogg!
[Fuck Beta]
o0t!
... but new ffdshow has been released quite recently (I got it only 2 days ago). Now VFW interface and audio decoding(!) are included. I have already got used to volume normalizing (yahoo!!! :-)