VP8 and H.264 Codecs Compared In Detail

An anonymous reader writes "Moscow State University's Graphics and Media lab have released their sixth MPEG-4 AVC/H.264 video codecs comparison. Also of note is a recently added appendix to the report which compares VP8, x264, and Xvid. The reference VP8 encoder holds its own against x264 despite the source material offering x264 a slight advantage. The VP8 developers comment in the report: 'We've been following the MSU tests since they began and respect the group's work. One issue we noticed in the test is that most input sequences were previously compressed using other codecs. These sequences have an inherent bias against VP8 in recompression tests. As pointed out by other developers, H.264 and MPEG-like encoders have slight advantages in reproducing some of their own typical artifacts, which helps their objective measurement numbers but not necessarily visual quality. This is reflected by relatively better results for VP8 on the only uncompressed input sequence, "mobile calendar."'"

Comments

In the real world

[DrXym]

Input content *will* usually have been compressed with H264. Even the likes of Google will find itself transcoding 99% of its content into VP8 from some other codec. That might suck for comparison tests but its a fact of life.

Re:In the real world

[westlake]

For now at least.

You have to be realistic.

H.264 isn't just about the cell phone phone and the web.

It's a broadcast, cable and sattelite video standard, a Blu-Ray standard. It is deeply entrenched in industrial and security video.

A search of Google Shopping for "H.264" will return 40,000 hits.

There are 847 AVC/H.264 Licensees

The Asian industrial giants like Mitsubishi are very well represented.

Misleading statements

[Lunix Nutcase]

Unfortunately their statement is very misleading considering how VP8 and H.264 and other MPEG codecs use basically the same transform so their statements of bias against VP8 ring untrue. One of the professors who was part of doing this test even confirmed that the VP8 developers statement was untrue and misleading.

Re:Misleading statements

[unix1]

First, that claim was made by an x264 developer. Second, that comment in itself is misleading. VP8 developers didn't claim the bias affected visual quality. Here's the exact quote:

H.264 and MPEG-like encoders have slight advantages in reproducing some of their own typical artifacts, which helps their objective measurement numbers but not necessarily visual quality.

x264 developers need to take it easy and let their implementation speak on its merits rather than attempt to discredit VP8 at every opportunity.

Re:Misleading statements

[unix1]

At what point will you "buy" the claims?

Is that a rhetorical question? Because it doesn't help their case that x264 developers have themselves made misleading comments about VP8 since it was introduced by Google.

here we have an independent expert disputing the claim that the VP8 developers made. Why isn't this expert good enough?

No that's not what you have. What you have is an x264 developer complaining about the comment made by a VP8 developer.

The professor replying to the complaining x264 developer agreed with the complaint but let the comments stand.

You just assume that they don't know enough about VP8, when this is his specialty. Why? I'm all for learning as much about a topic as possible before forming an opinion on it, but that doesn't mean you *have* to form an opinion on everything.

I didn't form an opinion about the claims either way, but merely stated the original VP8 comment was not misleading. It did not mislead me into anything.

The original poster of this thread, on the other hand, had actually taken the x264 developer's opinion as statement of fact when there was no evidence presented to support it. And based on that assumption, he formed an opinion that the VP8 comment was untrue.

At some point, you have to allow the specialists to do what they do. And if you have no idea what they are doing or talking about (like here), maybe you shouldn't bother being in the discussion, as you add nothing to it.

I always "love" these types of remarks where people are asked to shut up because [insert an assumption/generalization here]. If you believe I didn't add anything to the discussion, then you certainly didn't either - so why did you bother replying, or even reading to this point in this thread?

Re:Misleading statements

[evilviper]

VP8 and H.264 and other MPEG codecs use basically the same transform so their statements of bias against VP8 ring untrue

An x264 developer may have said it, but that doesn't make it true. He contradicts himself often enough, anyhow...

Yes, the macroblock size is the same, however, that's just about it... VPx codecs use rather different quantizer matrices, never use B-frames, store motion vectors differently, and in general just have a very different and unusual encoding method. There are many ways in which it is different, and more to the point, far more different from H.264 than H.264 is from MPEG-2... Of course it can be over-stated, but it's very, very real.

And BTW, what the hell is a "transform"? If you're going to pretend to be knowledgeable about a subject, you could at least have the decency to use precise terminology, so I can more efficiently ridicule you... If you're talking about the "T" in DCT, then I've got bad news for you. While nearly all lossy codecs are indeed using DCT, that's about as relevant as saying all vehicles burn fuel.

I'll see your x264 developer quote, and raise you one ffmpeg/Adobe Flash developer quote: http://multimedia.cx/eggs/dct-pr/

While he's talking about Theora, all points apply directly to VPx. Highly relevant quotes below:

"Theora is rather different than most video codecs, in just about every way you can name"

"As for the idea that most DCT-based codecs are all fundamentally the same, ironically, you can't even count on that with Theora- its DCT is different than the one found in MPEG-1/2/4, H.263, and JPEG (which all use the same DCT)."

But you can ignore him if you like. He's just the guy who actually wrote the VP3 and Theora decoders for FFmpeg, and has reverse engineered numerous other codecs, so I'm sure he doesn't have a clue... You know, unlike a vocal x264 developer...

And just to push further off topic, DCT isn't all that good, anyhow. The default in FFmpeg encoding isn't DCT, but SAD (Sum of Absolute Differences). SAD happens to be much faster, and more importantly, doesn't leave strangely colorful 16x16 blocks (usually green, but often red or other colors) as artifacts in low-bitrate encodings, which then have to be masked, wasting bits which could be better used elsewhere.

One of the professors who was part of doing this test even confirmed that the VP8 developers statement was untrue and misleading.

Facts aren't determined by popular opinion... And 2 people isn't much of a vote, anyhow.

VP8 holds its own?

[Anonymous Coward]

"The reference VP8 encoder holds its own against x264 despite the source material offering x264 a slight advantage."

Um, sure, if VP8 on its "best" preset being roughly equivalent to x264 on its "high speed" preset means it's holding its own, I guess that's a fair statement.

Re:VP8 holds its own?

[Sycraft-fu]

Well if your demand is "In every way as good or better," then no, VP8 is not that and probalby never will be. If that's your criteria go use H.264. Just don't come crying if you get smacked with a large licensing fee for your stuff.

However, if you ease off and say "Can it produce a good looking image, something of similar subjective quality, at a reasonable bitrate?" then yes, VP8 holds its own. So does VC-1 for that matter. In all cases you can encode a video at a similar bitrate and still have it look good. The H.264 will probalby look a bit better, but not so much as to really matter a lot. Also, perhaps more importantly in some applications, it is a similar bitrate when encoding artifacts start to drop below human perception. So if you need something that's "lossless" as far as the view is concerned, you are doing it in around the same amount of space. Sure, H.264 can squeeze it down a bit more, but not a massive amount.

To put it in sports terms they may not all be the best hitter, but they are all playing in the major leagues. This is as compared to an older codec like, say MPEG-2 or especially MPEG-1. For those, you need a substantially higher bitrate to look as good, and something even then it just isn't possible.

Reading Bitrate/Quality Graphs

[Anonymous Coward]

I don't understand the "VP8 holds its own against x264".... The graphs show that it certainly does not hold it's own against x264. For example, if you look at the best quality settings of x264 vs VP8 for the Ice Age clip, at the same quality (SSIM=0.97), x264 takes 800Kbps while VP8 takes ~1.2Mbps... So VP8 takes 50% more bits to achieve the same quality. This shows that VP8 is not nearly as efficient as x264. (Also, note that x264 is only one implementation of an H.264 encoder. There are other implementations that will make different tradeoffs to get better compression efficiency at the cost of performance).

Re:Reading Bitrate/Quality Graphs

[DJRumpy]

Agreed, this reads as if it's nothing more than a promotional ad for the VP8 codec. How do they expect everyone to take this seriously?

I don't understand the "VP8 holds its own against x264".... The graphs show that it certainly does not hold it's own against x264. For example, if you look at the best quality settings of x264 vs VP8 for the Ice Age clip, at the same quality (SSIM=0.97), x264 takes 800Kbps while VP8 takes ~1.2Mbps... So VP8 takes 50% more bits to achieve the same quality. This shows that VP8 is not nearly as efficient as x264. (Also, note that x264 is only one implementation of an H.264 encoder. There are other implementations that will make different tradeoffs to get better compression efficiency at the cost of performance).

Re:Near enough

[SheeEttin]

Since when did "near enough" become "good enough"? We might as all switch to Windows...

It's not just "near enough", it's "near enough and unencumbered by patents". (Of course, MPEG LA will contest that...)

Re:Very simple

[Shark]

Torsion is what you feel when someone twists your balls.

Extorsion is that lingering pain afterward.

Re:Near enough

[Klinky]

Well if you're into the FOSS philosophy, "near enough"(WebM/VP8) is better than "not at all"(H.264).

Also many cameras are MPEG-2

[Sycraft-fu]

While most of the card ones are AVCHD which is H.264, HDV cameras are MPEG-2. They are quite popular as there's reason to want tape as a storage medium.

Then of course in terms of pro video, it is still compressed, raw video is just too daunting to store, but again with different codecs. They are often takeoffs of DV where there is just very light per frame compression as well as chroma downsampling. That offers better quality on subsequent recompression and editing, as well as lower hardware requirements to encode.

This idea that everything will be H.264 as a source is inaccurate. It is popular no doubt, and I believe it will continue to be, but it isn't universal and probalby won't be.