Tighter Video Compression With Wavelets

RickMuller writes: "There is a Caltech Press Release here that talks about a new 3D video compression algorithm by Caltech's Peter Schroeder and Bell Labs' Wim Sweldens that they claim is 12 times smaller than MPEG4 and 6 times smaller than the previously best published algorithm. The algorithm uses wavelets for the data compression. Potential applications in real estate (digital walk-throughs of houses) are cited in the article. Anyone figure out a way to wire this stuff up to Q3 Arena yet? The results were presented in a talk at SIGGRAPH 2000 in New Orleans."

Comments

what this really means

[kashent]

This isn't going to make movies any smaller to download.
What it's going to do is make 3D worlds smaller to download.
It's not the compression technique that will allow you to view in complete 3D the inside of a house, but the fact that you can record a 3D model of a house and still have it small enough to download.
The biggest improvent would probably be for VRML type technologies. And it's not going to make quake faster, but it could possibly let someone on a 28.8 use a customized skin that can be quickly sent to all other computers. Most people download quake worlds before they start playing rather than on the fly. -Kashent

Careful

[captaineo]

This technique compresses 3D vertex information - not 2D video as the headline implies. It will be useful in sending high-res geometry across low-bandwidth connections. Two applications:

• Streaming 3D geometry over the Internet - fully buzzword compliant, but how useful is this *really* going to be?
• Sending compressed geometry to 3D cards - this is the truly interesting application, IMHO - compressed geometry may be the solution to feeding ultra-fast OpenGL cards, where bus bandwidth may soon become a limiting factor in geometry throughput

Better than MPEG4? Huh?

[David Leppik]

They are comparing polygon mesh compression with video compression. Sounds like apples-to-oranges to me. Also sounds like it will have no effect on video compression, and it will have limited impact on rendering time.

I say limited, because you still need to draw those polygons. However, one nice feature of wavelets (at least for images) is that you can easily extract just enough data for displaying at a particular resolution. If that property holds for polygon meshes, then you should be able to draw only as many polygons as are useful for your display resolution.

Already a better algorithm

[GangstaLean]

I have already developed an algorithm that is much better than this, using a technology called "bit stripping". It works on 3D worlds and movies.

I have included a sample of the technology compressing "The Matrix" below:

1

As well as Quake 3 demo:

0

Note: Also decreases viewing time, increasing the ability of the user to consume more media.

Re:Already a better algorithm

[Jerf]

I have contacted the RIAA and the MPAA, and have pointed them at this post, which contains so-called "bit stripped" versions of every movie and every song ever produced and that will ever be produced.

If Napster's damage can be measured in the trillions in a lawsuit, just imagine what you've opened yourself up to.

"Exadollars, and soon, petadollars. One thousand billion trillion dollars. How many lawsuits per second can your software handle?" with apologies to IBM

please forgive duplication if it occurs, I'm having trouble getting through

Full motion video over phone lines -- coming soon.

[Dr. Zowie]

I am very impressed with wavelet compression in general. The wavelet decompostion of an image isn't unique -- that gives the heuristics a litle more ``wiggle room'' to choose the most optimized representation of the image.

I just finished writing a proposal to NASA for some instruments on the Solar Probe spacecraft. That's a pretty telemetry-constrained mission. We tested a proprietary wavelet-compression algorithm at 50:1 on 14-bit images (yes, that's about a quarter-bit per pixel) and even at that level it's very hard to tell the difference between compressed and uncompressed images with the naked eye. (The algorithm seems to work by quantizing the sizes of features in the image).

At that level of compression, a 30Hz stream of 6bit-per-channel 640x480 images would only require just over 3Mbps of bandwidth -- and that's without taking any advantage of the relationship between frames. It's easy to believe that another factor of 50 could come out of a combination of more aggressive compression and either diferential encoding or 3-D wavelets. We could end up with full-motion, full-rate video being squirted through 60kbps connections.

Wavelet compression explained by an expert

[coult]

Okay, so all of the descriptions that people have given here for wavelet compression is wrong. I've got a Ph.D. in Applied Math and do research in wavelet compression of 3-D data (not geometry data, mind you, but 3 dimensional real data, like images, but in 3 dimensions instead of 2). The basic idea behind wavelet compression is the following:

In most natural or real-world data (i.e. images, geometry data, etc.) the information at a given point in the data is very highly dependent on the data at nearby points. Thus, there is a certain amount of redundancy in the data, and this redundancy is spatially localized. The concept in transform coding is to apply some transformation (either linear or nonlinear; the wavelet transform and Fourier transforms are linear) to this data to reduce the statistical redundancy.

Even after applying the transform, you haven't saved anything in terms of the space required to store the data; all you've done is change the basis used to represent the data. Now you take the transformed data and place it into a bunch of bins, each of which is identified with an integer. At this stage, called quantization, you are modifying the information present, because the best accuracy with which you can recover the data is given by the width of the bins. At this stage, you take the sequence of integers and apply a lossless coding scheme to it to reduce the number of bits required to represent the stream of integers. The compression happens at this stage. Wavelets do a better job than blocked discrete Cosing transform (used in JPEG) at reducing the statistical redundancy of the input data; thus wavelet-based image compression compresses more efficiently than JPEG.

What Schroeder and Sweldens have done is taken an a very general, widely applicable method for constructing wavelet transforms (known as the lifting scheme, invented by Sweldens) and adapted it for representing mesh nodes and connectivity information, i.e. geometry (which incidentally could just as easily be higher dimensional data). Thus they have a wavelet transform for geometry. They achieve compression by using the EZW coding scheme, developed for coding wavelet coefficients of images and used in the JPEG2000 standard, and applying it to their geometry wavelets.

It should be very nice for low-bitrate storage and transmission of geometry, as well as successive-refinement transmission (i.e. the 3-d data gets better and better looking as more bits arrive).