Slashdot Mirror
Breakthrough In JPEG Compression
Kris_J writes "The makers of the (classic) compression package Stuffit have written a program that can compress JPGs by roughly 30%. This isn't the raw image to JPG compression, this is lossless compression applied to the JPG file. Typical compression rates for JPGs are 2% to -1%. If you read the whitepaper (PDF), they are even proposing a new image format; StuffIt Image Format (SIF). Now I just need someone to write a SIF compressor for my old Kodak DC260."
It simply works on badly compressed jpeg files.
they are even proposing a new image format; StuffIt Image Format (SIF).
Gee, I wonder where you could license that format?
Man, they could have been a little bit more covert about their intentions and named it something a little less, umm, obvious.
The current formats might not be perfect, but at least they are (relativly) free.
"The Wright brothers were the first to fly with a heavier-than-air machine, but boy did they have a lousy plane"
For those of us who use DSLRs, 1-2 seconds is way too long. True enough, buffers help, but I wouldn't buy such a slow camera.
///<sig
And it effectively increases the size of a CDR used to store images to 1 gigabyte. Maybe not useful to everyone, but certainly more useful than a mere scientific curiosity.
If you read the whitepaper you will see that their algorithm is patent pending. The patent will almost certainly be granted, and, since no one else has done additional jpg compression before, it may even be deserved.
However, do we want to subject ourselves to a new tax on images? If they make it, we don't have to go! Just say NO to patented file formats!
I would go further: We don't need any new standards unless they are free of patents and open for use in FOSS.
Moll.
What you hear in the ear, preach from the rooftop Matthew 10.27b
Is he really going to be saving stuff in a lossy format like JPEG as a professional photographer though?
Archive a bunch of images sometime. Then it's useful. I needed to put several thousand images (scanlations of manga) onto a cd. It went over the 700mb limit. Using this, I could have saved $0.10 on cds and 2 minutes of time. Not a big deal, but if you do such things a lot, it could add up. So the program is probably worth about $5. Maybe $10, but that could be a bit much.
Not a sentence!
The linked page shows average decompression times of 6-8 seconds for 600-800 KB files, rising with the size of the file. Who would benefit from this?
Any websites with the primary purpose of hosting images would benefit greatly from this - such as art & photography sites. (Yes, and porn sites, too.)
Why? Because 99% of the traffic they generate is for their images. Of those images, 99% of them are in JPEG format. So this compression would give a good savings in bandwidth on all those pictures.
At large sites, a 30% cut in required bandwidth could mean a very large savings. Now, if they can take a large cut off their operating expenses, and all they need to do that is to make the users wait a few extra seconds for their pictures, I think we know what they'll do.
As for the decompression time, one thing to remember is that with how slow internet connections are (even broadband), you're much more likely to be waiting for one of these images to transfer than you are to be waiting for it to decompress (so long as it allows you to start the decompression without waiting for the end of the file, of course).
--The Rizz
"Man will never be free until the last king is strangled with the entrails of the last priest." -Denis Diderot
If they've really achieved 25-30% over jpg, and it looks like they have, then its a truly amazing invention considering that jpeg has been around for so long. It would save at least about ten dollars worth of space on every digital camera. If you look at the humongous image archives that NASA and other research projects generate and the cost of tape to store them all, we're talking tens of billions of dollars of savings here.
So, a question to slashdotters: do you think this kind of invention deserves to be protected by a patent? The standard response "software is already protected by copyright, patents are unnecessary" doesn't work, because anyone can study the code (even the binary will do), describe the algorithm to someone else, who can then reimplement it. Standard cleanroom process; takes only a couple of days for a competent team.
If you're RMS, you probably believe that no one has the right to own anything and all inventions and ideas belong to the public, but the majority of us will agree that that's a tad extreme. So whaddya'll think? Myself, I'm totally undecided.
This is not in competition with JPEG - more with LZW compressed TIFF.
.x% range, yet they've apparently improved by a factor of around 6x what the previous best programs were able to achieve. I should note that I developed the audio codec with the currently highest lossless audio compression - http://lossless-audio.com/ - so have some idea of what I'm talking about here.
You're misreading the whitepaper. What they're explaining here is different and actually quite clever.
Jpeg compression can be seen as consisting of two main steps. In the first, information in the image deemed unimportant is removed from the image. In the second, the remaining information is compressed losslessly.
It was already known that the second step is not the most efficient possible. Also, the jpeg standard supports both huffman and arithmetic-coding, however due to patent reasons I think arithmetic-coding (which is more efficient) is often not used. So what they're doing is just improving the efficiency of this second step. This works much better than attempting to encode the jpg binary itself, as anything performed following entropy coding will struggle to achieve much compression, as the data has been hugely complicated by the process and it is thus much harder to find any compressible patterns etc.
They've also improved the compression by a very impressive margin. Typically in the lossless compression world, gains are in the
Anyway, the end effect is the same regardless how the compression is achieved - they're taking a lossily-compressed jpg, and reducing its size. This obviously makes little sense to be say integrated into digital cameras, for which the jpeg2000 format is available anyway and features better gains and is much faster (yet is still awaiting mainstream adoption), however from the archival and theoretical-lossless-compression perspectives what they've allegedly achieved is pretty damn interesting.
To be honest i don't care about the 30% compression if there's the slightest danger that anyone might start a patent-war over the image-format or the compression algorithm.
I've really seen enough of that (gif, mp3, jpeg) and i prefer spending the additional storage/bandwith capacity to another uppity "IP-shop" coming out to "0wn0rz" the internet with lawyering (maybe after a management-change).
Let's have another look at that compression algorithm in 20 years or so.
"By the way if anyone here is in advertising or marketing... kill yourself." -- Bill Hicks
I concluded that it isnt practical for general use, it took too much time to compress an image (alright it was five years ago and so today it probably wouldnt matter), but most importantly there is no easy general compression solution for all images (for instance one that compresses tree pics well wont do faces well and vice versa).
For a general dip into fractal image compression try to get and read 'Fractal Image Compression By Yuval Fisher'. Damn good read.
Given that the software is likely to be proprietary and the algorithm will be patented, it becomes completely useless to me and it is completely unsuitable for archiving anything (smart people don't play nickel and dime games with archives and backups).
Maybe if computers were a lot faster and the compression worked on any array of pixels, not just those that have undergone the lossy transformation of JPEG compression. But even then, it would have to do better than what PNG can do in terms of all the other things PNG does and no patented format will beat PNG at its game.
In theory what you say sounds nice, but in practice I genuinely can't recall a situation where a little more compression would have allowed me to send all the photos I wanted to via e-mail. But the reasons I mentioned at the top of this post are more important reasons why this should be rejected out of hand.
What meager benefits this affords are far outweighed by the costs. I don't see this going anywhere, and for very good reason.
Digital Citizen
30%(even close to) is a very good reduction in filesize, when considering this is lossless, you cannot turn your nose up at this. People with alot of jpegs will want to use it. Creating a better jpeg compression algorithm with 50% (100% is impossible) filezize reduction, seems impossible. I would make the assumption that since jpeg is over 18years old, compression algroithms are at their limit for this image format.
Just a minor point here:
Fit more on CDs, DVDs, and other backup media
Anyone stupid enough to encode their data using a proprietary system deserve to lose all their data, IMNSHO.
With lossless wavelet compression. That's what I'm still waiting for. As the "2000" indicates it's nothing terribly new but as of yet still misses a big rollout. Yet I'd rather have such a new format than taking an old format with known limitations and adding another layer of crud to get sizes further down.
nmg196:
Fractal image compression is generally useless for high-quality output. It's useful for low-bitrate applications. You can read about this in the Mark Nelson book.
The main reason fractal image compression was not picked up is the same reason algorithms such as IDEA are not very popular --- software patents. IIRC The company which holds the patents for fractal image compression made it clear that it was ready to defend its IP back in the 90s.
You can read about software patents too in the Mark Nelson book (the ones which apply to data compression).
Even the IJG JPEG software (not the standard) which we use today so commonly avoids arithmetic coding and uses baseline huffman compression for compressing the quantized output.
Banu
Other formats have come and gone since - I think the wavelet-based LURAWAVE (.lwf) is even still going.
JPEG is like a lot of things - good ENOUGH for the job. Why break everything to get 50% better performance when it's good enough already?
That was classic intercourse!
Wavelet compression however is used in jpeg2000 which is a bit better than jpeg but even that isn't supported by any digital cameras.
I've been working with wavelets (eg.) in several contexts for many years now. Saying wavelet compression does 'a bit' better than jpeg is an enormous understatement. Especially in applications where you need serious compression ratios, wavelets are vastly better than the traditional jpeg compression algorithm.
Want proof? See for yourself.
But it sounds like this has more to do with improving transfer times for images that already exist in jpeg than developing a new standard for compression. But if some digital cameras started supporting jpeg2000, it'd be a boon: you could fit many more images in memory without a perceptiable decrease in quality OR could fit the same number at much higher quality.
After my experience there, can I expect to be led through a complicated and deceptive trick into downloading the trial version of some overpriced software, where I'm required to give up my email address, and the whole thing never works anyway? Stuffit might have great technology, but any company that wants to provide a proprietary format for anything will only be useful if *anyone* can open the format. Adobe knows that. And trying desperately to hook into people who *have* to turn to you to uncompress things and sell them things they most likely don't need isn't useful. It will just make StuffIt (.sit files) useless to people who *are* paying customers, because it's such a hassle for people to open the files.
samrolken
If it was so damned obvious then how come no one has done it before in the last 10 years?
It is part of the jpeg STANDARD. They explicitly define TWO different lossless backends - huffman and arithmetic. No free software, and little to no proprietary software use arithmetic encoding precisely because of software patents. Coming up with a third backend is OBVIOUSLY NO GREAT LEAP OF LOGIC.
What journal article or book or even usenet post mentions this possibility?
Oh, say, maybe the jpeg spec itself?
Thank you for playing.
Thank you for wasting my time.
Patents do not tempt me. First, a good (solid) patent is expensive, both in terms of time and money. SOlid can be an illusion, too. Either here or on Kuro5hin, I saw a tag line in someone's message that mentioned a patent they hoped to make some money from. Or maybe I saw it on the web site. Anyway, what it appeared to be on a quick look was a method for selecting a bunch of different types of compression for an image format based upon an analysis of compressability for the local cell. If that's what it was, then we did it way back in 1985, called it PMBC and used it commercially for years. It was really fun, we presented graphs during the analysis of the image and you could see what modes were most common. Filters that eventually ended up in PNG were in there, too. See what I mean? So much for solid ideas. If no one says anything, or doesn't file for a patent at all, how do you know you're not just wasting your time and money? Of course, you don't.
Second, a patent completely exposes what you're doing (in sharp contrast to trade secret.) If you're laboring under the idea that only big companies that are easily found and attacked in court are the source of "they copied my idea!" problems, you're sadly mistaken.
Third, if someone infringes, it is very expensive to prosecute them, and if they don't have bucks themselves, there is nothing to be gained from it other than shutting them down, which isn't worth spit more often than not.
Patents, some people claim, allow you to sell an idea. That's fine, as far as it goes. If that's what you want to do. However, I simply observe that if you do not have a patent, what's to stop you from selling your idea then? You can still show it to people, sell it if you can convince them they want it, etc. For that matter, you can make them buy a patent, if they're so minded. Now -- if your idea is so freaking simple that once you show the results of it to someone, then they know what to do, then I humbly submit it wasn't worth patenting anyway -- my opinion (which isn't in line with how patents work by any means) is that trivial things are, and should remain, trivial.
But that's just me. :)
I've fallen off your lawn, and I can't get up.