Slashdot Mirror
Breakthrough In JPEG Compression
Kris_J writes "The makers of the (classic) compression package Stuffit have written a program that can compress JPGs by roughly 30%. This isn't the raw image to JPG compression, this is lossless compression applied to the JPG file. Typical compression rates for JPGs are 2% to -1%. If you read the whitepaper (PDF), they are even proposing a new image format; StuffIt Image Format (SIF). Now I just need someone to write a SIF compressor for my old Kodak DC260."
I would have thought that rather than 'zipping' an existing image format to create a new one just to save 30%, they'd be better off improving the original image compression algorithm or coming up with a new one.
Quite a while ago (years!) I had a program which could compress images into a fractal image format. It was amazing - the files were much smaller than JPEGs but looked a lot better. The only drawback was that it took ages to compress the images. But with the extra CPU horsepower we have today I'm surprised fractal image compression hasn't become more prevailant. It would still probably be useless for digital cameras though as it would probably be impossible to implement the compression in hardware/firmware such that it could compress a 6+ megapixel image within the requisit 1-2 seconds.
Does anyone know what happened to fractal image format files (.fif) and why they never took off?
The linked page shows average decompression times of 6-8 seconds for 600-800 KB files, rising with the size of the file. Who would benefit from this? It's obviously too slow to speed up web pages, and would be far too CPU intensive for consumer cameras. Professional photographers would have no use for this since they would use RAW images.
I mean, it's cool and all to be able to compress JPGs by that much more, but the size gains are negated by the time it takes to decompress them. This seems just like those super high compression algorithms that have rather amazing compression rates, but take -forever- to compress or decompress, making them unusable. The difference is those are obviously and labeled as simply for scientific research into compression, but Aladdin seems to be trying to market this product for public consumption. The listed uses ( http://www.stuffit.com/imagecompression/ ) seem trivial at best.
Who's gonna be buying this?
-Cliff Spradlin
It simply works on badly compressed jpeg files.
I just installed an 800Gb hard disk in my system. I have a gigabyte worth of webmail space (more than that, if you consider that I can send myself gmail invitations). Storage is, as they say in the vernacular, very inexpensive.
Even in cameras where storage is tight, the bounds of memory is expanding all the time. Whereas a couple years ago the average storage card size was a measly 64Mb, today we are talking about gigabytes of storage memory inside our *cameras*!
So let's say we can squeeze another 30% of pictures onto a card. Does that really help us? Not really, if you consider that the compression itself rides atop JPEG compression and that computing time needs to be accounted for.
Currently, the fastest continuous shooting digital camera (the Nikon D2X) can only take 4 shots in a row before its memory buffers get full and the whole camera becomes useless. Compare that with the 9 shots per second F5 and you can see that the speed of shooting isn't going to cut it for digital cameras.
We need a compression method that is lossless, not one that creates compact files. Space is cheap, CPU cycles aren't.
The linked page did not answer some of my questions:
1. Does this only work for JPEG, or also for other (compressed or plain) files?
2a. If it only works for JPEG, why?
2b. If it works for others, how well?
Anybody who can answer these?
Please correct me if I got my facts wrong.
Great - now you can get your p0rn faster.
...although I don't even have the battery power to take as many pictures as my camera can hold; such compression would allow people to save some money on memory cards.
they are even proposing a new image format; StuffIt Image Format (SIF).
Gee, I wonder where you could license that format?
Man, they could have been a little bit more covert about their intentions and named it something a little less, umm, obvious.
The current formats might not be perfect, but at least they are (relativly) free.
"The Wright brothers were the first to fly with a heavier-than-air machine, but boy did they have a lousy plane"
Just think if this was opened uup and could be quickly adopted by browsers. All that pr0n could be compressed by 30% giving us all a new lease of life on internet bandwidth.
See my journal, I write things there
Stuffit repackages its expensive compression software every year (it seems). Now I would be happy to admit my mistake, but their main area of expertise seems to be marketing. Not technology. I reckon JPEG2000 or any number of other newer-than-JPEG formats already exceed whatever Stuffit purports to have accomplished. I suspect they are trying to tie their name to JPEG as a marketing gimmick to win hearts and minds. I doubt this is worth a mention on Slashdot.
We don't need any new standards unless they actually offer an improvement over recent technology. In technology years, JPEG is very old. A replacement for it should do better than 30% and offer other advantages. I suspect a company that actually specializes in imaging might have come up with a better solution a while ago.
So How well does the new standard comapare to the famous JPEG2000 standard?
Why does yahoo do this
I wonder if this can be applied to MP3's.
If you like what I've said here, and want to read more, go to http://www.krillrblog.com
I'm thinking that for cameras a high performance compression processor for this new algorithm might be the solution to the Camera issue. But the issue on PC's is more the processor time, not space.
Anyone know if the compression on a chip for the camera is a feasable idea, or am I just not awake yet.
Bacardi + slashdot = negative karma.
Would this technique apply to DV video?
If you read the whitepaper you will see that their algorithm is patent pending. The patent will almost certainly be granted, and, since no one else has done additional jpg compression before, it may even be deserved.
However, do we want to subject ourselves to a new tax on images? If they make it, we don't have to go! Just say NO to patented file formats!
I doubt the advantage of JPEG is that it creates lossy images. After all, if it could create lossless images of the same size, then people would use that instead. The fact that lossy compression allowed the images to be compressed even further was the advantage.
Reading the comments about speed, I doubt this will be used for much besides archival - and there I'm thinking more about very large image files, or scanned documents where the size really can mount up - in other words, compressed TIFF images as you mentioned.
I have a friend who's father is a professional photographer. He has gigabytes and gigabytes of images stored for his customers, should they want to order re-prints. They're thinking about setting up raid terabyte file server. I can certainyl say that this is good news for them!
Electrons are free; it is moving them that becomes expensive.
JPEG is (roughly) a discrete cosine transform, followed by a filter on high frequencies, followed by Huffmann encoding (which is lossless). This is probably the Huffmann encoding that they did remove and replace with one of the more efficient compression algorithms, and something that could indeed be much more efficient than simple huffmann encoding. So they still take advantage of all the strengths of JPEG related to the human perception model, but they still gain in compression. Huffmann is great to compress oft-appearing sequences, and is a great general-purpose lossless encoding, but there are other that do a better job of it.
In other words, you did not understand what they did.
"Professional photographers would have no use for this since they would use RAW images."
Who says its aimed at professional photographers, since they can justify spending big on storage (since its part of their business)? More lilkely this is aimed at granny, whose hand-me-down (up?) is running out of space for pix of the grandkids. Or those with huge pr0n collections.
"This seems just like those super high compression algorithms that have rather amazing compression rates, but take -forever- to compress or decompress, making them unusable."
Just like progressive JPEG did an 100MHz machines. It didn't stop the format becoming popular, though.
"Who's gonna be buying this?"
I could see it incorporated into Allume's (not Aladin, BTW) existing line of products, not for individual files but for large archives or portable storage (iPod photo, for example). If you were familiar with StuffIt's Finder integration on the Mac (which treats Stuffed archives like folders via contextual menus), you'd realize this is an ideal addition to their existing product.
I'm not an expert in the jpeg format but I thing I have a fair understanding of whats going on. First we have a lossy compression where the image ansformed with a windowed discrete cosinus transform and smal coefficients are discarded. Then the coeffifients are compressed with huffman coding. I thought of simply inserting a burrow-wheeler transform there, couldn't that account for 30% better compression? Still Jpeg2000 is probably a better way to go.
Okay, I do a bit of metalurgical crystalline micrography, and we keep the images in the raw bitmap format because we can't let ANY data be lost. But we also have gobs of drive space, so a good lossless, non-distorting compression format would be handy.
Bacardi + slashdot = negative karma.
...so why was tar not included?
An image compression comparison with no lenna? What's the world coming to?
I am trolling
My guess is that they're expanding the compressed JPEG co-efficients (which are entropy encoded using huffman - sometimes using pre-calculated huffman tables - see the standard) and re-compressing them with an optimized algorithm - something proprietary and tweaked extensively for standard jpeg images.
Sort of like saving space by converting gzip files to bzip2 files - except their compression scheme isn't documented or open.
Cool, but useless.
They should rather fix the basic Stuff-It which still keeps crashing if you throw corrupted download files at it. This is version 9 now ? The same bugs as version 4. Hello ? Bug fixes (basic error checking) is needed more than new bloatware.
The linked page shows average decompression times of 6-8 seconds for 600-800 KB files, rising with the size of the file. Who would benefit from this?
Any websites with the primary purpose of hosting images would benefit greatly from this - such as art & photography sites. (Yes, and porn sites, too.)
Why? Because 99% of the traffic they generate is for their images. Of those images, 99% of them are in JPEG format. So this compression would give a good savings in bandwidth on all those pictures.
At large sites, a 30% cut in required bandwidth could mean a very large savings. Now, if they can take a large cut off their operating expenses, and all they need to do that is to make the users wait a few extra seconds for their pictures, I think we know what they'll do.
As for the decompression time, one thing to remember is that with how slow internet connections are (even broadband), you're much more likely to be waiting for one of these images to transfer than you are to be waiting for it to decompress (so long as it allows you to start the decompression without waiting for the end of the file, of course).
--The Rizz
"Man will never be free until the last king is strangled with the entrails of the last priest." -Denis Diderot
The PNG format is lossless, and very widely deployed (pretty much all browsers and image programs, etc).
I rarely criticize things I don't care about.
I thought they had found a method to further compress a JPEG, while still maintaining the original format. i.e. it could still be viewed with a regular JPG viewer. That kind of optimization would've been great, especially if it could be used on webpages, forums etc.
But this is somewhat disappointing. The compression changes the format, and it must be decompressed to view it. Plus they don't intend on releasing the format, and their proposal for a new filetype which can be read by a "plugin" reeks of incompatibility issues.
StrayByte.Net
If they've really achieved 25-30% over jpg, and it looks like they have, then its a truly amazing invention considering that jpeg has been around for so long. It would save at least about ten dollars worth of space on every digital camera. If you look at the humongous image archives that NASA and other research projects generate and the cost of tape to store them all, we're talking tens of billions of dollars of savings here.
So, a question to slashdotters: do you think this kind of invention deserves to be protected by a patent? The standard response "software is already protected by copyright, patents are unnecessary" doesn't work, because anyone can study the code (even the binary will do), describe the algorithm to someone else, who can then reimplement it. Standard cleanroom process; takes only a couple of days for a competent team.
If you're RMS, you probably believe that no one has the right to own anything and all inventions and ideas belong to the public, but the majority of us will agree that that's a tad extreme. So whaddya'll think? Myself, I'm totally undecided.
i've been working on a compression technique that will get any piece of data down to one bit.
This is not in competition with JPEG - more with LZW compressed TIFF.
.x% range, yet they've apparently improved by a factor of around 6x what the previous best programs were able to achieve. I should note that I developed the audio codec with the currently highest lossless audio compression - http://lossless-audio.com/ - so have some idea of what I'm talking about here.
You're misreading the whitepaper. What they're explaining here is different and actually quite clever.
Jpeg compression can be seen as consisting of two main steps. In the first, information in the image deemed unimportant is removed from the image. In the second, the remaining information is compressed losslessly.
It was already known that the second step is not the most efficient possible. Also, the jpeg standard supports both huffman and arithmetic-coding, however due to patent reasons I think arithmetic-coding (which is more efficient) is often not used. So what they're doing is just improving the efficiency of this second step. This works much better than attempting to encode the jpg binary itself, as anything performed following entropy coding will struggle to achieve much compression, as the data has been hugely complicated by the process and it is thus much harder to find any compressible patterns etc.
They've also improved the compression by a very impressive margin. Typically in the lossless compression world, gains are in the
Anyway, the end effect is the same regardless how the compression is achieved - they're taking a lossily-compressed jpg, and reducing its size. This obviously makes little sense to be say integrated into digital cameras, for which the jpeg2000 format is available anyway and features better gains and is much faster (yet is still awaiting mainstream adoption), however from the archival and theoretical-lossless-compression perspectives what they've allegedly achieved is pretty damn interesting.
http://www.djvuzone.org/wid/
It's been around for a long time and open sourced.
The quantised DCT coefficients of a JPEG image are compressed using a JPEG standard huffman table. From what I've seen, this table is far from optimized even for "the average of the majority" of images out there.
Ogg Vorbis stores its own huffman table in its own stream. The default encoder uses a table optimized for the general audio you can find out there. There is a utility called "rehuff" (goggle it yourself please) that will calculate and build a huffman table optimized for a particular stream and it seems that on average it reduces an Ogg Vorbis filesize by about 5-10%.
Building an optimized huffman table for individual JPEGs will probably yield such improved compression rates too. If the original JPEG tables are less optimized than the Ogg Vorbis ones, the reduction will be even higher. But 30% seems a little... optimistic.
Are they getting a patent on it?
Try these links:
Stuffit
compress JPGs by roughly 30%
whitepaper (PDF)
To be honest i don't care about the 30% compression if there's the slightest danger that anyone might start a patent-war over the image-format or the compression algorithm.
I've really seen enough of that (gif, mp3, jpeg) and i prefer spending the additional storage/bandwith capacity to another uppity "IP-shop" coming out to "0wn0rz" the internet with lawyering (maybe after a management-change).
Let's have another look at that compression algorithm in 20 years or so.
"By the way if anyone here is in advertising or marketing... kill yourself." -- Bill Hicks
We don't need another image format... If you want better compression than jpeg then there are lots of alternatives. JPEG2000, Lurawave, FIF (I guess there are many more wavelet based options.) Unless they can convince Microsoft to incoporate the new image format in their browser it's not going to be used much.
So I'll just need yet another plugin for IrfanView or xnview so I can view it and convert it to a lossless format that i can open in my web browser, image editor or whatever. The only place I feel I need better image compression is on my digital camera. And storage is getting cheap now anyway. I can already get 1GB SD cards off ebay pretty cheaply.
Give it up, kibibibibytes are stupid and annoying! Argh!
SDDKJ
For compressing images prior to emailing them to my broadband-impaired relatives.
"If it's real, then it gets more interesting the closer you examine it. If it's not real, just the opposite is true." -
I wont bother going into the details of how it works (go read 'Fractal Image Compression' by Yuval Fisher*) but I concluded that fractal compression wasnt viable as there wasnt a general solution to suit all images. There are about 20 variables that you can decide, which give variable results to the final compressed image.
And one set of variables would be excellent to compressing pics of trees down 80%, another set of variables would be excellent compressing pics of animals down 80%, but using each others set would only give results equal or worse than normal compression algorithms.
Another factor at the time, five years ago, was that it took an hour to compress a 256x256 greyscale image on a 300MHz machine. Nowadays that isnt a factor.
* If anyone has this on ebook please post a link here, Im dying to read this again.
http://groups-beta.google.com/group/comp.compressi on/msg/2e05a5901e79a189?rnum=1
Damn, this was meant to be a reply to the first post. Thought my last post didnt get through. DOH!
Comment removed based on user account deletion
I did the test you suggested...
BMP: 1970 KB
BMP ZIP: 1080 KB
JPG (max quality): 724 KB
JPG ZIP: 704 KB
So it looks that a JPG is better at compressing an immage than that ZIP is.
Okay, so as near as I can figure out, this is a white paper because: .SIF as basically being having everything a jpg does except higher compression.
.sif : bandwidth savings will actually be on the order of 98% (30% image size reduction, 97% website traffic savings).
It describes a new format
Some might think it's a press release, since the "White Paper"'s discussion of the new format is largely limited to the benefits "OEMs" will have in using this (soon-to-be patented) technology, and explaining how it integrates into Allume's fine line of existence and future Digital Lifestyle(tm) products.
Some might further argue that often press releases will accompany white papers, explain the point for the consuming world, and link to the paper; this "white paper" seems to do exactly that: we are invited to examine a (currently 404'd) page to see the actual data involved.
To those who think it's a press release, it's not, because:
It don't say "For Immediate Release"
Press releases are proofread for grammatical errors, and working links capable of withstanding slashdot-class bandwidth
And by the way, they don't have a chance. Sure, bandwidth cost means a lot, but for the OEM who might consider adopting this technology, it means a new standard that offers at best 30% improvement (on those little 50kb jpegs we're asked to believe), paying for it, and passing the cost on to the consumer. It may be an improvement, but if you really want to improve your image-heavy website's bandwidth usage, switch to Fractal Image Format. Not only do you reduce the size of the images; you reduce the number of people downloading them. The same for
+5Mpix porn with no delay, here I come!
I think we can keep recursing like this until someone returns 1
I worked in this field for awhile, and the liscencing and other issues sent the company I was with running in the other direction. JPEG was good enough, everyone was using it, so JPEG it was.
Fractal compression is cool.. but encumbered by IP issues. Too bad.
..don't panic
stfu! =p
whoa... lol
Slow Down Cowboy!
Slashdot requires you to wait 20 seconds between hitting 'reply' and submitting a comment.
It's been 17 seconds since you hit 'reply'.
Chances are, you're behind a firewall or proxy, or clicked the Back button to accidentally reuse a form. Please try again. If the problem persists, and all other options have been tried, contact the site administrator.
the only permanence in existence, is the impermanence of existence.
10 GB of jpegs would take ~ 22 hours witch a P4-1.8Ghz to compress with this new technology.
Although it's very interesting that jpegs can theoretically be compressed, the processing time needed is too high, such technology will never make it to embedded systems, e.g. cameras etc.
Given that the software is likely to be proprietary and the algorithm will be patented, it becomes completely useless to me and it is completely unsuitable for archiving anything (smart people don't play nickel and dime games with archives and backups).
Maybe if computers were a lot faster and the compression worked on any array of pixels, not just those that have undergone the lossy transformation of JPEG compression. But even then, it would have to do better than what PNG can do in terms of all the other things PNG does and no patented format will beat PNG at its game.
In theory what you say sounds nice, but in practice I genuinely can't recall a situation where a little more compression would have allowed me to send all the photos I wanted to via e-mail. But the reasons I mentioned at the top of this post are more important reasons why this should be rejected out of hand.
What meager benefits this affords are far outweighed by the costs. I don't see this going anywhere, and for very good reason.
Digital Citizen
Yes, it is probably only doing huffman decoding and then context-based adaptive arithmetic coding (a simple google search will give you papers on this category of lossless coding). If you design the contexts carefully, you can gain quite a lot over simple Huffman, in fact about 28% :)
- YA
I take it you don't have children ;-)
The six tests mentioned in the article were:
"They were dipped into cola, put through a washing machine, dunked in coffee, trampled by a skateboard, run over by a child's toy car and given to a six-year-old boy to destroy."
Because in the saner parts of the world, algorithms can't be patented (yet).
The last time I looked into fractal technology (1997), it was impressive and it does work very well, but the problem with it appeared to be that the technology was plagued with patent issues. http://www.ross.net/compression/patent_us5384867.h tml
http://www.ross.net/compression/patent_us5065447.h tml
http://www.ross.net/compression/patent_us5430812.h tml
Stuffit my ass!
Emacs is good operating system, but it has one flaw: Its text editor could be better.
" Typical compression rates for JPGs are 2% to -1%. "
Does -1% mean that the file actually gets bigger?
Big Deal. With broadband ever expanding and storage being relatively cheap and inexpensive, this technology is a waste of time. Hell, I've got a $100 USB drive for X-mas that was a gig...a fucking gig.
Athletic Scholarships to universities make as much sense as academic scholarships to sports teams.
With lossless wavelet compression. That's what I'm still waiting for. As the "2000" indicates it's nothing terribly new but as of yet still misses a big rollout. Yet I'd rather have such a new format than taking an old format with known limitations and adding another layer of crud to get sizes further down.
...its IP status is currently ambiguous. jpeg2000 is currently being attacked by some patent holders and until it's all settled nobody is going to touch it.
If this idea is what's suggested here, ie. instead of
Image -> DCT -> Truncate spectrum -> RLE encode
they use
Image -> DCT -> Truncate spectrum -> some more efficient encode
it wouldn't surprise me if the JPEG Group discussed alternate compression encodings during the standardisation meetings. In fact, I'd be extremely surprised if the matter was not raised. If it was, it's pretty clear that this patent is not novel in the slightest...
Athletic Scholarships to universities make as much sense as academic scholarships to sports teams.
The JPEG standard specifies 2 entropy coding methods; Huffman coding and arithmic coding. As arithmic coding is patented it is not in use. The patents for this arithmic coding called Q-coding http://www.research.ibm.com/journal/rd/426/mitchel l.html are in hands of IBM.
Perhaps they will allow OSS to use this patent along with the 500 other patents recently allowed?
http://www.ibm.com/ibm/licensing/patents/pledgedpa tents.pdf
The particular variant of arithmetic coding specified by the JPEG standard is called Q-coding. This variant has the advantage of not requiring any multiplications in the inner loop. Q-coding was invented a few years ago at IBM, and IBM has obtained patents on the algorithm. Thus, in order to use the JPEG arithmetic coding process, you must obtain a license from IBM. It appears that AT&T and Mitsubishi may also hold relevant patents.
the bit output you get from jpeg coding is pretty noisy, so you're lucky if you get any compression at all from huffman. 1%-2% is typical.
mpeg is huffman encoded too, and it also does not compress very well. (again, 1%-2%)
i'm guessing they did something more than just replace the compression with 'one of the more efficient compression algorithms'. because thats already been done, and nobody got much better than huffman (which is why huffman is used).
it's more likely they analyze and process the jpeg data in a special way which leads it to be more easily compressed. knowing the specific type of data you are compressing, you can tune your coder to specifically handle it (vs huffman, which is applied in a generic way to the raw jpeg/mpeg bitstream).
wikipedia
http://en.wikipedia.org/wiki/Fractal_compression
Some sourceforge projects
http://sourceforge.net/projects/mpdslow/
http://sourceforge.net/projects/fractcompr/
for audio
http://sourceforge.net/projects/fraucomp/
I was talking to a dude in the Defence world in the early 90's and was talking about a compression algorythm that used fractels. This is 10+ years ago, is this the same thing? It was used for T.V. bombs I think.
Come the revolution, the Bourgeois, Capitalistic, "A PARKING STICKER HOLDERS", will be first against the wall!
short for siphilis.
I don't quite believe they will release it as open format.
Richard Pearse!
how well does it compress lossless jpegs?
...
Nick
Better compression of data is useful to many many sectors.
Everything from mobile phones, satellite comms. You will never be able to overcome say, the distance from here to Pluto by trying to go faster than light, instead if you decrease the amount of data being transferred enough, it can save millions of dollars in time.
All your base are belong to Google.
It's JPEG, not JPG. JPG is the file extension used for JPEG files on DOS systems because of 3 character file-extension limitation. JPEG is the name of the format/compression and the extension (which should be) used on systems that support 3+ character file-extensions. Because of the internet the JPG extension has spread and now people ignorantly use JPG to even refer to the format/compression.
My medium-cheap ranged Konica-Minolta Z3 digicam snaps max. 15 frames at 10fps at a 1600x1200 resolution.
I have no doubt more expensive cameras will shoot more frames at higher speed and higher resolutions.
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
or pr0n of 30% higher quality for the same filesize
:-)
or the same quality 30% faster
I think we can all appreciate the importance of this breakthrough!
For one thing, the size of images increases with the square of resolution. That means that to double the resolution od a 4Mpix image, you have to go to 16Mpix. So images can become very expensive, an that can grow much faster than the storage capacity of permanent media (if such a thing exists). You do not store that many pictures on a CD, and the technology does not evolve that fast (even with DVD).
Another point is that compression techniques may also apply to films, which are a lot more costly to preserve. More generally, for people and organizations who archive a lot, memory is always at a premium (I have read a confidential report from a major corporation that was considering the switch from MS Office to OpenOffice.org, one argument being that OpenOffice files are 60% smaller and it would reduce the strain on their archiving).
Regarding lossyness of compression, the point of lossy compression is that it can, for an a priori given memory space, give you a much better image than lossless compression. A lossless compression algorithm for a picture will limit the resolution of that picture as a consequence of memory limitation. So you do actually have information loss, because you cannot store a high resolution picture (if one would have been available). You make an a priori choice of losing information through loss of resolution. But it may not be the best choice.
On the other hand, with a lossy algorithm, you can keep the resolution as high as is available, and leave it to the algorithm to decide how best to compress all the available information to fit the available memory, or bandwith.
Regarding cameras, which what everyone seems to have in mind (though there are other applications in this world :-), memory is not that
cheap, especially if you are on a long trip in an exotic country, and
want to keep several hundred pictures, or more, which you can hardly
screen directly on the camera, without heavy equipment, to prune the
less interesting ones. This is personal experience.
Where not only logical inconsistencies are frequently ignored, but logical consistencies are unwelcomed.
I'll have to check, but I think I still have the program that converted JPG adn BMP to FIF.
First problem however was that no other program had the filters to open FIF.
Second problem was that it took ages to encode a file. I'm talking minutes. Most times even tens of minutes. Way to long to wait for saving a file.
However, I was running a 486DX2-66 at the time. Maybe I should try to find the program an try again.
So wouldn't it be possible to calculate (predictive) a bunch of alternatives, then pick the one with the best compression and store the the used variables in the same file as the compressed data?
For the internet, compression time isn't by far as important as filesize and decompression time.
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
Or we can simply use JPEG 2000 that has better compression (even a lossless mode), progressive viewing, random access areas so you can view them with going through the whole file, support for alpha and transparency planes, multiple dynamic ranges (1- to 16-bit).
JPEG is good, and the new technique may extend it, but it would probably be worth more to simply upgrade to the new system.
The JPEG algorithm breaks up the image into chunks of 8x8 pixels, takes the DCT coefficients of those chunks, then packs them into a Huffman-encoded stream.
If you can repack an existing JPEG file with a better Huffman code, then you can shrink the file without any loss of image data, while still remaining fully compatible.
jpegoptim usually can't further shrink JPEG files produced by the GIMP and Photoshop (which have their own optimized JPEG routines). It seems to do a lot better, though, on JPEG files coming out of digital cameras (gains of 30% - 50% are pretty common).
It's incompatible with JPEG.
It has worse compression than other non-JPEG file formats.
It has worse image quality than those file formats.
So why would anybody bother?
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
Seriously folks, whilst this is nice it is kludging. Heck anybody could do a JPG>RAW>new jpg format say JP2k or PNG, but to call it JPG still is like wrong, let aklone get that excited by it. If your going to redo a format in such a way that it changes the original file in that it needs another processing stage then it is a different file and not the .jpg all apps no and love and understand. Look at the sucess JPG 2k has had oevr .jpg. Or even .PNG. Basicly soon as you stick another abstraction layer over the .jpg file it ceases to be a .jpg file.
Now they are talking lossless, ie not adding any more loss in quality which is an achievement, but it aint .jpg.
Basicly this is and should be viewed as a .jpg storage achieve format. With this in mind perhaps they could expand it a little so they can add to .jpg's lackings and effectivly create animated .jpg's, that would add value to what there doing, enough that people WOULD adopt it into there browsers, from there it just goes and goes.
So my sugegstion would be to forget this new extension and flesh it some more so you have some timming control for slideshowing the `zipped` .jpg's and present there new format as animated .jpgs's or .jpg archive storage. Just dont kludge another wrapper for the hel of it, if its a new wheel, call it a new wheel.
Also, the source of these three test images is described as "I used my Nikon Coolpix 3MP digital camera to generate JPEG files..." However, the pictures are from Canada, the US, and Japan. It's not like he went out and took three random photos. While he doesn't mention the specific model used, the coolpix cameras appear to be your average consumer models, so the wording is quite suspect. As the total compression times are under a minute, why were only these three pictures chosen? Perhaps they compress better than average? Why can't we download the original .JPG files? There is no way to reproduce this test.
Lastly, the article consistently says 30%, but the average actual compression is 25.53333%. That's 17.5% bull puck.
So, the reviewer isn't objective, the picture sources are suspect, and the numbers are suspect. This sounds like a slashdot-sponsored spin machine to me.
If Stuffit really wanted to prove themselves, they'd put a link on their website to compress/decompress images and have it only work for a week or so. Then they could publicly demonstrate things without giving away an executable to be leaked.
Instead they give a full, working EXE to some guy who's home page is linked to only 83 times (most [all?] of which are junk/link farms). Don't believe everything you read. Especially not on the Internet, and especially not this junk.
<tinfoil hat>
As stated above, this website effectively has no google presence. How did it survive the slashdot effect with pictures? It appears to be hosted by Roger's Cable in Toronto. Who is paying (presumably) big bucks for bandwidth for an otherwise unremarkable site?
Why does he first describe "The test computer used", then go on to mention "Machine A" and "Machine B"? Also, why is a compression expert using such wimpy hardware? Some quick research on the author only shows how unremarkable it all is for such an important announcement.
Of course, you also have to wonder how Kris_J (apparently from Australia) found out about the story to post it to begin with, since it's so obscure. (No offence Kris_J, I'm just in ultra-skeptic mode here)
</tinfoil hat>
For one thing, they are only talking about pretty small JPEG files (20k-40k or so), so we have some header overhead to be expected. The payload will not be shrunk that much, however.
For another, all of their test files have been generated by a single converter from PNG files. We have no information as to the quality of the PNG files, and also no information as to the quality of that particular converter.
My Minolta Dimage 7Hi (5 megapixel) has a 64Mb buffer (and comes with a 16Mb card, go figure!)
i /p age9.asp
taking 1.3Mp images at 7 frames a second, it can take about 140 pictures (so about 20 seconds) but then takes two minutes to write them to the memory card.
at 5Mp (fine) it can do about 2 frames a second for about nine seconds (18 5Mp images) which then takes an additional five seconds before the next picture can be taken
http://www.dpreview.com/reviews/minoltadimage7h
... but if you are using RAW mode, the time from shutter release until the light goes off on the card writing is 25 seconds (on CF cards, 15 seconds on a microdrive)
What do you mean it isn't a factor??? of course it's a factor... even on a 3 GHz machine that hour has only come down to 6 minutes...
Donald 'Duck' Dunn: We had a band powerful enough to turn goat piss into gasoline.
Another part of my project was to see if theres a magic set of pixel blocks that could be stored to save on some of the compression calculations. Turned out to be hit and miss. It didnt matter that much.
True professionals don't use JPEG
Not true. I work in a advertising agency and we use JPEG quite often. We still use a lot of TIFF and EPS and whatever (I'm a texter, so I don't know too much). But when I show a JPEG picture to the art director he asks "what resolution?" and "let me see it" to determine if he can be use it.
If you leave the slider on the "high-quality"-side when you compress a picture it's virtually identical to the uncompressed one, but much, much smaller.
I don't need a signature.
I guess that must be using maximum compression settings. Using average settings my prog does the same greyscale in about 2 seconds on my 2GHZ machine. Havent really tested it out recently though. I must try to hoke out the disk from the attic and test it again :)
The original JPEG compression algorithm had Huffmann coding for the DCT variables, but it also had some fixed-length codes for the beginning and end of blocks. If you set your compression at about 10x then you can hardly see the difference with real images. bring it up to 15x and the changes are still modest. However, yank it much over about 22x, and the image will go to hell. The reason is the block handling codes meant that a JPEG image with no data at all - a flat tint - would only compress at about 64x, so at 22x compression these block handling codes are about 1/3 of your overall code. The fractional bit wastage you get with using Huffmannn coding instead of arithmetic coding mops up some of the rest as you are usig shorter Huffmann codes. The codes are also very regular, as about 1/3 of the code is not particularly random. The 1/3 figure also matches the 30% compression figures too, which isn't surprising.
Why didn't the original JPEG developers make a better job of this? Well, doing an experimental DCT compression used to take me hours or days when I was developing on a shared PDP-11, and there was always the worry that a dropped bit would lose your place in the code, and scramble the rest of the image. A little regular overhead was also useful for things like progressive JPEG control. I guess we all knew it was not as tight as things could have been, but it got the job done. We knew if you want to get 40x compression, then reducing your image to half size, and the compressing that by x10 will look better. Unfortunately, people who just drag a slider to get more compression don't always know that.
The right solution would be to use JPEG2000 which has a much smaller block overhead, and so fails much more gracefully at higher compressions.
Anyone know where the reference implementation is posted?
Oh, there isnt a reference implementation? Then I guess they dont intend for thius to become a standard or anything. Surely no one here is going to support making something standard for which there is only one proprietary, closed source (eg secret) implentation of?
Would you buy a compression algorithm from somebody that wastes a byte with that
redundant "F" at the end of their acronym? SIF Format = Stuffit Image Format Format.
What are your requirements in terms of resolution, data channels, bit depth ? compression/decompression time?
you could try OpenEXR
AFAIK, there is no huffman table in jpeg and the compression is good old RLE. (As in what we used to use on Compuserve before they invented GIF). I didn't read the article, but I suspect that they are in fact replacing RLE with something like huffman encoding, but I seriously doubt that they would have an optimized table for each image since storing the table would offset too much of the space saved from the better compression.
-- It only takes 20 minutes for a liberal to become a conservative thanks to our new outpatient surgical procedure!
BTW: contrary to factal-based compression and other proprietary formats, the IP-situation is unproblematic for standard-complying implementations.
You're a bit confused here. My lossless algorithm is an audio one - nothing to do with RLE. The one under discussion also has nothing to do with RLE. Nor does JPEG in fact. So none of them can be meaningfully compared to RLE.
My codec could be compared to other lossless audio codecs. The differences in compression ratios between all these codecs is relatively small. That was my point. You can also compare various state-of-the-art archivers on how they perform compressing JPGs. Again - very small differences, until you come to this new StuffIt compressor, which suddenly achieves a claimed average of 28% compression (vs the rest which are all less than 4%, i.e. a factor of 6-7x improvement vs the normal say 1-2% improvements in the field).
...however from the archival and theoretical-lossless-compression perspectives what they've allegedly achieved is pretty damn interesting....
Yes interesting but also kinda mysterious and a little hard to believe [The test results in TFA were important in overcoming my skepticism]...I saw no hint of what math or technique they came up with, just claims of effectiveness. Or did I miss something in the white paper?
SLASHDOT: news for people who can't concentrate on work or have no life at all and got tired of yelling back at the TV.
The sick thing about patents is that an idea like this probably have been thought of by many of the experts in that technology (I mean one of the 100 or so people actually doing basic research on it), and they might have discarded it since it may be too computationally expensive and the improvement is marginal in the cases they care about, and the idea seems too trivial for them to even bother writing a paper about it. Then someone patents it, because the price/performance ratio of the idea seems to be better in his situation (maybe his computer is faster, his data size is different, or he cares for the 0.5% improvement that other experts find it too much trouble to bother). Since only a small number of experts know it already, the patent examiner can hardly reject the patent as being obvious, and it is of course novel since no one has bothered to publish any prior art. Then, when the idea actually becomes useful, no one can use it freely.
As a graduate student, a large percentage of the research papers I have read contains ideas just like this, e.g. reverting a performance-computation time tradeoff that an existing method has made. Such non-ground-breaking ideas are so many that it is not practical to ask the experts to publish them all (for that matter, there is only so much pages in an academic magazine, and if publishers publish too much for people to read they would lose money), but since such ideas are usually novel, non-trivial to everyone but the 100 or so best experts in the field, and useful (at the patent application time) in some very special cases that only the author can think of, such patents will probably be granted, and other experts who know the idea for a long time but didn't bother to put it on paper find that they can't use the trick anymore.
bz2 ? zip ?
You know what ? The breakthrough in JPEG compression was mainly JPEG-2000.
There had been, was, and has been possible to achieve better compression ratios, and guess what, even in the high times of JPEG we knew that what's inside, it's not the optimal solution. But there were certain aspects which made it stay the way it was, and that was good to be so. The same goes with JPEG-2000. Since the appearance of it there have been many attempts to make it better, and there have been some good results achieved (even I have read and sometimes even reviewed papers dealing with the subject).
It's really no question whether one can make an X% better compression to JPEG with the same quality (expecially today, when JPEG is so old that every joe and his dog had time to develop better ways). The question is, has it enough practical usability to justify its presence ? Is there a well-justified reason why we should use it ? Does it deliver
- better compression rates (smaller size with the same objective & subjective quality) ?
- lower compression times ?
- compatibility ?
- is it any better for hardware implementation purposes (same or lower computation times) ?
If it's just a "better" compression for the compression's own sake and not for our sakes, then this is even less news than me cleaning my room.
I am putting myself to the fullest possible use, which is all I can think that any conscious entity can ever hope to do.
So when do some of these patents start expiring?
Score 2 seems too low for that aclaration level, please mod parent up.
My point is, Stuffit's 28% improvement is achieved by replacing JPEG's somewhat inefficent Huffman compression with something else a whole lot better. But most compression algorithms are better than a fixed-table Huffman.
I'm just wondering how much better your algorithm is at compressing audio than a Huffman algorithm would be...
Athletic Scholarships to universities make as much sense as academic scholarships to sports teams.
The JPEG algorithm is unrivaled in its ability to compress grayscale images. This is partly achieved by the ingenious introduction of lovely green and purple artifacts. I know we've all thought, "why [slap to forehead] didn't I think of that!" It seems so obvious now.
is because it is a way to compress (some types of) file that are ALREADY compressed. If you've ever zip'd (or RAR's or 7Zip'd or Gzip'd etc) a jpeg, a gif, a png or even another zip file - ie. any other type of already compressed file - you'll be lucky to get any savings, or if you do it will be a few % at most. That's because most of the redundancy (ie. compressibility) has already been removed from the data in the file.
This method can compress jpg files significantly, and that is NEW, in fact it was previously though to be something that was more or less impossible (or at least, extremely difficult with little to gain).
Of course it is slow, but it works and it will find practical uses.
I'm a perfectionist but I'm trying to cut back.
How long did it take to decompress/view the image? Was it significantly longer than viewing the original? Would it be a viable format for webviewing?
- shazow
I don't to compress files produced by my Kodak DC260 because someone stole it :-(
Depends on whether or not the pharmaceutical companies can get Cher to be their spokeswoman for a proposed Patent Term "Harmonization" Act.
Seriously. Is there a setting to filter out 'stuff' like this?
...to compress motion JPEG. Knocking 30% off the size of a video stream while still allowing frame-by-frame editing, makes it more competitive with standards that compress in 3D (x,y,time). *Especially* because there is no quality loss compared to a simple JPEG stream.
Visit http://ringbreak.dnd.utwente.nl/~mrjb/growingbettersoftware to download your free copy of the book
Digital pictures are the future of film - digital cameras began outselling traditional cameras last year
Does this include the cheap one-time-use cameras that are returned with film inside to the photo store? Or are you including one-time-use cameras that give a Picture CD along with prints as "digital" cameras?
... a number of people do use "Mb" for Megabyte, but you're right, it has become far more common to use an upper case B for byte.
So my camera actually has 64megabytes of built in frame buffer, but I think the card that comes with it actually only holds about 16bits!
It's ridiculous when a camera comes (as standard) with less memory than an equivalent disposable film camera (in terms of number of frames you can take), but it shaves an extra 10 or 20 dollars off the price and that's all that matters when you're selling cameras.
The first thing I did when I bought the camera was go out and buy a 1GB CF card.
This stinks of nothing more than Stuffit desperately trying to find relevance 2004. Nobody really needs their product any more what with speedy net connections and super-giant hard drives.
This may have been useful 10 years ago when modems were slow and hard drives were small. But not any more.
My asian porn collection (I'm honest) doesn't take up THAT much space, and even if it did, drives are big, fast, and cheap. I don't have any need to recompress my images. And I would be worried about making the image worse. JPEG is already pretty crappy. Smaller file = less data = less image. You can't get smaller without losing data.
well rle coding (combined with huffman or whatever) would if anything make most audio files bigger. because audio is typically 16-bit (vs 8-bit for images) theres very little chance two concurrent values will be the same. my codec does use internally for the entropy part something based on range coding, which is similar to, but a little more efficient than, huffman coding, but it does a lot of other stuff as part of the entropy coding as well.
the point about stuffit's improvement is that its so huge. sure huffman is inefficient, but other methods are nowhere near 28% better i can assure you (else other state-of-the-art lossless archivers would have done better than the 4% they achieve before now).
If we really need to compress our images even more, why not push the JPEG 2000 standard, which has even more impressive results?
Mod parent up!
The AACS key is NOT 0xF606EEFD628B1CA427BEA93A9CA9773F
Is this it? The end of shitty looking, distorted pics on peoples crappy sites?
What if Tetris was invented by Nazis?
Sounds good.
But Aladdin (which seems to now be called Allume): please don't use the SIF file extension. It's already taken. It's used for the Smalltalk Interchange File format, which part of the ANSI Smalltalk standard- ANSI NCITS 319-1998. If anyone else thinks they should find a new extension, please feel free to join me in emailing them at:
Jennifer Watson (PR)
jwatson@allume.com
Matthew Covington (Product manager)
mcovington@allume.com
Working toward a usable PDA environment in the spirit of Newton OS: Dynapad
If it was so damned obvious then how come no one has done it before in the last 10 years?
What journal article or book or even usenet post mentions this possibility?
Thank you for playing.
Lizardtech already has an algorithm that successfully compresses images up to 300% of their initial size. It's been around for years and years and is very good. It is nearly impossible to see and loss of resolution. It is already used for satellite image compression and is proven. We actually use it to transmit images of newspaper pages in high quality, high resolution over the web for purposes of advertisers 'oking' their adverts in the layout.
30% compression absolutely sucks and is like someone heralding that they have built the first plane today. It's moronic actually to think that someone wasted their time doing something that has already been done and then compounding it by doing a poor job at it. I am not trying to be obtuse, just realistic.
Sorry if not enthused. I remember when all these
compression routines were all over the net and all free. Now everybody runs after the almighty dollar in squeexing the public for stuff that really is in the public domain. C'mon, this new routing is for an Apple box, one of the biggest litigators in history was Apple. It is stuffit, a copyrighted, trademarked, patented whatever of an obsolete system; and now some 'person' wants to describe the 'new and improved' system on a PDF file. Now this PDF will probably want Acrobat 6 to decode it, and we all know that Acrobat 6 is a windows XP program and it is spyware and an address collector for spammers and other shady characters.
go visit Lizardtech.com and see what they have been doing for years. 300% compression that has already been used in satelite technology. Successful, proven and really good. This Slashdot story only serves to sucker the uninformed. I would be careful and make sure you research anything that is posted as an article here. They can really make people look like fools by posting things, such as this, and completely misrepresenting it's worth.
you're throwing your own cultural assumptions around.
... and we used acoustic couplers to connect the Heathkit terminal we built (from transistors and resistors and such, all soldered by hand onto circuit boards) to the dial up mini-computer. The biggest memory I used up until 1983 was 64KB and it wasn't called "64KB", it was 64K.
Or maybe I'm just older than you.
I grew up in the UK and the US (moved back and forth) and my first computer predated the IBM PC (I used my first computer in 1976, I wrote my first graphics programs in 1979) and I can tell you for sure that MB, Mb and mb were interchangeable in those days, because only hard disks had Mb/MB/mb. Modems worked at 300bits per second, if you were lucky
So yes, I said I should have used MB since that's the usage NOW, but it was just flame bait to throw in that
The concepts of capital and small letters used in abbreviations was something everyone should have learned before grade 9 science
bit.
"m" and "M" are perfectly standard metric abbreviations, but the "b" and "B" thing is probably less than ten years old as any kind of general standard (outside of, say, telecoms or computer hardware design)
So now I get the picture, only smaller!
After my experience there, can I expect to be led through a complicated and deceptive trick into downloading the trial version of some overpriced software, where I'm required to give up my email address, and the whole thing never works anyway? Stuffit might have great technology, but any company that wants to provide a proprietary format for anything will only be useful if *anyone* can open the format. Adobe knows that. And trying desperately to hook into people who *have* to turn to you to uncompress things and sell them things they most likely don't need isn't useful. It will just make StuffIt (.sit files) useless to people who *are* paying customers, because it's such a hassle for people to open the files.
samrolken
The appropriate tradeoff depends on the circumstances.
At first I was baffled by the amazingly low number of pictures I got on my little 1.3mp radio shack flatfoto camera (as compared to cameras with larger images).
Then I looked at what was on the card. This little bugger doesn't compress *at all* on its own. It waits until it talks to the host computer, which turns it's raw pictures into jpgs at a fraction of the space.
And as I thought more, this makes a lot of sense for this type of product. It doesn't save significantly on the cost of a processor for the camera, but the size of the battery. I can take over a hundred pictures with this thing before the battery runs low.
OTOH, this would be an unacceptable tradeoff in the 4mp camera attached to my 10:1 zoom lense . . . on that, I can just swap AA batteries, which are small compared to the camera size.
hawk
Why, you ask?
Because it reduces bandwidth considerably more than this method. For comparable image quality, JPEG2000 images are much smaller. The savings is greater than the 30% being discussed here. Closer to 60%. Also decompression is fast, about as noticable as normal JPEGs. Also, you don't have to have all the data to decompress a full sized image. If you have 20% of the data, you can decompress and the image just won't be as detailed. In fact, if you want to make a JPEG2000 file smaller, you can just cut off the last half of the file, and it will still be viewable. A webserver under heavy load could decide on the fly to simply send the first half of all images to save bandwidth. No processing would be required, just only send the first half of the file. Oh, and don't forget that it is a superior format that supports things such as alpha channel that JPEG doesn't have.
But the real reason that I want browsers to support JPEG2000 compression is so that camera makers will feel pressure to support it. Imagine being able to take 4x the pictures with your existing memory card. Also, imagine having a full card, and then using the bit slicing technique mentioned above to slightly reduced the quality of some of the photos in order to take more photos.
When you look at the benefits of JPEG2000 compared to this method of compressing a JPEG, it is pretty clear which one browser makers should be looking at.
Lasers Controlled Games!
Because they're applied for a patent, its unlikely this will see adoption in anything but StuffIt... and who besides Mac users actually use StuffIt! (Don't forget as late as a couple years ago StuffIt couldn't handle more than 2^24 in an archive MACing it useless for many.)
Hype.sit
/\/\icro/\/\uncher
No files released. No home page created.
When I woke up this morning and found surfing the net was so slow, I wondered what was going on. Then I found the /. story and that explained everything... ;) I'm surprised my server kept running and while not fast, is not insanely slow either. Go Linux!
- Jeff Gilchrist
They have another imaging technology that they purchased from AT&T called DjVu. They've Open Sourced the viewer for that technology under the GPL.
I believe an encoder/decoder is also available under a GPL license, though LizardTech doesn't appear to be happy with the GPL because they are pro software patents, and the GPL is not. The encoder/decoder may or may not be a fractal engine, someone more knowledgeable will have to answer that question.
LizardTech may be involved in a squable over the JPEG2000 technology. Something to do with patent litigation.
= 9J =
I wonder if they handle compressing broken jpg correctly.... Like, a file that was broken, but at least most of the file was viewable, but when you compressed it and then extracted it from the archive, it did not come out bit exact.
Another example of progresses silenced by patents,stupid regulations and bureaucracy.
(Note: Please don't mod this as "insightful". "informative" preferred)
... I've looked up the SI units, and MB, Mb etc. are not there. They are, however, IEEE standards, which is nearly the same, but since you insist on precise accuracy, you're wrong.
s .html
It is important to recognize that the new prefixes for binary multiples are not part of the International System of Units (SI)
http://physics.nist.gov/cuu/Units/binary.html
And this NON-SI standard dates all the way back to when you were in nappies, i.e. 2000.
And in fact, I believe my camera has a 64MiB (mebibyte) buffer according to this standard.
Oh, and "b" for bit is IEEE, it's "bit" for IEC, and "B" is usually an octet, but not necessarily. So we could/should be using "o" instead for definitely 8-bit bytes.
This means my camera has a 64Mio buffer memory.
http://www.cofc.edu/~frysingj/binprefixe
Love, Grandpa.
p.s. I've enjoyed looking up and finding out this stuff, I hope you've enjoyed it equally.
At 5 seconds per megabyte, it will take almost 58 days to compress 1 Terabyte of images. Wouldn't you rather just buy 30% more storage space and be done archiving this month?
As another person who has implemented JPEG, I vouch for his accuracy :-)
It's simply that these are the first people who've ever bothered to try it.
yahoo messengeruses jpeg2000 for compression in webcam..
Why does yahoo do this
If it was so damned obvious then how come no one has done it before in the last 10 years?
It is part of the jpeg STANDARD. They explicitly define TWO different lossless backends - huffman and arithmetic. No free software, and little to no proprietary software use arithmetic encoding precisely because of software patents. Coming up with a third backend is OBVIOUSLY NO GREAT LEAP OF LOGIC.
What journal article or book or even usenet post mentions this possibility?
Oh, say, maybe the jpeg spec itself?
Thank you for playing.
Thank you for wasting my time.
http://www.dogma.net/markn/articles/bwt/bwt.htm
If you replace the diagonalization step with a BWT, you can merge all of the zeros together at the end. Then, since the values will be in order, you can achieve higher compression than you'd get with the existing Huffman coding. And, there is no patent problem.
not what you think I wrote
/. isn't that "clever". Which is often good, I had sites that try to guess what I mean and then change what I've typed.
... that's not good enough for you, it isn't enough that you win, others must lose.
... you know...) :-)
I didn't say anything that equates to
you don't use OUTDATED experience to suggest you were correct in your representation of units
I said "Fair enough" which meant I agreed with you, or is that the first time that that has happened and you're not used to it. I also explained that claiming these were SI units I should have learned in 9th grade was wrong as they aren't SI units and they weren't even IEC/IEEE units when I was in 9th grade.
I have heard of HTML, I've also used lots of other sites that are clever enough to convert URLs into clickable links without requiring the user to type the stuff, obviously
And perhaps you should spend your time doing something constructive rather than picking at other people who have made a mistake? I agreed with you, it was a mistake, I should have used a capital "B". But no
Anyway, have a lovely evening, I'm off home now from my job where I write technical documentation (and yes, I am embarrassed I typed the wrong case for the "B", but to err is human, to really screw things up
When will the arithmetic coding patent be expired?
Depends on whether the software patent lobby and the pharmaceutical lobby can get Cher as a spokesperson the way the copyright industry got the late Sonny Bono.
Such non-ground-breaking ideas are so many that it is not practical to ask the experts to publish them all ... but since such ideas are usually novel ... such patents will probably be granted
So if a given method of image compression improvement isn't good enough for a peer-reviewed publication, then why don't the ideas just get tossed up on the web, with Archive.org documenting the prior art?
I'll take an educated guess that tar wasn't included because it would have been redundant. The compressor most often used with tar is gzip, which uses the same algorithm as zip, which was tested.
But what happens if I open that file in Photoshop and Save for Web using the JPEG HighO preset?
Original: 7K 12 Jan 12:39 DSCN3974_TN2.jpg
bzip2 -9: 8K 12 Jan 12:39 DSCN3974_TN2.jpg.bz2
Larger (-5.12% "savings"), but I suspect that that's due to bzip2 not working well on small files. Let's try gzipping those files!That's 0% savings on the "web optimized" version.
Difference? No embedded previews for the "Web optimized" version.
Move along, nothing to see here. I'll be contacting the author of the original article shortly.
Stating on Slashdot that I like cheese since 1997.
My apologies! I should RTFA first. ;-) I see now that apparently their test images only managed 1% savings using bzip2. Oops! I guess they saved them correctly after all. :-}
Stating on Slashdot that I like cheese since 1997.
What did you use as a fitness function?
ReadThe ReflectionEngine, a cyberpunk style n
The point is easily seen by considering who developed this. Allume Systems (StuffIt) is a company which compresses files for archive purposes. In such a situation - and only in such a situation - is the time/space trade-off we see here worthwhile. In a digital camera, if it could work at all, it would likely not be worth the battery power. On the Internet, bandwidth (which is already cheap) could be better saved using JPEG 2000, which is much likelier to catch on than this proprietary format. For computer photo albums, one would not wish to wait a few seconds before viewing every picture.
This is not just a gimmick, but neither is it a real breakthrough, either technologically or in terms of the market. What it will mean that StuffIt will compress much better (assuming you have a fair number of images) and that's good news (especially for Allume). All else - whether this is a better-predictive context-based arithmetic code or a "rehuff," whether this will eventually be efficient enough to be useful for other applications, whether JPEG 2000 will eventually make this obselete - is mere speculation.
The JPEG algorithm is unrivaled in its ability to compress grayscale images. This is partly achieved by the ingenious introduction of lovely green and purple artifacts.
Then your encoder is broken. Try converting the image to grayscale format (in GIMP, Image > Mode > Grayscale) before saving it as JPEG; a good encoder will notice that the image is in a grayscale format and write a JPEG file without any chroma channels. Or if you want to remove the chroma channels entirely without adding artifacts to luma, jpegtran (distributed as part of the popular IJG implementation of JPEG compression) can do that.
Thats interesting. I just assumed JPG were some kind of cosine transform. shows how little I know.
Since compressed information looks pseudo random [not much in the way of patterns repeated and therefor not much for codebook compressions like Lev-Zimpel or Huffman to work with], why can't I get much improvemenet when I ZIP a jpg if its got 30% fluff in it?
SLASHDOT: news for people who can't concentrate on work or have no life at all and got tired of yelling back at the TV.
When consuming an average of 60 megabits per second of outgoing bandwidth, I have some interest in seeing that reduced to 50 megabits per second through better JPEG compression. A difference of $700-$1500 per month in recurring costs isn't insignificant. Average outgoing bandwidth use for the last 10 minutes is 90 megabits/s. Transfer between 1 Jan and 11 Jan inclusive is 6.3TB. Last month was 13TB.
JPEG does a DCT followed by huffman coding.
There are lossless transforms which (a) increase entropy and (b) use no less space (or even more space).
As a trivial example, replace each bit with two bits. The first new bit is random; the second is the original bit xor the first new bit. It's clear that this is lossless (xor the two new bits to get the old bit back). It also doubles file size, and increases entropy.
Become a FSF associate member before the low #s are used
http://www.jpeg.org/jpeg2000/j2kpart1.html
I, for one, welcome our new overlords of the SIF! ...or so submits Darth Maul...
It is this last step which is not particularly efficient. There are several reasons why ZIP cannot significantly compress this data. First, ZIP is a byte-oriented compressor. Huffman codes are bit-oriented. This causes ZIP to miss many patterns that could be backreferenced simply because they fall on a weird bit position. Second, data can be highly compressible and yet have no repetitive patterns in it.
For example, consider the sequence 1,2,3,4,5,6,7,8,9,10,...,1000. It has no repeating pattern, but it would be absurd to claim that it cannot be compressed. In this case, applying the simple transform x[i] --> x[i+1] - x[i] transforms the sequence to all zeros, and that is trivially compressible. To decompress, just apply the inverse transform.
ZIP is too naive to use any sort of transforms. It really was only intended for textual data, program executables, etc.
Oops, I meant x[i] --> x[i+1] - x[i] - 1, obviously.
That's not a white paper, that's an advertisement!
The one time I really wish I had mod points. Modders with spare mod points please do so.
no.
I hope this gets incorporated into the next version of the ElecticSheep p2p distributed screensaver...
The chains are broken
Loki is free
Ragnarok is at hand...
You do realize that thebes was the one that replied to your original comment? I think you're confusing us...
I remember compressing with JPEG. Very nice to cut down on the filesize when it was necessary. But there are 2 things that made me think this article was obsolete:
a. compression at 30% and more with JPEG doesn't decrease quality in the image so that it's noticable. Only the trained eye will see the difference.
b. harddisks are huge these days. The problem of space which made JPEG-compression so popular is gone, since CD/DVD's have replaced floppies and +200GB HDD's have replaced your average 210MB HDD from 1996.
210MB was a lot of space back then, but it was nice to have JPEG, just to cut back on the size so that you could use the extra megabytes for more usefull data, and not pron...
I give massages and reiki treatments (for real!). More info here: http://www.universele-levensenergie.be
Iterated is not around because they had no marketing. They were the classic group of really, really smart phd's with a good idea, and it ended up going nowhere because no one spread the word.
I worked at a graphic arts/prepress company when Iterated was really coming on strong. We worked with them quite a bit on a prepress solution. Problem was, Iterated didn't really see the big picture, and my company was run by PHB's that didn't see the realistic view. It was frustrating then, and frustrating now, knowing the tech that's out there and not seeing it in common place.
Damn. For now.
J
Just out of curiosity, what kind of time are we talking about? Say a 1000x1000 pixel 24-bit image, what would be the max. time to compress such an image on a common machine, say a P4-2GHZ?
I personally wouldn't mind waiting a (few) minute(s) if it meant shrinking the image from 500K to 100K for internet usage.
How does a worst-case fractal compression (using any one random algorithm) compare to a good (ie. 90%) JPEG compressed file in terms of performance, filesize and quality?
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
Couldnt tell you. Havent used the program I made in ages.
But nobody has mentioned that in the linked whitepaper, the company refers to this new JPG compression algorithm as "patent-pending technology". They want others to support this new proposed file format, but nowhere do they mention whether their patent (if granted) will be licensed or under what terms. In fact at the bottom of the document a contact is listed for "Partnership and Licensing information."
I'd really be wary about supporting a new file format, only to discover later I owe someone money for patent licensing.
The reason that he's assuming that "it's not written to flash immediately" is that writing to flash is slow, and uncompressed image files are big, and it takes time to write them. If you're trying to do back-to-back shots, 1 second apart, you might not be able to do write the first 18-meg image from RAM into flash before you want to take the next shot. So your choices are "sell a camera that's too slow to take back-to-back pictures" or "include more RAM". Most cameras include enough RAM for one or maybe two uncompressed pictures plus the amount they need for compression calculations. Also, using up flash limits the number of pictures you can take. These days, an extra 72MB of flash is only ~$20 or so, but that's still non-trivial on a cheap camera, and it reduces the number of pictures you can store significantly, as well as requiring the storage space management algorithms to be more complex (e.g. if you ever store an uncompressed picture, you need to also reserve enough storage to hold the compressed version, and you don't know how tightly it will compress so you need to be conservative.)
Battery life has been a major issue for any camera I've used that had enough flash memory to not run out of storage first. The way you design for maximum battery life is to find out what components use the most power and keep them small or run them for short periods of time or put them in standby mode when you can. RAM is power-hungry, and quadrupling the amount of RAM is going to increase the amount of power you need even when you're not using it, unless you get really fancy and make some of the RAM power down when it's not active, which is probably very difficult.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
The problem is that wavelets are a CPU hog, and digital cameras don't have any to spare. One estimate I saw was that JPEG2000 needs about 5 times as much CPU as JPEG. It didn't say how much CPU it takes to decompress JPEG2000, but the decompression direction is usually much faster than compression and usually runs on devices with enough horsepower. A 20% savings in memory capacity probably doesn't justify the costs of the faster CPU, bigger battery, larger case, etc.
I've got one cheapo $29 camera that seems to make JPEGs that are twice as big for the same number of pixels as other digital cameras I've used - that probably let them use a cheaper CPU, and it's still a battery-burner (and they probably decided that a $29 camera could get away with only storing 24 pictures instead of 48. If only they hadn't done such a blazingly bad design in other ways - you can't change the battery without it resetting back to Picture#1, so it'll overwrite earlier pictures you haven't downloaded, and it uses a leftover Twain driver instead of using its USB as a disk drive like everybody else, so you can really only use the thing if you upload it to your laptop frequently.)
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
even on a 3 GHz machine that hour has only come down to 6 minutes...
Processing has improved very much beyond just the factor evident from the clock frequency. A much bigger chunk of the working set can now fit on the on-die cache, and many other things contribute to the speedup, so it's probably far less than 6 minutes. It's still a major delay, but less than that.
[an error occurred while processing this directive]
Here, arithmetic encoding explained:
x t
x t this performs half as good as the stuffit, and the source if available NOW as a tar.gz. lets try and get 31% compression, and stuffit in thier pipe :-)
http://sylvana.net/jpeg-ari/uncompressed/READ.t
This is basically a better entropy compressor. I am wary of stuffit... this smacks of a little hype. I am glad in one way that a patent means it will be published (anyone have link to patent submission?)
*thinks of other graphics/zip patens and thier success*
mmmm burn all gifs anyone? why do we have png, and are stuffit trying to patent a tweaked arithmetic encoder? Only time will tell.
Disadvantages:
- Waiting an extra 8 seconds to download an image (to decompress)
- no incremental downloads (blurry to refined)
- less redundancy in the data, more error prone
I say all good and stuff, but the patent leaves a bad taste in my mouth.
http://sylvana.net/jpeg-ari/uncompressed/READ.t
#hostfile 0.0.0.0 primidi.com 0.0.0.0 www.primidi.com 0.0.0.0 radio.weblogs.com
Really?
so, you cut 30% of a download time, but add 8 seconds to the viewing time...
you save bandwidth, but not time. I downloaded thier test image in 4 summat seconds.
Shave 30 percent off that. then add 8 seconds. then add the number you first thought of. I cannot see this making a splash in porn.
So jpeg2000 visible in 4 seconds, thiers in 10. Plus I have a million windows open on a pr0n site, each one takes 8 seconds of real time CPU. nice going. bye bye os. to be fair they list this use (intarwebnet, not pr0n) 3rd, after archiving and posting CD's... (imagine waiting 8 seconds to load each image of a family album (or even the thumbnail!! which is EXIF'd in the image! no more!! perhaps keep exif outside the arithmetic jizzness??)) a thumbnail view of a folder would take like FOREVARRR!!
If stuffit amend thier patent with the idea to embedd this in a jpeg, and not compress the EXIF, well, you read it here first.
I suppose bandwidth arguments are diminishing at the same rate as net usage is developing? so perhaps in the future the idea of a bandwidth hog will be redundant? (after all you choose what to serve on your connection).
I say the net is racing along faster than cpu speeds... but still I am interested from a backup point of view... why did they even tag this to JPEGs? yes that is one very important application of it... I hope the patent doesn't muddy itself with terms like jpeg...
#hostfile 0.0.0.0 primidi.com 0.0.0.0 www.primidi.com 0.0.0.0 radio.weblogs.com
Which isn't even white! :-)
The type of image data contained in a SIF file will be either a compressed JPEG, a compressed 24-bit image, or a compressed 8-bit indexed image. In addition to compressed image data, the SIF format will include a thumbnail of the stored image, plus an extendable set of image attributes (metadata) that could include such information as the type of camera the image was taken on, the resolution of the picture, or even camera specific information such as any custom settings that were active when the picture was taken.
blah blah, I bet they will break EXIF standard on this... silly if they do!
I realise camera manuf. will be interested in this, I would say keep an open mind. I might want to save $10 license costs of your camera, and buy a 1gb card instead of a 700mb card.
I noted that thier 'test' images were comrpessed at 50% quality to jpegs... are they good at compressing virtually lossless (large) jpegs? no word on it as yet.
Well thier white paper doesn't go into details beyond that... so thumbnail view will be normal quick, but viewing an image will still take 8 seconds on todays CPU's.
Count to 8 from 0. 0 elephant 1 elephant....
#hostfile 0.0.0.0 primidi.com 0.0.0.0 www.primidi.com 0.0.0.0 radio.weblogs.com