FEAD Compressing Compressed Files by 50-75%?

An anonymous reader asks: "I just installed Acrobat Reader and found that it was using FEAD which claims - 'FEAD© Optimizer© significantly reduces the size of application programs on average by 50% (in some cases up to 75%, depending on the specific software), even when they are already compressed with common compression technology like ZIP or CAB.' . It seems that they optimize each application individually at thieir labs. But an average of 50% compression on already compressed binary files seems to be too good to be true. Anyone familiar with how someone may be able to achieve this?"

Comments

Marketing obfuscation

[jrpascucci]

I think that it reads as you interpret: if you put some stuff in a .ZIP, it will further compress it. But, on a very close reading, they are only comparing sizes, and not necessarily saying they are compressing the zip file.

From the article: "Netopsystems specialists combine and customize these tools and processes for each individual software product so that optimal size reduction results are achieved."

Note the following from the whitepaper: "Usually software producers compress their data by generating cabinet files or the like...Applying a conventional compression tool like WinZip or WinRAR on such data does not lead to appreciable - often negative - results."

Read strictly, this says what we know: compressing a compressed file generally doesn't work. They aren't saying they compress the compressed file here.

Note that towards the bottom, they are comparing 'lossless compressed' data to what they do.

So, here's my bet: they probably do something like crack open a cab or zip, parse a PDF, for example, for 'magic things' that can be ignored without changing the functionality ('lossy' but nothing of significance lost), or take an HTML file and strip all spaces and newlines between tags. Similar things could be done for other file types: Removing quotes and instead, magic-quoting commas in a CDF. Etc, ad inifinitum.

All in all, it's lame, but so is most software.

If you have a gigantic amount (hundreds of gigs terabytes) of different files to back up or move around, with so many file formats that you can't keep them straight, then it might be worth it. If you are lazy and it's cheap, it might be worth it. Other than that, I fail to see the real utility here - disk is cheap, bandwidth is getting cheaper, and reasonably assuming the bulk of this data is generated (an adequate assumption), you can do very similar things by fiddling around with the the output formatting in code.

J

Re:EXE compressor?

[Naikrovek]

wrong. you say "unlikely to give any benefit," but the truth is that it can be beneficial.

zip an .exe file. or gzip an .so, whatever you want. then zip or gzip (or bzip2 for that matter) the file again - the doubly compressed file will be smaller than the compressed file it contains.

this is why people zip up movie files on their sites, it does make a difference, and if you only save one meg on your 40 meg movie, and 1,000 people download it, you just saved yourself one gigabyte of transfer fees, during whatever timeframe those 1000 people downloaded.

i'd say there's a benefit. it depends on what's in the compressed files, but if you're serving this file on a high traffic site it can often pay to double compress your files.

Statistical encoders

[afroborg]

I couldn't say for sure, but it's possible that theyre just using a better coding scheme. ZIP et al use (as far as I know) variations on the LZ type compression algorithms. These are fast, but definitely not the best entropy removal methods available. Arithmetic coding OTOH is very effective, removes more entropy than LZ, LZW, or Huffman, but is slow because it needs to collect statistics on the entire file before compression. I dunno about decompression speed though Arithmetic coding is patented though, same as LZW,so not just anyone can use it. Just my $0.02...