FEAD Compressing Compressed Files by 50-75%?

An anonymous reader asks: "I just installed Acrobat Reader and found that it was using FEAD which claims - 'FEAD© Optimizer© significantly reduces the size of application programs on average by 50% (in some cases up to 75%, depending on the specific software), even when they are already compressed with common compression technology like ZIP or CAB.' . It seems that they optimize each application individually at thieir labs. But an average of 50% compression on already compressed binary files seems to be too good to be true. Anyone familiar with how someone may be able to achieve this?"

Comments

This is a common hoax.

[Futurepower(R)]

This is a common hoax. Maybe 2 years ago, another Slashdot editor posted this hoax. So, it's a repeat hoax for Slashdot, too.

Re:This is a common hoax.

[thumperward]

remember this comment well, for when you get the opportunity to meta-mod the parent post's crack-addict "Informative" rating into the Netherworld./b.

It's not compression

[photon317]

The thing they tout as FEAD is basically a load-over-network-on-demand thingy. They haven't actually developed anything that does compression, they're just storing some of the app on a server somewhere to be downloaded on demand. The hype at their site mislead you, like it was meant to do.

Re:It's not compression

[bobbozzo]

When you download Acrobat, it usually will download an "Adobe Download Manager" or something like that.

That is NOT what is being discussed here.

Even if you bypass using the download manager, it still uses FEAD to decompress and install AcroRead.

One could easily disprove your theory by unplugging their net connection during the FEAD decompression... Done... no adverse affect.

Nonetheless, the installer is VERY slow, and is still bigger than the AcroRead 5.1 installer, which did not use FEAD.

Making users go through this many steps (download the download manager. run it. wait for it to download. wait for fead...) and slowness is insane.

Compression is easy

[cybermage]

It's decompressing the file that's hard.

You can compress all your files down to a single bit using this patented two step process:

1. Discard all zeros.
2. Use one to represent any length sequence of ones.

This is as reliable a compression scheme as most backups to tape I've ever seen, and you can fit a huge number of files onto a single floppy.

Re:Compression is easy

[optikSmoke]

Thats too easy.

First, you expand the 1 to the requisite number of 1s:
1 -> 1111111

Then, reinsert the 0s:
1111111 -> 1101101001001

Thus, 1 -> 1101101001001

Marketing obfuscation

[jrpascucci]

I think that it reads as you interpret: if you put some stuff in a .ZIP, it will further compress it. But, on a very close reading, they are only comparing sizes, and not necessarily saying they are compressing the zip file.

From the article: "Netopsystems specialists combine and customize these tools and processes for each individual software product so that optimal size reduction results are achieved."

Note the following from the whitepaper: "Usually software producers compress their data by generating cabinet files or the like...Applying a conventional compression tool like WinZip or WinRAR on such data does not lead to appreciable - often negative - results."

Read strictly, this says what we know: compressing a compressed file generally doesn't work. They aren't saying they compress the compressed file here.

Note that towards the bottom, they are comparing 'lossless compressed' data to what they do.

So, here's my bet: they probably do something like crack open a cab or zip, parse a PDF, for example, for 'magic things' that can be ignored without changing the functionality ('lossy' but nothing of significance lost), or take an HTML file and strip all spaces and newlines between tags. Similar things could be done for other file types: Removing quotes and instead, magic-quoting commas in a CDF. Etc, ad inifinitum.

All in all, it's lame, but so is most software.

If you have a gigantic amount (hundreds of gigs terabytes) of different files to back up or move around, with so many file formats that you can't keep them straight, then it might be worth it. If you are lazy and it's cheap, it might be worth it. Other than that, I fail to see the real utility here - disk is cheap, bandwidth is getting cheaper, and reasonably assuming the bulk of this data is generated (an adequate assumption), you can do very similar things by fiddling around with the the output formatting in code.

J

Re: Marketing obfuscation

[Black Parrot]

> Similar things could be done for other file types: Removing quotes [...]

When I have files with lots of quotes in them I reduce the size by using single quotes instead of double quotes.

How to do anything

[kurosawdust]

"Anyone familiar with how someone may be able to achieve this?"

"Lying through one's teeth" comes to mind...

Re:How to do anything

[natmsincome.com]

I ready the white paper and it looks like they are actually providing a number of products:

Download on demand:
Think Quicktime/IE/etc. Download a small download then download the components they want. I expect they'd also use a protocol similar to rsync which makes downloading alot faster.

Code Inspection:
This is where they say then can decrease to size of the executable. If you've done progaming then you know that you can make executables alot smaller. Here's some examples: Remove inline macros and make them functions, remove debug information.

Ziped executables:
You can have realtime zipped executables. There are about 5 different forms of this that I've seen - they zip/encrypt the executable this makes it about the same size as if it was zipped but you don't have to unzip it to run it(It still uses the space in memory) You also have to break the encryption before you can reverse enginer it. The overhead is about 5 - 20% loading times.

Basically the provide servers that campanies could do themselves but get someone with experiance to do instead.

If I did the following:
*Used realtime compresion on the exe
*Optimized my code so that it didn't incluse useless code.
*Moduralised my code (reuse etc)
*Made the code more plugin like.
*Added a bootstrap downloader.
*Make the software come in three versions Lite, Full and express.

If I did all of this I'd easily be able to half the bandwidth needed for a file without really changing it that much.

As for the CD the data has to be over 700MB before they can decrease the size so I'm fairly sure they'd be able to optimse it.

The service they provide isn't unique. It's just a convient package of 10 or more technologies to make life easy for other companies.

i know this one

[onya]

by employing the latest in smoke and mirrors technology. they've invented a new mirror that reflects 110% of all light. neat huh?

Wow.

[Lendrick]

That Site (c) is an Eyesore (c). I wonder if these Dipshits (c) realize that all those "(c)" marks make their Site (c) Difficult (c) to Read (c).

Re:Wow.

[stefanlasiewski]

Difficult (c) to Read (c).

It may be difficult to read, but it sure is easy to FEAD© !!!

Re:Wow.

[Kelerain]

Difficult to read, but easy to compress.

EXE compressor?

[almightyjustin]

Sounds to me like an EXE compressor like UPX - they can compress EXE files better than a ZIP archive can (by taking advantage of known aspects of executable files); so by unzipping, EXE-compressing, and re-zipping, one can reduce the size of an already existing ZIP archive.

Re:EXE compressor?

[Naikrovek]

wrong. you say "unlikely to give any benefit," but the truth is that it can be beneficial.

zip an .exe file. or gzip an .so, whatever you want. then zip or gzip (or bzip2 for that matter) the file again - the doubly compressed file will be smaller than the compressed file it contains.

this is why people zip up movie files on their sites, it does make a difference, and if you only save one meg on your 40 meg movie, and 1,000 people download it, you just saved yourself one gigabyte of transfer fees, during whatever timeframe those 1000 people downloaded.

i'd say there's a benefit. it depends on what's in the compressed files, but if you're serving this file on a high traffic site it can often pay to double compress your files.

Compressing data exe compressors don't

[TheSHAD0W]

When you use an executable compressor, like PKLITE, on an executable file, it can't compress all the data. This is because EXEs will dynamically load more data, and if that data is compressed, the code can't read it.

I suspect these guys are going in and manually altering the code to perform a decompression. This would certainly produce a benefit.

Here's something for you to try: Take an executable and zip it. If it compresses, then there's probably SOME give in it. And most executables I see are compressable.

Hmm...

[bobbozzo]

I just RAR'd (RAR 3.11 with -m5) my AcroRead folder... it came out 300k bigger than the 16MB full installer...

Using ZIP -9 gives a 20MB file.

So, FEAD offers slightly better compression. (I know there's other crap, including the installer, registry settings, icons, ...)

Still, is it worth the annoyance of the greatly increased install time?

Also, how is FEAD saying they are 50% better than other compressors?

I have a better algorithm

[Skim123]

I have a far superior algorithm in both time and space complexity. Start with 1. Then simply transform it to the requisite number of 1s and 0s, a la 1101101001001. Bah to your two-step process. :-)

Statistical encoders

[afroborg]

I couldn't say for sure, but it's possible that theyre just using a better coding scheme. ZIP et al use (as far as I know) variations on the LZ type compression algorithms. These are fast, but definitely not the best entropy removal methods available. Arithmetic coding OTOH is very effective, removes more entropy than LZ, LZW, or Huffman, but is slow because it needs to collect statistics on the entire file before compression. I dunno about decompression speed though Arithmetic coding is patented though, same as LZW,so not just anyone can use it. Just my $0.02...