Slashdot Mirror
Google Launches Brotli, a New Open Source Compression Algorithm For the Web
Mark Wilson writes: As websites and online services become ever more demanding, the need for compression increases exponentially. Fans of Silicon Valley will be aware of the Pied Piper compression algorithm, and now Google has a more efficient one of its own. Brotli is open source and is an entirely new data format that offers 20-26 percent greater compression than Zopfli, another compression algorithm from Google. Just like Zopfli, Brotli has been designed with the internet in mind, with the simple aim of making web pages load faster. It is a "lossless compressed data format that compresses data using a combination of the LZ77 algorithm and Huffman coding, with efficiency comparable to the best currently available general-purpose compression methods". Compression is better than LZMA and bzip2, and Google says that Brotli is "roughly as fast" as zlib's Deflate implementation.
What's the Weissman score?
It is a "lossless compressed data format that ompresses data
... by discarding random bits and pieces of redundant occurrences of words.
`echo $[0x853204FA81]|tr 0-9 ionbsdeaml`@gmail.com
If they want to make webpages load quicker, remove ads.
Hey, Google! I have a compression algorithm that can compress any size to a single byte. I just need a little help with the decompress and we can really speed things up.
Stop making my browser run 500 trips to DNS in order to run 500 trips to every ad server in the world.
Also, for the everloving sake of Christ, you don't need megabytes of scripts, or CSS, or any other shit loaded from 50 more random domains in order to serve up an article consisting of TEXT. One kilobyte of script will be permitted to setup a picture slideshow, if desired. /Get off my e-lawn
if Google was serious about speeding things up, they'd make a top 1000 list of stupid websites that load a crapton of useless scripts and ads just to present simple content across the web.
the preceding comment is my own and in no way reflects the opinion of the Joint Chiefs of Staff
7z is pretty slow. Modern internet connection speeds are already between 100 Mbps (mobile) to up to 1 Gbps (fiber optics). Of course developing countries are still at about 50-100 Mbps level.
Thus something like LZ4 compresses much less, but total time of transmission is typically better. Who cares of the compression ratio, total time to complete the transaction matters.
From the paper:
Unlike other algorithms compared here, brotli includes a static dictionary. It contains 13’504
words or syllables of English, Spanish, Chinese, Hindi, Russian and Arabic, as well as common
phrases used in machine readable languages, particularly HTML and JavaScript.
This means that brotli isn't a general purpose algorithm, but only built for the web, not more. I guess that future versions of the algorithm will include customized support for other, smaller languages, whose compression databases are only downloaded if you open a web page in that language.
Let it be said that the USA resents being called a developing country.
Minimum threshold fixed. Thanks!
It is a "lossless compressed data format..."
As opposed to what, a lossy compression formula for data?
Well hell, if you don't need the original data back I've got a compressor algorithm that'll reduce a 50GB file to 1 byte every time. Sometimes even less than a byte, like maybe 0.25 bytes. In fact it reduces the file to such a small size that just finding it again turns out to be a real challenge...
Just cruising through this digital world at 33 1/3 rpm...
Websites maybe, web applications are a different story. Problem is that businesses make heavy use of web applications and data lists can grow out of control quickly if you include the formatting required to make it look right. One could say that these web application have bad data presentation strategies and I would argue that you'd probably be right in many cases. Unfortunately not all problems can be solved with forced filters and paging.
Take /. for example. The HTML alone of this page (with very few comments at this point) is 200KB. If you add up the CSS and JS you are well above 1MB. Data alone would probably take only 100KB but data that's hard to decipher through is not fun data hence the overhead.
Compression is a no brainer if used properly. Bandwidth is limited and so is processing power. Just a matter of deciding which one is more important at any given time.