Slashdot Mirror
Google Launches Brotli, a New Open Source Compression Algorithm For the Web
Mark Wilson writes: As websites and online services become ever more demanding, the need for compression increases exponentially. Fans of Silicon Valley will be aware of the Pied Piper compression algorithm, and now Google has a more efficient one of its own. Brotli is open source and is an entirely new data format that offers 20-26 percent greater compression than Zopfli, another compression algorithm from Google. Just like Zopfli, Brotli has been designed with the internet in mind, with the simple aim of making web pages load faster. It is a "lossless compressed data format that compresses data using a combination of the LZ77 algorithm and Huffman coding, with efficiency comparable to the best currently available general-purpose compression methods". Compression is better than LZMA and bzip2, and Google says that Brotli is "roughly as fast" as zlib's Deflate implementation.
What's the Weissman score?
It is a "lossless compressed data format that ompresses data
... by discarding random bits and pieces of redundant occurrences of words.
`echo $[0x853204FA81]|tr 0-9 ionbsdeaml`@gmail.com
If they want to make webpages load quicker, remove ads.
Hey, Google! I have a compression algorithm that can compress any size to a single byte. I just need a little help with the decompress and we can really speed things up.
for a better user experience. The real reason is so that ad blockers will no longer work. It will no longer be web "pages", they will be "apps". The walled garden will move to the web and have the doorway sealed shut.
OOoops. Forgot to check the Anonymous checkbox again, sexconker? I think we know who the cow troll is. Now it is time to delete his account.
But why? Cows introduce a nice element of levity.
No no no... you don't understand. It's just THAT good of a ompression gorithm.
Ompression is meta Compression, the type of compression performed by Omniscient beings. It's that good.
Stop making my browser run 500 trips to DNS in order to run 500 trips to every ad server in the world.
Also, for the everloving sake of Christ, you don't need megabytes of scripts, or CSS, or any other shit loaded from 50 more random domains in order to serve up an article consisting of TEXT. One kilobyte of script will be permitted to setup a picture slideshow, if desired. /Get off my e-lawn
if Google was serious about speeding things up, they'd make a top 1000 list of stupid websites that load a crapton of useless scripts and ads just to present simple content across the web.
the preceding comment is my own and in no way reflects the opinion of the Joint Chiefs of Staff
Brotli, Zopfli... is Google now run by Austrians, according to the Austrian nicknames?
And lossless too? I'd prefer if they lost the ads, then the compression wouldn't be needed.
7z is pretty slow. Modern internet connection speeds are already between 100 Mbps (mobile) to up to 1 Gbps (fiber optics). Of course developing countries are still at about 50-100 Mbps level.
Thus something like LZ4 compresses much less, but total time of transmission is typically better. Who cares of the compression ratio, total time to complete the transaction matters.
From the paper:
Unlike other algorithms compared here, brotli includes a static dictionary. It contains 13’504
words or syllables of English, Spanish, Chinese, Hindi, Russian and Arabic, as well as common
phrases used in machine readable languages, particularly HTML and JavaScript.
This means that brotli isn't a general purpose algorithm, but only built for the web, not more. I guess that future versions of the algorithm will include customized support for other, smaller languages, whose compression databases are only downloaded if you open a web page in that language.
It wants its bottleneck back. From what I can see it's plugins, scripts and adverts loading from fifty different sites that clog web pages, not large file sizes or what have you. Yes I get that compression is vital for an outfit like Google and they want to showcase what they've been doing but most websites don't have their traffic volume.
Let it be said that the USA resents being called a developing country.
Minimum threshold fixed. Thanks!
AdBlock developed a webpage compression algorithm that works much better than that. Although it simultaneously functions as a virus scanner, removing many malicious scripts and even most social engineering attack vectors, it not only doesn't slow down your computer, but rather makes pages load faster!
Don't waste your vote! Vote for whoever you want, unless you live in a swing state it won't matter anyways
Brotli has been designed with the internet in mind, with the simple aim of making web pages load faster.
Intall AdBlock, and it'll be faster.
No need for compression, when the ad server are the ones who slow page loading to a crawl.
Try it! Library of Babel
As websites and online services become ever more demanding, the need for compression increases exponentially.
I hope not, because we're not going to get exponential increases in compression over what we have now.
"First they came for the slanderers and i said nothing."
Oh, not that shitty spam again? Who are you? You sit at home all day, glued to Slashdot, and as soon as someone mention "adblock" you cut & paste your shit?
And drop the bold on important words. Bold or CAPS on some words instantly rings my "scammer" alarm.
Try it! Library of Babel
I was like "Haven't I seen this before?", and thinking that I had, because I work on Chrome, but then looked at:
https://en.wikipedia.org/wiki/...
which says "This page was last modified on 27 February 2015, at 18:32."
Maybe it's just coming out of beta.
It is a "lossless compressed data format..."
As opposed to what, a lossy compression formula for data?
Well hell, if you don't need the original data back I've got a compressor algorithm that'll reduce a 50GB file to 1 byte every time. Sometimes even less than a byte, like maybe 0.25 bytes. In fact it reduces the file to such a small size that just finding it again turns out to be a real challenge...
Just cruising through this digital world at 33 1/3 rpm...
Why, do you think he's a mooooooooocher?
Don't waste your vote! Vote for whoever you want, unless you live in a swing state it won't matter anyways
LOL, really? You must have a script, as soon as it sees the key word it post you shitty reply.
This is a test: AdBlock
Maybe we should all add AdBlock to our signature, mayber it would crash his script?
Try it! Library of Babel
I grabbed source, compiled and tests on latest linux kernel tarball. All compressors at 'best', so -9 for gzip, bzip2 and xz. Brotli at 11 (I think that's the highest). Brotli took twice as long as xz -9, but took 1/3 of the memory of xz -9.
82M Sep 22 14:12 linux-4.2.1.tar.xz
87M Sep 22 14:46 linux-4.2.1.tar.bro
98M Sep 22 14:13 linux-4.2.1.tar.bz2
125M Sep 22 14:13 linux-4.2.1.tar.gz
602M Sep 22 14:14 linux-4.2.1.tar
7z is pretty slow. Modern internet connection speeds are already between 100 Mbps (mobile) to up to 1 Gbps (fiber optics).
But the compression does not have to be "on the fly". Large files can be compressed once, and then transmitted many times.
So you should. I'm in Australia with a 100mbit residential connection for gods sake! :P
Truly unlimited too. For $90 AUD/month which is about $63 USD/month.
Then it needs to start acting like a developed country. It can start by having some decent telecom regulation so that internet speeds are sufficient and inexpensive (if the US can't beat Romania of all places, then it deserves to be called "third world"), and cellular coverage is as good as northern Finland while as inexpensive.
How do I block your ads for "hosts"?
So, have you figured out why privilege escalation is a bad thing yet?
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
This is not about speed, this is about GOOGLE's bandwidth. Because they process so many transactions a second, they see cost savings even for small improvements.
"lossless compressed data format that ompresses data
You've lost a "c" already.
If you don't like the cow trolls, you're free to skip over them. As for me, I regard them as "mostly harmless", and I've even posted one myself.
Il n'y a pas de Planet B.
IMHO, at first read, it sounds like it's saying it's more efficient than Pied Piper's (fictional) algorithm... Which of course is impossible, since Pied Piper's will compress everything.
I was thinking the same. The names are too much Swiss.
It goes up to eleven
It is more complicated than that.
100 Mbit/s is 10 MB/s. If the web page uncompresses 1:4, it is enough if the decompressor runs a 40 MB/s, anything faster at the expense of the compression ratio is going to make things slower (well, unless the decompression reads from a compressed local cache soon and repeatedly). Transmission, decompression and building the html tree all happen in parallel in the browser. For a 1 Gbit/s network, i.e., 100 MB/s compressed data in, you need about a 400 MB/s decompression ratio at a compression ratio of 1:4. If you have 2 GB/s decompression speed at the cost of having a worse decompression ratio, again you are going to be slower overall.
But hey, if you actually load a website with your 1 Gbit/s internet, it will still take 4-8 seconds or so for some sites. Why? Not because it is 3 gigabytes of data. Typical systems are throttled in more than one way. The bandwidth problem can be in the sending server, or somewhere in between. Perhaps you only get the full bandwidth once the connection has been open for some time. I have a 250 Mbit/s internet, and often I end up waiting for images to load. The server is somehow struggling with the load and unable to give more, or the service provider wants to save money with bandwidth costs. For all these case an increase of compression actually improves things even when your connection is super fast. It takes two to communicate.
Sepiroth!
If I have been able to see further than others, it is because I bought a pair of binoculars.
There are much better general purpose lossless compression algorithms than what is in 7-Zip (LZMA2) or WinRAR. They are also much slower, like the paq series that uses context modeling and arithmetic coding.
The challenge here is to do something that is also fast and memory efficient. In fact, we could improve the compression of Brotly very easily just by using arithmetic coding instead of Huffman coding. But the improvement probably wasn't worth the performance loss.
7-Zip isn't slow unless you tell it to be slow. You can tell it to be fast if you want to. You can tell it to use a specific algorithm with whatever parameters you want.
Besides, all static content is compress once, deliver many.
Dynamic content only needs to be compressed as often as it is updated.
I have no idea what you are doing, but it's really hilarious.
"First they came for the slanderers and i said nothing."
Of course developing countries are still at about 50-100 Mbps level.
Not even close. I think you're off by an order of ten. I was in Central America last year and it was 4-6Mbps as "High Speed" internet.
I have 3 Meg DSL when the weather is good. An electric co-op for a rural area is putting fiber in to get 1 gig to the home over a 5 year rollout. It unfortunately will stop about 1/2 mile from my current location. If I move to the middle of some farmland I'll have far better Internet - that's just weird.
if only we had some unified app framework so that you didn't have to install a new app for every site. you could give it a little text bar at the top where you would type in the name of the app's data you want, and instead of having to have the app installed, it would just load the data FROM the source the app loads from, and then render it. you might have to add in some markup language tags here and there to help with the formatting, but it should be doable. this would save a lot on internal storage on the phone.
Google's algorithm addresses those issues by having a fixed dictionary. Compression is faster because there is no need to compute the dictionary, transmission is faster because there is no need to transmit the dictionary, and decompression uses less memory and is faster because the dictionary is shared between multiple streams.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
Depends on what you're compressing. Most large static content (images, video) are already compressed. This is aimed at HTTP-level compression, so the output from whatever is generating your dynamic pages will be streamed through the compressor on the way to the socket and then through the decompressor at the far end.
I am TheRaven on Soylent News
The very fact that your software requires a privilege escalation every time it updates indicates you don't understand security. Most people have no way to know what your software does when it updates, what is to stop you from having a Trojan load on your update? There is nothing, whereas your "rival" adblock requires no escalation.
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
Bob and weave, bob and weave.
You just don't get security do you? In your mind it is all about trust. Security isn't about a chain of trust, there is the verify step there too, which is not possible with your application. But go ahead and keep being wrong, it is what you are good at.
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
My antivirus software requires privileges in order to install, not to update.
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
Antivirus doesn't use elevated privileges to update.
Adblock doesn't use elevated privileges at all
Your host file software isn't a virus scanner, nor does it replace one.
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
Really?
https://www.sophos.com/en-us/s...
How do you stop conflicker with your Hosts file, or any of the numerous viruses that spread through network protocols on local networks?
http://www.thorschrock.com/200...
It looks like you haven't changed much, did it ever occur to you that maybe people respond the way they do because of your hostility?
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
It is nice to see you still don't understand how security works.
2. Spreads via Windows file sharing
Once on the network the virus can spread using the Microsoft exploit (above) or by accessing the file and admin shares on the network.
When it infects a computer it creates a file with a random name and a random extension within the System32 folder. A scheduled task (running as SYSTEM) will execute this file using rundll32.exe.
3. Spreads via removable media such as USB drives
When a removable drive is connected to an infected computer, the Conficker worm will
create a copy of itself in the RECYCLER\S-x-x-xx-xxxxxxxxxx-xxxxxxxxxx-xxxxxxxxx-xxxx folder on that drive (where x consists of random numbers)
drop the file autorun.inf in the root director of the drive.
How will your hosts file stop either of these attacks? It can't, which is why I mention this particular virus, but there are many like it that you have no defense against if you use hosts files instead of virus scanners.
Good luck with your Pwning.
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
This is where you say your hosts file stops all threats:
* My program utterly DISPLACES THE NEED FOR ANTIVIRUS since it blocks you getting infected in the 1st place (negating the need for antivirus @ all) & then it also stops botnets from communicating with their C&C servers even IF you get an infestation...
How can it displace the need for antivirus when it can't stop most of the threats?
What I have done is secure numerous systems on the internet. I have followed many procedures, but not one suggests editing a system file such as hosts. You are the one overloading and abusing the hosts file which wasn't meant for that purpose, not I. I don't have to defend what I do, I am not claiming to be the security expert like you are. I am not claiming to have a product that 100% protects you from all threats on the internet better than antivirus, malware protection and ad blocking software; you are.
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
I can't help if you are so forgetful. I didn't post AC claiming to be APK:
http://tech.slashdot.org/comme...
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
Again, I will explain to you as you seem to be mentally challenged. Administrator privilages used when installing a program (ONCE!) is not the same from a security perspective of administrator privileges every time an update is needed.
And your hosts file does not replace AV, you said it did, now eat your words.
http://slashdot.org/comments.p...
http://slashdot.org/comments.p...
http://slashdot.org/comments.p...
I didn't put any words in your mouth, you did it yourself, now eat them and go home. You have ZERO security chops, and it shows, you argue with people by simply attacking them, but you never answer the actual questions, just bob and weave Kowalski.
http://www.subzin.com/quotes/M...!
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
As always that is going to update yourself with old algo. So be prepare for new update or may be it will be rolling out with old one. Even they can give the best result as we know that is in google.