Slashdot Mirror

It doesn't look like you're running mod_gzip on that site...any reason why not? If you did, it has been known to decrease traffic load by 4x-8x in some cases...

I'm not sure if the Cobalt distro of Apache has mod_gzip by default, but I'm gonna guess not...if so, try this before you spend the extra change on a higher service tier.

Even Slashdot is using mod_gzip...

A new HTTP command is not necessary because HTTP 1.1 supports compression as a content encoding (the "Content-Encoding" HTTP header). The mod_gzip module enables compression for Apache. As you suggested, mod_gzip can be configured to compress or not compress certain files matching given criteria.

It would simply be a matter of Apache and IIS supporting it

Or, as an alternative, how much bandwidth could be saved if webservers used compression, like mod_gzip?

From mod_gzip website:
"All you need is your current HTTP 1.1 compliant browser. All modern browsers released since early 1999 are already capable of receiving compressed Internet content via standard IETF Content Encoding if they are HTTP 1.1 compliant."

PNG files are compressed using the LZH algorithm first used in ... gzip

This is slightly off topic but almost anyone can compress text, too. Of course, JPG compression is better than text compression but every little bit counts.

You can download a copy of mod_gzip here for Apache.

The best thing I ever did for my modem users, was to install mod_gzip, which compresses every page (be that HTML or any dynamically generated PHP page) apache sends out.

Result? Up to 92% compression! My pages are loading so much faster than before, AND I am saving on my bandwidth bill.

Installation is a breeze, the mod is a beauty.

- the mod_gzip project
- scoop article on mod_gzip
- some stats for intune.org

I can't quite give a rundown of which browsers support it, but the browser will advertise its capability to support it in its request headers (for example: "Accept-Encoding: gzip, compress"). This is part of the HTTP/1.1 compliance standard, so it should be in any browser from 1999 or later.

It's quite simple to install with Apache. Also, the logging options of mod_gzip allow you to see just how much bandwidth savings you're getting and/or if the requesting browser will support the request, so just a little homework will give you all the information you need to know.

Anyway, most of your answers are here.

Has any XML specific compression algorithms been made as a module for Apache?

And there is mod_gzip for Apache, if you go the gzip route.

There is a content-encoding plugin for Apache called mod_gzip that will do the server end, for any output including dynamic. I've not tried it, but on face value it's a standards-based way of getting what you want.

I think, although I can't find it for sure, that LWP supports gzip content-encoding too, which would mean that things like SOAP::Lite and XML-RPC would benefit too.

more about the content-encoding thing

It still amazes me the # of users of my websites that still use modems. We are now planning to install mod_gzip for Apache to help modem users download our larger pages faster. It didn't seem worth it at first with folks moving to broadband, but we still found many of our users listing 'modem' as their primary access method when they register. Plus it'll reduce our bandwidth demand for users who have broadband - they'll get larger files faster too. Yeah, it adds overhead on teh server CPU, but for us its worth it since we have headroom to spare.

It's mod_gzip from Remote Communications. You can also get dedicated hardware to manage compression from Packeteer.

Also, have the Slash crew explored Apache's on-the-fly zip compression abilities (it's a separate module, I don't know the name)? It eats CPU power, obviously, but HTML can be compressed by 90% or more when zipped. The cost of more web boxen would be more than paid for by the bandwidth savings, I'd wager... especially if Slashdot is getting free hosting from it's parent company.

You are probably remembering mod_gzip. It's pretty nice as far as savings go (approx 80% savings on text/html, equating to about 20-25% on the typical image-heavy page), but it does slow your server down somewhat. Without tweaking, it cut my apache server(1.3.x) down to about 130 flat html pages per second from the IIRC 750 or so it was pushing out. (Was an old processor, so YMMV) You could do better than that if you spent some effort on it (putting tempdir into ramdisk, etc) but the point is that most of the work is CPU-intensive, which doesn't impact your speed much if your bottleneck is your I/O. Besides, CPUs are cheap.

I recommend mod_gzip; it's a simple way to cut your bandwidth on 90% of sites without shooting yourself in the foot. I can't vouch for /. though: I'm pretty sure their kung foo is stronger than mine :)

Nope, the only module Slashdot is using is mod_perl (here) The compression I'm using on the page you mentioned (glad you like it) is called mod_gzip and you can get that here. The latest release finally addressed some of the dynamic and cache related issues we were having.. Since I'm running the Shack server on BSD had to wait around for the port which came out a couple of days ago. It's improved vastly since the earlier builds which was pretty much a novelty but hard to use in any real world scenario. The only issues I've run into with this version is the transmit error showing up a little more than it should (but that doesnt break anything)
-Steve Gibson

And don't forget mod_gzip which gzips content for any browser that will accept it transparently. Users don't have to download anything new or do anything different, but your bandwidth bill drops like a stone. One of my clients tried it and reduced bandwidth by 60% overall. It rocks!

note: i have no ties to this product whatsoever, it's just a kick-ass module.

mod_gzip works with most non-ancient browsers, YMMV. Check it out

Actually, Internet Exploiter can handle gzip, as long as the web server specifies the MIME type correctly. This is how mod_gzip works. (Yes, they are slightly different things on the server side, but the same stuff happens on the client side.