New Breed Of Web Accelerators Actually Work

axlrosen writes "Web accelerators first came around years ago, and they didn't live up to the hype. Now TV commercials are advertising accelerators that speed up your dial-up connection by up to 5 times, they say. AOL and EarthLink throw them in for free; some ISPs charge a monthly fee. Tests by PC World, PC Magazine and CNET show that they do speed up your surfing quite a bit. They work by using improved compression and caching. The downside is they don't help streaming video or audio." And they require non-Free software on the client's end, too.

Comments

Faster porn?

[DigitalNinja7]

Unfortunately, these caches store only the most accessed pages, so anything of any value to the Slashdot audience will be as slow as ever. But you can be sure your porn will be delivered at 5x the speed of your normal dial-up! (yawn)

Re:Faster porn?

[Anonymous Coward]

Are you saying Slashdotter's don't value pron?

Re:Faster porn?

[onecrazyfoo]

Actually, how it was described to me, is that the requested page is retrieved by the ISP's server, cached and compressed, then sent along to the client. Which, with the compression they are able to get, is much faster for the dial-up user. At least that is how is supposed to work with Slipstream's product (what NetZero uses).

Re:Faster porn?

[Cramer]

1) If you're using dialup, the speed of the internet will not be a bottleneck (slashdot effect not withstanding.)

2) If you're already using compression (stac, predictor, MPPC, etc.), this will make ZERO difference. The cache has to be on the near side of the slowest link -- which is the dialup user's modem. Now, in the instances where the ISP disables software compression -- like, for instance, the "idiots" at Bellsouth.Net who disable CCP to "speed up connection times" [exact words of the Cisco engineer who helped them set things up] (the time it takes to connect and pass traffic is 100% modem training. For us ISDN users, 3 of the 3 seconds it takes to connect are IPCP; I'll accept that as they do tend to return the same IP most of the time) -- it'll help some.

3) A lot of what's moving around the internet isn't measurablly compressable... GIFs, JPGs, mpegs, zip files, etc. (I shall have to perform an analysis.)

You mean...

[HungWeiLo]

"Web accelerators"...You mean highly-advanced technology like mod-gzip?

Re:You mean...

[pla]

"Web accelerators"...You mean highly-advanced technology like mod-gzip?

Sounds pretty much like that... Which Apache already supports, and the major browsers already support, making something like this redundant.

Moreover, dialup modems already use a fairly high level of compression at the hardware layer. While not exactly "gzip -9" quality, you can only realistically squeeze another 10% out of those streams no matter how much CPU power you devote to the task.


Others have mentioned image recompression, which has traditionally used VERY poor implementations, nothing more than converting everything to a low quality JPEG. I would point out that a more intelligent approach to image compression could yield a 2-3x savings without noticeable loss of quality (smoothing undifferentiated backgrounds, stripping headers, drop the quality a tad (ie, to 75-85%, not the 20-40% AOL tried to pass off as acceptible), downgrading the subsampling on anything better than 2:2, etc). But no, not a 5x savings.

Re:You mean...

[Anonymous Coward]

Uh, no. mod_gzip will compress _much_ better than your modem. Your modem can only compress in small chunks, and so there is less redundancy to remove. Even small(ish) web pages can be compressed, as a whole, to a much, much smaller size than what your modem can.

In fact, using mod_gzip I've experienced a very discernible speed-up in access and display over DSL!

Re:You mean...

[pla]

why is it that downloading a 1meg file full of ZEROS still takes ages on a modem?

Because v.42bis has a maximum compression ratio of 4:1 (MNP5 only does 2:1).

Now, for a file of all zeros, hey, I agree, you can do a lot better. So, how often do you download files containing nothing but zero? For a typical text file, you might get better than 90% with gzip (while still only 75% from v.42bis). But from binary content? very rarely better than 50%.


In any case, web content consists of five basic types of information - Text, graphics, sound, multimedia (flash, MPEGs, AVIs, etc), and already-Zipped packages.

Of those, only the first benefits from any lossless method, and only the second really leaves any room for saving bits via lossy compression without horrible loss of quality at the same time. (Some of the fourth type could also possibly endure lossy compression, but takes far too long to recompress on-the-fly).

Unfortunately, text comprises the least bothersome (in terms of relative size) of all of those major types of web content.


Don't get me wrong, I fully encourage people to turn mod_gzip on in their Apache installs. But When a company hawks its product with claims that simply cannot occur in a normal webbrowsing situation, I have to call foul.

I see only two situations whereby they could claim 5:1 compression - Either VERY text-heavy material, such as reading something from Project Gutenberg, or they strip every possible non-critical image from a page. I already do the latter via my hosts file and a paranoid userContent.css, so what does that leave?

Hope you only like reading text, in which case, have you ever heard of "Gopher"?

Re:You mean...

[flug]

>Now, for a file of all zeros, hey, I agree, you
>can do a lot better. So, how often do you download
>files containing nothing but zero?

Er, every time I visit Slashdot?

I still don't understand

[Anonymous Coward]

Why so many content providers aren't using gzip compression? The cpu time required is MUCH cheaper than the bandwidth, AND it makes users happiers because they get it faster. Oh, and it's free (for Apache anyway) and easy to set up. It even works with 99% of browsers these days.

Yeah, right!

[Anonymous Coward]

And they require non-Free software on the client's end, too.

And I'll just bet that none of that software includes any popups, spyware or intrusive monitoring!

But really, why?

[agent dero]

This seems like a really niche market nowadays. Not _too_ many people that need fast internet, that could use this, don't have broadband availible. The one key thing is price, which is even starting to get iffy.

Something like $10-20 monthly for "speedy" earhtlink dial-up, or an extra $10-20 slapped on my monthly cable bill for broadband? (Charter Communications, they suck anyways)

I guess if you need to read /. or pr0n that much fast, it works, tell me if I am wrong, but I am seeing a small market for this much hype

Next thing...

[koi88]

They actually work?
Next thing they find out is the new generation of penis enlargement devices actually work, too...

Non-Free software?

[divisionbyzero]

OMG, not that! I know this won't get much play here, but I don't care if it's free or not as long as it works. I use the Free software that I do because it is better than Fee software, not because it is free. Shame on me for not being an ideologue.

tradeoff

[nstrom]

It's just the old tradeoff between CPU power (decompression time) and bandwidth usage (download time). Much easier (and more smartly) implemented on the server side with something like mod_gzip, like HungWeiLo said.

And graphic compression's been done before too, since around AOL 3.0 or so. Most people turn it off because it makes pages look like crap.

What the hell?

[elwoodblues16]

Snake oil that works? What do you even call something like that?

They aren't really that great.

[Jerk City Troll]

My former company was checking out NetAccelerator recently to resell to our clients.

These things are a joke. The primary performance increase comes from recompressing images into really nasty JPEGs. AOL was doing this years ago (and getting blasted for it). If you turn that off, the performance improvement is not even measurable.

Furthermore, you tend to get a lot of stale caches on your machine. Most browsers don't even get this right, so they add yet another layer of potentially buggy cache abstraction.

No, these things are junk. They act as proxy servers and their source is closed. How can you trust them to handle your data? Even with all their compression features turned on, the performance improvement is seriously overrated. Don't bother. You simply cannot get something for nothing in cases like these.

Now, what would improve the download speed of the web is if web designers would start building standards compliant markup. Many web sites have as much as 700kb overhead in markup from tools that create loads of font tags and their ilk. Pure XHTML + CSS layout would do a hell of a lot more to speed up the web than these scams. Of course, don't take my word for it--read Zeldman.

Re:They aren't really that great.

[Reziac]

AOLpress makes the cleanest, most legible HTML you'll ever see (better than most people bother with their hand-crafted HTML, in fact). It's also utterly anal about correct tags. I use it as a validator and code beautifier even when I've built the page in something else. Between that and its ability to work as a browser (a huge timesaver when a site is all unique pages and you need to follow links back and forth between several of 'em as you edit), it's completely ruined me -- now I expect *every* editor to do as well :)

Save As HTML woulda been my first thought too.. except knowing the kark that Word thinks is HTML, I'd probably do this instead: save as WordPerfect 5.1, then (assuming I didn't have WP available to cut the middleman) I'd run it thru one of the WP-to-HTML tools, which usually do pretty well on tables, then load and save in AOLpress to clean up artifacts and mismatched tags. And when it magically appears on the website well before the deadline, the boss thinks you're a genius. :)

Compressing the already compressed?

[insecuritiez]

They compress the packets of data. Where will this help? In compressible places that aren't already compressed. Such as the HTML markup for webpages. This wont help already compressed JPGs, or already compressed MP3s or already compressed ZIP/GZIP files or already compressed videos (MPG/AVI/ASF). So is this really going to help much? Sure, there is always going to be a small percent of space (and therefore time) saved even transferring these formats. Is it going to make a 5X difference? No. Is it going to make a noticeable difference? It's unlikely but possible. The only way this "new technology" is going to help is if you are a dialup user without broadband options.

rproxy -- also actually works, and open source

[mattbee]

rproxy is a really interesting project, and back when I tried it over a 56K dial-up connection, it did actually work to speed things up. You sit an rproxy web cache at each end of the dial-up connection (so you need somewhere to deply your custom proxy to make it work, but bear with me...) and then request web pages as usual. Each end caches the pages that pass through it, but the clever part is that when you re-request a page, the proxy at the far end (on the fast connection) can fetch the page and compare with the last copy in the cache. Then it transmits only the differences using the rsync algorithm. Unforunately it's not being actively developed any more given the increasing availability of high-bandwidth connections, and the decreasing fraction of web traffic that is suitable for delta-compression. Shame, since it did seem to be a real "web accelerator" without any of the illusory techniques used by the garish banner-ad accelerators.

Re:Does anybody really care?

[toomuchPerl]

I'm ...
on ....

dialup....
you....
insensitive ...
clod!

And my connection is wheezing just trying to post this!

because IIS's is garbage

[DrSkwid]

mod_gzip is manna from heaven

I turned mine off by accident once and got a phone call from the co-lo wanting to know why I was suddenly maxing out.

gotta love that 70% saving.

Re:because IIS's is garbage

[Micah]

Of course, I should also add that both numbers would be a lot lower if the Slashcode theme remotely resembled web standards instead of horrendous amounts of nested tables and "spacer" graphics, but that's getting off-topic.....

Re:because IIS's is garbage

[Jerf]

Of course, I should also add that both numbers would be a lot lower if the Slashcode theme remotely resembled web standards instead of horrendous amounts of nested tables and "spacer" graphics, but that's getting off-topic.....

Actually, try downloading your page, copying it, gzipping the original, cleaning up the copy to your specs, gzipping that, and comparing the two file sizes. While you may kill a lot of text in the uncompressed version, I would strongly suspect you'll find that the gzip'ped version saved much less then you think.

Those "spacer gifs" that take up perhaps 100bytes apiece in the original file (perhaps a bit generous) will compress away down to very little (if there are several near each other, they may literally compress down to a handful of bits after the first one), whereas the story text compresses much less well.

If you're compressing things, XML, CSS, and a lot of other things that look awfully redundent in plain-text are suddenly downright bandwidth-efficient technologies, being dwarfed in their compressed representations by the plain-text payloads. This is one of the reasons that fundamentally XML is so cool; you get human readability, but for the very small effort of invoking gzip or similar compresion technology, you also get something that is very nearly as bandwidth-efficient as possible, because compression technologies dynamically determine the best binary encodings for such messages (including their plain-text payloads), whereas supposed "efficient" binary protocols may actually waste a lot of space. (Compressing the both of them may equalize them, but the binary file, perversely, will still be "harder" to compress, even with nearly the same information in both files.)

How compression behaves is not necessarily intuitive.

Squid & mod_gzip

[chill]

ISPs could simply put some squid caches between the net and their dial-up banks. Turn mod_gzip on and you'll accomplish a lot of the same thing.

Instead of having to traverse the Internet, with all the associated latency, pages are pulled locally - 1 hop away. Pages are also compressed.

A better way would be to figure out how to transfer pages via CVS, so only .diffs came across. :-)

Just remember

[El]

GIFs, JPEGs, MPEGs, and MP3s are already compressed, so compression doesn't make them any smaller. That really leaves only HTTP, HTML and CSS to benefit from compression. And caching only helps if you're in the habit of looking at the same pages multiple times... so where's the benefit for the average porn-downloading, RIAA-infringing geek? Does it speculatively preread links before I click on them?

Re:Just remember

[MerlynEmrys67]

GIFs, JPEGs, MPEGs, and MP3s are already compressed

For a given representation these are all compressed. However in all cases these have lossy compression, where you can degrade the quality of the final output and send a smaller bitrate over the wire. Want me to prove my point... Take your favorite CD quality MP3 - lets say the track is 100 K. Now take it and convert the quality to minimum quality - the file will be like 20 K now (if even that much)... you can still hear what is going on... but the quality will suck. Can do the same thing with the rest of the compressed formats as well.

Re:Just remember

[MerlynEmrys67]

Yup... this is exactly what they are doing... Remember I have a local proxy cache - and multiple T-3 links to the internet - you have a 33kbit connection to this. If I can get a 100K file - spend time compressing it by 5x and get it to you in less time than it would take you to get the 100K file (24 seconds right) I have won. And guess what - the next sucker that asks for it, I get to give the recompressed data too for free.

In many cases CPU power on the internet is free, bandwidth is expensive and worth spending free CPU cycles dealing with... Oh - how do YOU know that you are getting a degraded image anyway ? the average idiot going through an ISP that would do this only sees the internet this way.

Nasty JPEGs?

[Hayzeus]

OK -- now you'vegot my attention. Like, um... just how nasty?

Another class of people who can't get broadband

[yerricde]

You admit that a $200,000 setup fee isn't "a few bucks more." Thank you; most people miss this.

But what about people who are so mobile that they need to be able to jack in and access the Internet from any of several locations, and they can't afford the price of a broadband subscription for each location? I was in just that situation for four years. Dial-up has the advantage of a last mile in almost every home in the States, brought to you by the Universal Service Tax, meaning that no matter whose house I was visiting, I could always plug my laptop into the wall and dial my Verizon Online account.

Wow, what an amazing concept!

[default luser]

WOW, a webcache and real-time compression!

My browser and my modem with .v42Bis compression have only been able to do that for, what, nearly a decade?

Re:Awwww boo hoo

[stratjakt]

Nah, this is different altogether. Gzip is not the alpha and omega of compression.

Different algorithms lend themselves better to different applications, so it seems to me a good accelerator would use a mix of algorithms based on MIME type.

Ie; is the source data formatted in 24 byte words? 16 bit words? 8 bit words? If you have 8 bit data you don't want to look at 16 bit chunks, because then the string "abacadaeafag" doesnt compress for you. Dictionary sizes and blah blah blah... Even format conversion - turn all those BMPs that dingbats put on their pages into PNGs or lossless jpegs..

And as for caching, it seems to me like more of a prefetch than a squid-type cache.. Ie, you request page, proxy at IP gets page, compresses it on the fly, then sends it. Caching it locally is more of an advantage WRT latency, not throughput.

There's a lot of common sense tricks you could use. And according to these articles, they work.

Free Web Accelerators

[wang232]

There are two free software projects building web accelerator proxies. One is RabbIT . The other is ziproxy. They are both web proxies which do not require any special software on the client side. They both compress HTML by gzip, and compress images into lower quality JPEG's. RabbIT is written in JAVA whereas ziproxy is written in C. RabbIT has more features than ziproxy, such as caching and removing ads. Give them a try if you're using a slow line! Disclaimer: I'm a ziproxy user and developer.

Oh yeah?

[identity0]

If they're so good, why isn't this first post?!

A little bit of insight..

[ewhenn]

Netzero offered this a while ago (maybe they still do). Basically it does speed up loading of pages greatly, however there is a drawback, and a big one at that, the pictures look like crap. The GIFs/JPEGs/etc are compressed. Compressed so much to the extent that they look like, for lack of better terms, crap. A rule of thumb applies here, if it sounds too good to be true, it probably is. You really cant expect broadband speed for the cost of a dial up, and if you do I have some lovely penis pills to sell you for the low low price of 69.95.

Tests by PC World, PC Magazine and CNET show ...

[noahbagels]

That's about all the article had to say:

Tests by PC World, PC Magazine and CNET show

These are the same magazines with full color, multi-page reviews of the new 0.025% faster hardware. They are the same magazines that review each micro$oft product and say that the TCO is lower than ever before. Take one look at any of their websites, and you will see:

These magazines are Advertisements

Taking anything from them seriously is like taking a presidential speech to be a serious economic discussion, or taking a realtor's web-site as gospel in the market.

Funny - just went to CNET.com to research my post, and guess what? Over 50% of the page is advertising. The rest is 'reviews' of which 100% have links to affiliate programs to purchase said hard/software and give a kickback to CNET.

They will try hard to sell anything, and get their commission. It's like they are the used car salesman of the internet - only everything is new and they don't look you in the eyes when lying to you.

Can't compress twice

[fm6]

I would prefer more browsers to use gzip for retrieving web pages.

I might be wrong, but I'm pretty sure this would have not help dialup users at all. They're already using hardware data compression in the modem. When you're using lossless compression, there's an absolute limit as to how much compression you can get -- and you can't get around that limit by running your data through multiple compressors.

(I should check this out by timing various downloads, but I'm too lazy. Somebody else can prove me wrong!)

So why do JPEG files with "more" compression download faster? Because JPEG is a lossy format: when you increase the "compession" you're not encoding data more efficiently, you're throwing data away. Depending on the image, you can do this and still end up with something that looks the same. But push it far enough and you end up with crap.

Re:Can't compress twice

[HeghmoH]

You're making the incorrect assumption that all lossy compression is equal.

Modem compressors work very, very poorly. This isn't just because the people who come up with them suck, there are fundamental problems with doing compression in the modem. In order to avoid introducing really horrible latency, you have to compress the data in fairly small chunks. You can't wait for 50k of data to arrive from the computer and compress it all at once. Yet any decent compression scheme will achieve better compression ratios on longer chunks of data than shorter ones in non-pathological cases. So you have a modem which is stuck trying to compress a hundred bytes at a time, and a web server which can compress all 100k of page at once, and you have a significant difference in size. Also, gzip runs on a computer with a truly mind-boggling amount of number-crunching power available compared to a modem, which has a CPU just powerful enough to handle complicated commands like "ATH". With more CPU power, you can achieve better compression ratios.

In the end, modem hardware compression is basically a hack, and mostly a worthless one. There's a reason why everybody who distributes a file for download compresses it first, and it's not because it makes the file look prettier.

Re: Awwww boo hoo

[Master Bait]

Well, why don't you go ahead and write some Free software to accomplish the same thing?

Running Squid with a 256mb ram disk cache is all the speedup we need, and it does so without altering the data being fed from upstream.

Hey weren't you in my class?

[narftrek]

You......
went.....
to the.....
William.....
Shatner....
school of.....
acting.....
didn't you?

compression with PHP

[mboedick]

If you have a recent version of PHP, you don't even need mod_gzip. Just put the following lines in your .htaccess file:

php_flag zlib.output_compression on

Does everything on the fly. I once had a shell script that would wget a url with the accept encoding gzip header, and then wget it again without and show the percent savings. Was fairly interesting to see what sites were using compression, and what sites that weren't could have saved in bandwidth by using compression.