Slashdot Mirror

← Back to Stories (view on slashdot.org)

Gzip on a PCI card

Posted by michael on from the what-will-they-think-of-next dept.

steve writes "The German tech news site heise.de is reporting here (in German, of course) about a PCI card developed by the Universiy of Wuppertal and Vigos AG being shown at CeBIT, which does Gzip compression in hardware, thus freeing the CPU to do other tasks. The PCI card can compress 32MB/sec, which is more than enough to compress a 100Mbit LAN in realtime. A future version will do 64MB/sec. The article mentions that this will be of particular interest for web servers. The card should be on sale by the end of the year."

6 of 141 comments (clear)

  1. Useful for netbackups too by walt-sjc · · Score: 5, Insightful

    Seems this would be a great help to those doing backups over a LAN. Shouldn't take too much to alter a version of tar , rsync, etc. to use this card.

  2. bandwidth saving by buro9 · · Score: 5, Insightful

    the key to using gzip is really not to compress at too high a ratio... a low rate of compression offers a pretty sizeable saving in bandwidth for an acceptable CPU usage... once you edge up to the higher compression levels then you pay for it in the CPU and your app slows.

    i love the idea of a hardware based gzip... but i'd start by educating the software users on the cost vs benefit ratio of their existing configuration... i always seem to find that those who don't know what they're doing are the ones that have it set to maximum compression

  3. How cute but useless. by _Eric · · Score: 5, Interesting

    The general trend in the industry goes to non-intelligent interconnections (Gigabit card used to have a processor (Alteon), they don't anymore (see latest intels)). I2O never took off because you don't really need to relieve a computer from computation when your computation power is pletoric.

    On a Xeon 2.8GHz, I just got 71 MB/s for gzip.

    What's the use for such hardware then?

    Plus it will eat the PCI bus because data has to go out of memory to processing card, back to memory, then to network card. You triple the PCI bus bandwidth. (Not true if the compression is embedded in the network card).

  4. Reconfigurable by KingPrad · · Score: 5, Interesting
    This is cool - dedicated chips can process monstrous amounts of data and much faster than a general purpose CPU. So it's a good idea to let this card do the heavy lifting of compression. Of course the use extends to many types of data analysis: encryption, scientific number crunching, graphics compression.

    The best idea would be to make the chip an FPGA not a specially-designed processor. Then you could load in different chip designs for whatever was currently needed. Need to do RSA encryption? The board reconfigures the FPGA for it. Same goes for Divx compression, gzip, SETI@Home, etc. FPGAs take a few milliseconds to reconfigure but when they operate as a dedicated signal processor they can leave a general purpose processor in the dust - leaving the main CPU to run the other apps, the desktop, etc.

    Check out the IEEE archives and journals, searching for "adaptive computing" or "reconfigurable computing".

    KingPrad

    --
    Stop the Slashdot Effect! Don't read the articles!
  5. Cool by arvindn · · Score: 5, Informative
    gzip was designed with such considerations in mind. Throughput of the algorithm took precedence over compression level. Good to see their farsightedness paying off. And the algorithm is pretty simple so that it can be implemented in hardware directly.

    Another thing about gzip is that it is assymmetric: decompression is much faster than compression. Again this is a nice feature, because most files will be decompressed many times but compressed only once. Thus for instance, all man pages are stored in gzipped form and decompressed on demand.

    But I can't see the point of implementing it in a PCI card. Wouldn't it be better to integrate it with either the processor or the network interface?

  6. Re:A bzip2 version would be nice ... by arvindn · · Score: 5, Informative
    No, bzip2 is something that won't work for applications like serving web pages.

    gzip works with streams, producing input as it gets output. OTOH bzip2 treats the input as blocks. Thus it needs to get a whole block before it produces any output. Similarly the client needs to get a whole block of data before it can even start rendering the page. The man page of bzip2 says that the default block size is 900,000 (!) bytes. So while using bzip2 may improve bandwidth it will result in large latency.