Slashdot Mirror

← Back to Stories (view on slashdot.org)

A New Web Image Format

Posted by CmdrTaco on from the just-what-we-need dept.

MrP- writes: "BetaNews is reporting that a company called LizardTech has developed a new image format for the Web called DjVu." Apparently, it differentiates between forground and background components of an image, and compresses each appropriately. Good idea, but I'm skeptical of improvements (especially because they say it's "20 times faster then gifs" -- which measure compression in terms of speed? And they also say it compresses faster then pdf, but pdf isn't really an image format). No Linux support. And I don't see any source code on the format, so don't expect it to get a lot of support on any major Web sites, regardless of the compression.

10 of 154 comments (clear)

  1. Why are people so stupid? by GoRK · · Score: 4

    DjVu is almost three years old. It was developed by AT&T not LizardTech. LizardTech just bought it about 8 months ago. It is not, was not, and will never be designed to replace PDF, GIF, JPEG, or PNG. I have been using DjVu as the core of a web based document management system for over a year and a half. It is absolutely bar none the best, fastest, and most cross platform way to go from paper->web out there.

    Look at the ways to get a scanned document on the web:

    1) GIF, PNG, JPEG: Large filesize or bad rendering. If I need to send a 300dpi page to a web browser, the browser isn't going to let a user pan and zoom on it and it certainly won't print it correctly. JPEG is the only one of the three formats that actually has a place to store the document DPI regardless.

    2) PDF: Creating a pdf from a scanned image means either encapsulating a lossy or losless image in a file or doing OCR and risking unreliable information.

    DjVu regularly achieves compression ratios of 1200:1 or more at very very acceptible quality. There is a IW44 fractal compressed background layer and a loslessly compressed foreground layer. The information is progressive also. As the file downloads the foreground shows, then the background, then the color information loads. Example documents on the DjVu website have shown entire 300 dpi full color sharper image catalogs compressed to fit on a floppy disk.

    Btw not only are djvu plugins available for windows, macintosh, linux, and solaris. Let's not forget HP-UX and IRIX. How's that for covering the bases? If youre not supported, you can write your own for your particular flavor of UNIX.

    Geez get it straight.

    ~GoRK

  2. They got it from AT&T by dne · · Score: 4

    See DjVu "non-commercial" site.

    --Daniel

    1. Re:They got it from AT&T by SEWilco · · Score: 4
      And we discussed it here on Slashdot:
  3. Re:Some facts off the top of my head... by Anonymous Coward · · Score: 5
    Gary got is mostly right.

    I am one of the four persons who created DjVu in the first place. The events took place in AT&T-Labs Research between 1997 and 1999.

    1. There is Linux support. Just go to the download page and select the Linux platform. Most of DjVu was first coded under Linux.
    2. After the Lizardtech deal, we set up a "non commercial site" named DjVuZone. It contains general information, benchmarks. links, a searchable digital library, etc.
    3. There is source code. Lizardtech recently had the good idea to relicense version 2 of the DjVu reference library under the GPL. We have the corresponding online documentation on DjVuZone. We are just waiting for the release of version 3 to redo that part of the site.
    4. DjVu combines several new technologies including new approaches to arithmetic coding (Z'-coder), new compression methods for textual images (Soft pattern matching, JB2), new wavelet method (IW44), and new ways of combining them together. The current implementation is geared toward compressing scanned document images in 24 bit colors around 300 dpi (raw size is 25MB) and typically packs them into 50-60KB. Neither TIFF, nor JPEG, nor JPEG-2000 nor Fax-G4 can do that. None of these technologies will let you realistically view such documents over the web. DjVu can.

    Hope this helps :-).

    - Leon Bottou, AT&T-Labs Research.

  4. -1 redundant by buttfucker2000 · · Score: 4

    The article has it about right. Png is vastly superior - excellent lossless compression, sometimes better than lossy methods (plus technical features like a full alpha channel), and, most importantly for its dominance over gif, it is unencumbered by patents or closed source algorithms.

    Speed of compression is not a factor in compression - otherwise we would use bmps or xpms, which have zero compression time - because they're uncompressed. Size matters. Speed doesn't.

    I really can't see much market, and very little application for this compression. On-the-fly compression of images for web download would be redundant, since a png would be smaller than this format, so the speedy on-the-fly compression of uncompressed images is pointless.

    And in any case, modern PCs are more than powerful enough to almost transparently display well compressed images, so a simpler format is about 10 years out of time.

    If it was open source, it could perhaps have a market in replacing things like xpms, which are used in games for processing speed, but even it was, the benefit would be marginal, since hard disk space is, relative to image size, almost infinite, so compressing them slightly wouldn't make much difference - and for download those images would be gzipped anyway.

    --
    Free Anne Tomlinson!!
  5. Re:Actually quite an old product by Suydam · · Score: 4
    On that note, I've been using DjVu's open-source encoder for several years to encode text documents. It's compression ratios are incredible and the plugin is also free and easy to install.

    The big problem I have with this article is that DJVu isn't a "new image format". It doesn't even display things inline (like GIF, PNG and JPEG). It is however an excellent alternative to PDF if size of file is your main concern.

    The extensive references to "speed" when compared to GIFs and PDFs could be one of two things. They could be talking about Download speed (my personal experiences show DjVu files to be about 10 times smaller than GIFs and even more when compared to PDFs. Or, they might be speaking of encoding speed which DjVu seems to excel in

    Here is a problem however: the command line encoder used to be free for non-commercial use. I was using DjVu for encoding swim-team documents for a small non-scholarship collegiate swim team. Certainly this counts as non-commercial. HOwever, the new version from Lizard Tech would cost me $2,000 USD to run. That is absurd by comparison. So I'm abandoning DjVu since I can no longer afford the encoder.

    Incidentally, if you want to see how it worked for me, I used it on nearly every swim meet results page for a few years. Here is an example, just click on the links next to the word "Splits" in each event: http://www.k-swimming.org/cgi-bin/swimming/results /meet_view.pl?8

    --


    Werd.
  6. There is Linux support by snookums · · Score: 4

    They have a browser plugin for Linux/x86/glibc2 available for download here

    Yes, I know that link is broken because /. put a space in it -- you'll have to get rid of it yourself
    --
    Be careful. People in masks cannot be trusted.
  7. Actually quite an old product by Nailer · · Score: 5

    DjVu has been around for 2 years, and isn't anything new. In fact, it wasn't actually designed by Lizardtech - it was developed as an Open Source technology in the Olivetti and Oracle Researtch labs in Camridge, UK, and was sold when US telco AT&T purcahsed the labs.

    Hence the Open Source products generally only seem to be there to satisfy existing licensing requirements from prior to Lizardtech's purchase. It's doubtful Lizardtech tend to encorage that aspect of the technology, and they're only promoting the closed source stuff.

    However, the compression is indeed very real and the cross platform nature makes it quite useful for archiving stuff that won't be modified frequently in the future - remeber, that text ain't vectorized, it's just another layered image, AFAICT.

    1. Re:Actually quite an old product by dkh2 · · Score: 5
      True. If you want to see DjVu in action, go get the plugin at djvu.com and visit one of my projects here at CWRU. http://www.cwru.edu/UL/DigiLib/Hours/homepage.html

      Picture this: Start with a 15th century Flemish "Book of Hours", hand illuminated on vellum (goat skin). Scan it at 600dpi 24bit for archival purposes. Reduce your tiffs to 300dpi and you still have 1.06 GB of image data (not very downloadable). Using the DjVu compressor we achieved 205:1 compression so the final product totals 5.44MB. By separating the pages so they only download when called for the initial download is a mere 45.06KB (including all of the HTML and other images on the page) with an average download of subsequent pages only 21.34KB.

      DjVu was developed by AT&T Research. It was then purchased by LizardTech last year.

      Code commentary is like sex.
      If it's good, it's VERY good.

      --
      My office has been taken over by iPod people.
  8. Some facts off the top of my head... by gary.flake · · Score: 4
    As far as I can tell CT's post and the article have anumber of things wrong. I've known some of the people involved with DjVu for a couple of years, so let me list a couple of facts in no particular order:
    1. DjVu was originally developed at AT&T by a group that has traditionally worked in machine learning. LizardTech purchased the technology from AT&T.
    2. This format is specialized for scanned documents.
    3. The technology is very different from just about everything else because it seperates background and foreground planes. The background is compressed with wavelets, and the foreground probably uses a form of clustering on characters shapes (in a typeface and language independent manner). As a result of the latter, you get a form of OCR almost for free. You can also do text search.
    4. Everything can be viewed at 300dpi directly in your browser and in realtime (you normally only view at 100dpi but you can zoom in).
    5. The linux viewer plugin and compressor has been available for years.

    The main attraction of DjVu is that your scanned documents are tiny (typically less than 50KB) which makes it feasible for putting them on the web. Just about every other format results in files too big for easy distribution on the web. Interestingly, you can convert a *.ps.gz file into a DjVu file, and see a dramatic improvement in file size while preserving almost all of the detail. I am not talking about simple pages here, by very complex ones with a mixture or real images / artwork, and text.

    Apologies for any mistakes, but I think that I got most of it right.

    -- GWF

    --
    The Computational Beauty of Nature