Slashdot Mirror

Steven Levy's Crypto is a brief history of the men involved in developing modern cryptography. If you've read Applied Cryptography or another work with a mathematical emphasis on crypto, you've heard their names -- Diffie, Hellman, Chaum, Rivest, Shamir, Adleman, Zimmermann, and so forth. But the other books on cryptography typically neglect the human side in favor of the math. Crypto aims to fill that hole.

Crypto author Steven Levy pages 356 publisher Viking/Penguin rating 9/10 reviewer Michael Sims, drfalken, topeka ISBN 0-670-85950-8 summary A history of the people involved in developing modern cryptography

Several people were interested in reviewing this book. We try to be accomodating, so this is a mega-review by myself and slashdot readers drfalken and topeka. I'll try to be brief.

Michael's review:

I didn't expect to like Crypto. I was frankly put off by the subtitle on the front cover: "How the Code Rebels Beat the Government -- Saving Privacy in the Digital Age." Every time I send an unencrypted email (because none of my correspondents use encryption, because it isn't built-in) or think about the law (CALEA) which requires my ISP and telephone company to accomodate the government in wire-tapping my communications, I realize that this just isn't true. While the cryptographers thought they were winning battles, the government has so far been winning the war. From the sub-title, I expected the book to be a rah-rah cheerleading history of these noble crypto-knights wielding their ciphersabers with gleeful abandon against the fascist, corrupt, and evil Big Brother.

It turns out to be a much better book than I had expected. The author has collected most of his information through personal interviews, and it ends up being a very readable and very personal account of the past 30 years of cryptographic research and commercial development -- both in the public sphere, and, to some extent, in U.S. and British intelligence agencies. The author treats his subjects fairly - the government is not demonized as I expected, and the cryptographers are not idolized (much). There is essentially no math in this book, beyond the bare minimum necessary to understand the main concepts of cryptography. Together with, say, The Codebreakers for early history and Applied Cryptography for the math, it would make a comprehensive and thorough look at the history and science of cryptography.

drfalken's review: The ubiquity of encryption technology employed by everything from bank machines to e-tailers is now taken for granted. Most people fail to realize, though, the profound impact that this component of the digital world has had on the Information Age. Illumination of this point is the formidable task of Crypto.

The renowned author of Hackers and Insanely Great remains true to form, transforming an obscure, dry and complex subject into an addictive page-turning thriller. He takes us from the hippie culture of academic math research in the 70s, through the dark underworld of government intelligence, into the development of the modern information age. Each step emphasizes the central conflict of the story: American national security vs. the right to individual privacy.

While this conflict has largely been resolved, the story contains important lessons that can be applied to the contemporary struggles over technologies like DeCSS and peer-to-peer media 'sharing.' Levy doesn't make any such connections in the book, but it is impossible to read Crypto without seeing how history is repeating itself in these other areas. This makes Crypto and important book to read. Everyone from the RIAA to 2600 subscribers can learn a lot from this well organized retelling of the past 30 years of crypto history. There's a certain futility involved in trying to put the genie of progress back in a bottle. There's also a case to be made for the management of progress so that it is used with the greatest benefit and smallest detriment to all. Perhaps the most remarkable revelation in the book is how the adversarial nature of 'the geeks' vs. 'the spooks' allowed for the maturation of a sensitive technology in a safe and thoughtful manner.

Anyone who has read Wired or Newsweek over the past 5 years will have read excerpts from Crypto. Levy spent a long time researching this book, which makes sense considering the story he is telling is one that was developing during his period of research. Many of the events he recounts are ones he covered as a journalist at the time that they happened. Some time spent in the Wired archives shows the extent to which he has been one of the journalists closest to the crypto revolution since the release of PGP and the popularization of the Internet.

The book begins with the story of Whit Diffie and his wild ambition to simply learn more about the black art of electronic cryptography. In the early 70s the government monopoly on information relating to serious crypto was nearly complete. Coming from the mindset of the Open Source community, Levy's tale of the early crypto research climate describes a cathedral that makes Microsoft look like the Debian project. The resulting story, therefore, highlights the magnificence of the public key breakthrough, the boldness of the RSA discovery and the daring of Paul Zimmermann's PGP.

If you're looking for a history of Cryptography, get The Code Book by Simon Singh, or Codebreakers by David Kahn instead of this book. Crypto is a contained story dealing exclusively with the American Cryptographic Experience from Diffie-Hellman, through RSA, and PGP. It is effectively a collection of short, intertwined biographies of the saviors of privacy, from Adleman to Zimmermann. This is not to say that Levy ignores the math; on the contrary, his explanation of the magnitude of the public key concept hits home even harder than the impressive work by Simon Singh.

Especially in light of recent Slashdot stories, Crypto is highly recommended, for novices and Cypherpunks alike. It's a coming of age story for American technology, and a great addition to the bookshelf of modern American history.

topeka's review:

The first time I heard the term "elegant" applied to a technical problem was a bit of a revelation for me. Until then, elegance, to me, was a visual quality that could only be achieved by painters and poets. When I began to see the elegance in solutions to technical and mathematical problems, I was hooked into a world of intellectual curiosity. Cryptography immediately filled the mold of a highly complex and technical problem with a beautiful and elegant solution when it was first explained to me several years ago. The idea clicked again when I read Raymond's The Cathedral and the Bazaar and equated that elegance to "scratching a particular itch". This intellectual curiosity seems to drive the open source community.

However, in 1967, when James Ellis (of the secret British agency, GCHQ) first came up with the idea of public key cryptography, his theory was buried. Until then, solutions to cryptographic problems were a dirty process. If it was easy to create a cipher, than it was just as easy to break it. As such, Ellis's breakthrough was simply too pretty to be trusted and as a result, it lay locked away until 1997. Steven Levy's new book, Crypto is the story of the individuals who transformed cryptography from a dirty art, which only the most elite governments dabbled in, to an elegant mathematical solution available to the public in hundreds of different forms. It was all done by a community of individuals who preached openness and sought out clean solutions to tough, technical problems.

Levy starts out his story in the same place as he started with an earlier famous work, at the Massachusetts Institute of Technology. He narrates the story of Whitfield Diffie, the co-creator of public key cryptography. Starting in 1969 as Diffie sought shelter from the Vietnam war working for a defense contractor, Levy discusses Diffie's transformation from examining ideas about cryptography as merely a hobby, to an all out obsession. Diffie is transformed from a man thinking about cryptography on the weekends to a man criss-crossing the country in one run-down Datsun after another, searching for any and every piece of information about cryptography. Diffie would not broach the wall of cryptography until he was pointed to another researcher in California, who seemed to be investigating the same concepts. Levy chronicles the fateful partnership that occurred with Marty Hellman and the subsequent invention of public key cryptography, at least its theory.

At this time, there were few works published on the subject of cryptography. In fact, only government agents and a few privileged defense contractors were able to expend meaningful resources on crypto research. It seems that while Levy's work is a story of the people who waged a war to bring crypto to the public, it is also the story of that wars' enemy, the National Security Agency. The cryptography bureaucracy, gaining most of its resources during the Second World War, had built quite a palace around anything that involved codes. In the years to come, the NSA would fiercely defend its position of strength. From its early attempts to classify David Kahn's famous work, The Codebreakers, to its involvement in the creation of the Digital Encryption Standard and its invention of the Clipper Chip. As Crypto defines it, the spooks were able to keep their lock on cryptography by invoking a mentality of "if only you knew what I know..." in classified briefings to politicians and contract negotiations with defense contractors like IBM. What the NSA never expected, was for anyone to try and find out what it was that they knew. With the publishing of the Diffie-Hellman paper, "New Directions in Cryptography," one of the NSA's most viable opponents would begin their work where Diffie and Hellman's theories left off, implementation.

Ron Rivest, Adi Shamir and Leonard Adleman, through a four-month period of intense brainstorming, would eventually implement and patent the Diffie-Hellman concept of public key cryptography while working as faculty at MIT. As Levy chronicles it, the algorithm, which would become popularly known as RSA, was named for the order in which each mathematician gave to the project. Rivest, who spearheaded the search for the implementation was listed first and Adelman, who merely poked holes in Rivest and Shamir's proposals, had to be convinced that he had even contributed enough to the project to be listed on the paper. Until this point, the description of cryptographic algorithms in scientific texts had always been done using letters of the alphabet to depict members in a cryptographic exchange. The creators of RSA introduced the now famous cryptographic characters, Alice, Bob and the unruly Eve, to describe their new breed of algorithms. Levy is able to highlight the mentality of the three mathematicians, some of which at first, thought the problem was nothing more than a clever puzzle and too grounded in the real world to be successfully dealt with by mathematicians. He shows their transformation to the church of cryptography, as the elegance of the new algorithms would prove as beautiful as the theorems of Gauss and Euclid.

The story continues with RSA Data Security, the vehicle Rivest would use to commercialize his algorithm. To talk about RSA Data Security is to talk about patent use. Both the Diffie-Hellman algorithm, as well as RSA, were actually patented by Stanford University and MIT, respectively. When the patents were granted, those Universities then had the option to either free the patents or restrict them. As history has painfully shown, they did not choose to free them. RSA Data security was built on this decision -- an MIT patent. It was sometimes difficult to read this section of the book with the same exuberance that Levy writes about it. Nonetheless, it is a reminder of the state of our intellectual property laws today in the United States.

Levy's narration eventually leaves the story of RSA to tell that of Phil Zimmerman, someone who could rightly be called a crypto-anarchist. Once again we are treated to an in depth discussion of the motivation that created Pretty Good Privacy. Levy contrasts the use of legal patents by RSA Data Security to bring encryption to the masses, to the complete ignorance of them by Zimmerman in his creation of PGP to achieve the same goal.

Finally, in my favorite section of the book, Levy discusses the controversy that surrounded a device known as the Clipper Chip. It was originally invented by the NSA as a complete key-escrow system, named the Capstone Chip. Later, as AT&T attempted to market the first encrypted telephone device, the Capstone chip became the Clipper Chip as the FBI and other Executive branch officers rushed to implement a brain-dead subset of the original system before the AT&T device made it to market. An entirely amusing fiasco, Levy lays the entire story out from beginning to end.

Lastly, includes an epilogue telling the story of the British agents at GHCQ, who beat Whitfield-Diffie and RSA -- a story that the GCHQ refused to let surface until the mid 1990s.

Levy tells a story about people. If you are looking for a technical discussion of the different aspects of cryptography then you would be better off with Schneier's Applied Cryptography or Singh's The Code Book. However, to understand the freedom that cryptographic technologies bring us, we must understand the history that it stands on. This is what Levy provides. A comprehensive history of the events that took cryptography out of the hands of the NSA and into the hands of political dissidents, CEOs, Nazis, you and me (not to mention mozilla, pgp, ssh, and gpg).

You can purchase Crypto at ThinkGeek.

Thanks to darrn chamberlain for an excellent review of the Lincoln Stein and Doug MacEachern book Writing Apache Modules with Perl and C. This is an excellent book for those considering working with Apache and mod_perl, and helpful for C programmers. Click below for more details. Writing Apache Modules with Perl and C author Li pages 724 publisher O'Reilly, ISBN: 156592567X rating 9.5/10 reviewer darren chamberlain ISBN summary Absolutely essential for anyone who is considering using Apache and mod_perl. C programmers may need more. The Scenario

If you're like me, your first introduction to Perl ^[?] was in the form of CGI ^[?] scripts. A few years ago, I inherited a few dozen ancient CGI scripts (Perl and otherwise) that required Immediate Attention. CGI led to Perl, and to Apache ^[?] ; Perl and Apache led, naturally enough, to mod_perl ^[?] , once I started hitting the performance bottlenecks inherenent in CGI programming. After researching mod_perl, building a mod_perl-enalbed Apache, and reading all the available online documentation, I got it up and running--and I was suitably impressed.

So, when O'Reilly ^[?] announced a book devoted to programming Apache with Perl, I was extremely excited. The book starts with an introduction and history of web programming, introduces CGI and other types of web programming (server API ^[?] 's, such as ISAPI and NSAPI; embedded processors, such as mod_perl, mod_dtcl, and mod_pyapache; FastCGI; Java ^[?] servlets ^[?] ; ActiveX ^[?] ; and client-side scripting languages, such as VBScript ^[?] and JavaScript ^[?] ), and then describes the Apache module architecture, using some simple examples ("Hello, World" in Perl and in C). Then it gets good, covering dynamically generated content; the hobgoblin of HTTP, state; and all the other stuff that gives CGI programmer nightmares (like authentication and authorization).

What's Bad?

Although the title reads '... with Perl and C', the emphasis is very obviously on Perl. The C API reference chapters (chapters 10 and 11, pages 505 through 631) are very thorough, but almost all the examples are in Perl only. In fact, the authors go so far as to recommend that almost all Apache modules be written in Perl, and not C, except for very small modules or modules that need that extra speed boost or small memory footprint of being compiled into the server (page 13: "Anything you can do with the C API you can do with mod_perl with less fuss and bother."). Their reasoning is sound: mod_perl modules and scripts require a server restart at most, and often not even that, while for C modules, Apache itself must be recompiled; but I was expecting more in this area, perhaps a larger section on using DSO. After the book was published, however, several of the Perl-only examples were ported to the C API, and are available for download.

A few of these examples have already been published, and in these cases the book is mostly redundant. Notably, the Apache::NavBar module (which Lincoln uses on the server in his lab) and the Apache::AdBlocker module (chapters 4 and 7), appeared in The Perl Journal last year (issues 12 and 11). This is not that big a deal, since both of these modules are incredibly useful and probably deserve to be published in a few more places, but two brand new modules would have been most welcome, especially since the book's target audience probably also reads The Perl Journal.

What's Good?

There's a lot to like here. Since I'm a Perl programmer by trade and disposition, I personally liked the fact that 99.9% of the examples were written in Perl. With only a few exceptions, the modules could be copied into the right locations and run immediately; the exceptions were the modules that made use of either other programs (Chapter 5's Hangman program which uses a relational database to store state information) or specialized Apache features (Chapter 7's Apache::AdBlocker module, which requires proxy functionality).

Much of the text and all of the source code is available on the web at www.modperl.com. Chapters 6, 7, 8, and 9 can be found on the web site for the book, as can all the Perl modules and some of the examples in functional form (Apache::Magic and hangman).

Chapter 9 is the key chapter, and the heart of the book. It describes in great detail all the Apache:: modules. If you use mod_perl at all, download and print this chapter. Memorize it. Use your favorite indexing script to make it searchable. Everything you need to know about mod_perl is here in this chapter.

The appendices are also excellent, although, because it is an Apache book, I would have figured that several of the sections would be regular chapters, and not relegated to the end. The appendices are divided pretty evenly between concentrating on Perl and on C, unlike most of the rest of the book.

So What's In It For Me?

Fortunately for people like me, there is a lot of information about mod_perl on the web; The Perl Journal has had several articles on it, WebMonkey has had an article or two, and so on. There is a comprehensive mod_perl developer's guide on the offical Apache/Perl site. Lincoln Stein uses it a lot on his site and in his software. And, of course, we have the man pages and perldocs. So why do we need a book?

A few reasons. First and foremost, few of those sources go into the kind of detail that this book does, while still being approachable. Second, the book focuses on Apache, programming Apache, and (to a lesser extent) programming applications on the web; Perl and C are the means here, not the end. The in-depth technical discussions are about Apache: how it translates URI's to filenames, how it handles subrequests and internal redirects, how it maps files to MIME types. It then presents techniques for usurping these functions, customizing each phase of the reponse process, and explains when and why you would want to do this, instead of letting Apache do it's own thing. Creating checksums on the fly, compressing and decompressing data, creating extremely flexible HTML preprocessors, and modifying outgoing and incoming headers are some just some of the given examples.

The reference chapters are probably the single most valuable thing about the book. If you are a Perl programmer on a budget, you can download chapter 9 from the web site, but the C programmers out there have to buy the book to get the C API refernce. The C reference is 2 chapters (126 pages) long, and covers all the functions in precise detail.

For those among you who are using Microsoft operating systems, the book pays special attention to building, installing, and configuring mod_perl and Apache on Win32 systems, where it is different from Unix and Unix-like systems. Most of the actual modules are very similar (except for the obvious ones, such as scripts that call sendmail and the scripts that access MySQL), but the installation and building of mod_perl (or ApacheModulePerl.dll) are very different. The process is described in enough detail to make it possible, without boring those readers to whom it is irrelevant.

Conclusion

Programming Apache/mod_perl without this book is like writing Perl without the camel book. It can be done, but it is much easier and more enjoyable with the book. The writing is clear, informative, straight-forward, and, at times, amusing. The authors are the definitive sources for information on mod_perl and CGI programming, and this is reflected in every aspect of the book. While not as definitive for C programmers, it is still the best Apache API reference out there, other than the actual source code itself.

Purchase this book at Amazon.

Errata Table of Contents

1. Server-Side Programming with Apache
2. A First Module
3. The Apache Module Architecture and API
4. Content Handlers
5. Maintaining State
6. Authentication and Authorization
7. Other Request Phases
8. Customizing the Apache Configuration Process
9. Perl API Reference Guide
10. C API Reference Guide, Part I
11. API Reference Guide, Part II

1. Standard Noncore Modules
2. Building and Installing mod_perl
3. Building Multifile C API Modules
4. Apache:: Modules Available on CPAN
5. Third-Party C Modules
6. HTML::Embperl--Embedding Perl Code in HTML

What do Microsoft, AOL, IBM, MCI Worldcom, Bell Canada, British Telecommunications (BT), Bertelsmann, Demon Internet, Cable and Wireless, Deutsche Telecom, the Japanese Electronic Network Consortium, EuroISPA, and UUNet have in common with the United Kingdom, Germany, the European Union, and Australia? They're all working together on a plan to censor the Internet.

Hundreds of people from around the world are coming together in Munich for a three-day conference, September 9-11. They represent the largest internet corporations and first-world countries. They've been working on this for years. They have millions of dollars. They're very, very serious. And someone forgot to tell them that information wants to be free.

What's going on?

Labels are the big thing. Labels are everywhere. Television has labels, after Congress threatened to not renew station broadcast licenses if the networks didn't comply. Video games have labels, after Congress threatened the gaming industry. Music has labels, after Congress and Tipper Gore (Al's wife) threatened the recording industry. Anyone remember the 80s, when musicians and fans both seethed at the very idea of labels slapped on our music by some politician? Now even MP3.com has a parental advisory icon.

And of course, movies have labels, the motion picture industry being the most dangerous threat to America's youth next to the internet. Hollywood labors under hundreds of censorship laws.

Now Senator Lieberman wants to rate every audio-visual product produced in the U.S. with a violence labeling system. (Lieberman was primarily responsible for the video game ratings and television ratings as well.)

Proponents of these censorship systems sometimes like to call them "voluntary". They're as voluntary as death and taxes. Or as voluntary as not being able to sell your product at all - that's what Lieberman's bill would dictate, if you don't comply. Salon said it well:

"The point has always been to change what actually gets broadcast through the flexing of government muscle. In simpler times, this was known as censorship."

Labels and censorship go hand in hand. The American Library Association speaks plainly: "Labeling is an attempt to prejudice attitudes and as such, it is a censor's tool." Some groups do stand up for what's right. You'll notice you don't see parental advisories on library books. Yet.

Think of how it works in practice: items with labels are stigmatized, attacked by Congress and pressure groups, and eventually - through law or simple bullying - they aren't available anymore. Think of the NC-17 label. All it's supposed to indicate is fare fit for adults - and since adults are 80% of the population, there ought to be plenty of movies made for them. But since most theaters (over 90%) won't run NC-17 movies, and most newspapers won't carry ads for them, any NC-17 movie is doomed to be a failure. And thus the only movies that make it to the theater are those deemed fit for children. Movies bearing that label were easy to attack - just take the most horrible movie you've ever seen (Debbie Does Dallas? The Texas Chainsaw Massacre? Stargate?) and whip up a public frenzy, then say, "We can get rid of this filth if only you'll stop showing NC-17 movies, Mr. Theater Owner." The pressure was applied at different steps in the distribution process - at the movie theater chains and newspapers, rather than at the consumer's end - but the result is the same: you can't see it.

Or you can't see it the way it was intended. Stanley Kubrick was known first for his work, and second for the exacting craft with which he set up every single shot. If even Kubrick's famous final-cut contract couldn't keep the MPAA vultures from digitally painting over his sex scenes, how is any director safe?

But we digress. We were talking about labels, and Internet censorship. These things intersect in a technology called PICS.

PICS stands for Platform for an Internet Censorship System - well, close enough. It's a specification for attaching labels to internet content - Web pages, Usenet posts, chatroom messages, emails... anything. In theory, you could rate anything on any scale you chose - journalist Simson Garfinkel made a tongue-in-cheek PICS rating system to rate pages based on the amount of Simson they contain.

But that is theory. In the real world, you could rate music or video games on the basis of Simson too, but nobody does - because life is short. Just like all the other labeling systems, it turns out that the only Internet labeling systems that anyone cares about are pejorative labels - rating pages for sex, or foul language, or heresy, or violence. Why? Because these are what the censors want to get rid of.

The people getting together in Munich are doing so for the purpose of developing a single, uniform, international rating system to be applied to all Internet content worldwide. It's not a voluntary system - several countries have already declared their intent to make it mandatory, and Jim Miller of W3C (and co-creator of PICS) put it nicely when he said -

"It's going to happen and the publishers are going to resist it as long as they can, but they'll have to realise that they must rate their content or face prosecution."

Who's a publisher? We are. You are, if you post a reply to this thread. If the system gets set up as scheduled, you'll be forced to add a rating to every post you make, every email you send, every webpage you publish - or face prosecution. After all, you're protecting the children.

Or more precisely, the adults. Australia wants to ban the sex categories from its entire population - Germany wants to ban the hate speech categories. Just like at the movies, it's easier if you attack higher up in the distribution chain.

Rather than making it illegal to download Mein Kampf or purchase it from Amazon.com, it's much easier if you make a law that applies to the telecommunications providers. They're big companies. The bigger they are, the less likely they are to buck the laws - and since there aren't many of them, they're easy to monitor for compliance. Civil disobedience isn't in their vocabulary: give them a law, and they'll just implement it. Such as censoring out all material with a certain rating at the backbone.

Oh, it's true that it won't be 100% effective. Banned documents will still be smuggled across the electronic borders. But for most people, in most circumstances, it will be plenty effective. If you like your internet unlabeled, it's just about too late.

by Michael Sims and Jamie McCarthy

(More tomorrow on the Munich conference and recent events in the development of the Global Rating System.)

HotWired Washington Bureau Chief Declan McCullagh reveals the sordid truth about how he spent his teenage years in this article published by IntellectualCapital.com. But Declan is not nearly as sympathetic to the current generation of crackers (who will continue to be called "hackers" in the non-geek press no matter what you or I say) as the headline would lead you to believe. Here's a quote: "Oh, I know. I have become a humorless curmudgeon who cannot appreciate hackerdom's stellar exploits when I see them...."

Elizabeth Childs sent us an excellent review of Gecko, along with some notes from the Mozilla Party 2.0. *sigh* Now, that sounds like it would have been fun. The review itself does a good job of talking about why Gecko can be such a big deal for developers. Mmm...standards.

Greg Roelofs, author of PNG: The Definitive Guide, has written a feature on the PNG graphic format. The format has many technical advantages, yet it still isn't gaining acceptance. Personally, I just want a real alpha channel on web pages (well, and anti-aliased fonts, but lets cross one bridge at a time), Anyway, read what Greg has to say on the subject of PNG:

I hadn't intended to write anything up so soon, but lately there's been a lot of FUD and some general cluelessness about the Portable Network Graphics (PNG) image format, so here's an update from somebody who knows a tiny bit about it.

First of all, PNG is certainly not dead, although it obviously has not taken the Widely Webbed World completely by storm (which, in the eyes of the esteemed Mr. Veen, amounts to the same thing). For better or for worse, Netscape Navigator and Microsoft Internet Explorer pretty much define what counts as acceptable Web technology, and they only began supporting PNG natively in the autumn of 1997 (versions 4.04 and 4.0, respectively). As various Slashdotters have noted, neither one really supports PNG well yet, at least with respect to alpha transparency and gamma correction, but that's coming; let me return to that issue in a moment.

PNG has been making steady progress, however, particularly in non-Web applications. It has advanced from being a newsworthy ``extra'' to being an expected, standard component of image applications; in other words, a viewer or editor that ships without PNG support is considered deficient by both consumers and the trade press. That's a moderately subtle point--it doesn't necessarily leap right out at you and scream, PNG has arrived, dammit!, but it's nevertheless quite a significant milestone for any new technology. Mundane can be good. (By the way, I maintain the PNG home site and list known PNG-supporting applications of various persuasions on half a dozen pages; stop by if you're in need of something, and please let me know if I've missed any!)

But that's just one data point. Everybody's favorite technical publisher, O'Reilly and Associates, not only includes PNG chapters in a number of its books (including Web Design in a Nutshell and the soon-to-be-released Programming Web Graphics with Perl & GNU Software ), they also agreed to publish an entire 700-page book completely devoted to the Portable Network Graphics format: PNG: The Definitive Guide . It consists of around 300 pages of main text, 100 pages of program listings (both Unix/X and 32-bit Windows, under a BSDish license, freely downloadable soon), 250 pages of specifications, some fairly cool color figures, and assorted odds and ends; I happen to know because I wrote it. :-) (It just went into production on Monday, so it should hit the shelves in a few months, plug plug. I'll be updating the web page with dates and whatnot as soon as I find out myself.)

In addition, PNG has been published as an informational Internet RFC and a a W3C Recommendation (the very first one), and it's now wending its way through the slowly grinding gears of joint ISO/IEC standardization. That will all help ensure its longevity, but it's also fairly boring to most of you, I'm sure.

So getting back to the Web issue, let me briefly summarize PNG's basic capabilities:

• palette-based support (1, 2, 4, 8-bit), like GIF
• grayscale support (1, 2, 4, 8, 16-bit)
• truecolor support (24, 48-bit), like TIFF or (sort of) JPEG
• binary transparency, like GIF (except including grayscale and RGB modes, not just palette-based)
• alpha transparency (256 or 65536 levels of partial transparency), like TIFF
• alpha-palette transparency (that is, palette has RGBA entries, not just RGB), unlike anything else on the planet
• direct support for gamma correction and color correction
• lossless, unpatented compression
• 2D interlacing, somewhat like progressive JPEG
• no animation (but a closely related format called MNG)

The patent issue is largely history, except to shareware and freeware authors, for whom it's still quite real--Unisys lawyers continue their apparent crusade to kill all low-cost GIF-supporting software. What really matters to Web developers, however, is PNG's support for palette images (no, they don't all have to be really fat, 24-bit monsters); its support for alpha transparency even in palette images; its support for gamma and color correction; and the fact that its compression is lossless (which is why 24-bit PNG images are so fat, especially compared to lossy JPEG). In other words, for the same number of bytes as a binary-transparency GIF image, you can have a lovely alpha-palette PNG image that, thanks to gamma correction, will not look too light on Macs or too dark on PCs. The alpha support means it can be anti-aliased or drop-shadowed to look good against any background, not just a single, flat color. Note that MSIE 4.0 already supports gamma correction, and 5.0 is supposed to do full alpha transparency; we'll find out next month, I guess. Mozilla/Netscape 5.0 will also support both alpha and gamma, or else--I'm the nominal ``owner'' of Mozilla's PNG support, and now that the book is done, I intend to do some serious hacking. (Apologies for the 10-month delay!)

Since PNG pushes the envelope on a lot of image-related stuff--alpha transparency (no other Web formats), RGBA-palette images (no other format, period), gamma and color correction (almost no other formats), and even compression/filtering (it has a bunch of free parameters that one can tweak)--many of the current applications are somewhat behind in supporting some of the features. Be patient; things are steadily improving. I won't point fingers at any underperformers here, but I will note that the GIMP is quite strong in compression and 32-bit RGBA and should have fully working gamma code in the next release after 1.0.2; and Fireworks 1.0 is already partway there with RGBA-palette support and should be completely spiffy by version 2.1 or 3.0. (I didn't quite get my feedback in on time for it to affect the 2.0 release. If only they'd had a Linux version to try...)

I'm hopeful the book will help many of the others--it includes a lot of material aimed at helping programmers to improve their code, but also a lot of stuff to help users avoid problems and make the best of what options they've got. And if I have time while I'm working on Mozilla, I plan to release a free, automatic 32-bit to 8-bit (RGBA to RGBA-palette) converter to handle what seems to be the hardest PNG feature to support. Such conversion literally gives you a factor-of-four reduction in file size with essentially no visible loss (no more than normal RGB-to-palette conversion with nice dithering, anyway).

So...that, in a largish nutshell, is the past, present and future status of PNG. As for new competition, well, let's just say that there have been lots and lots and lots of fantastically improved, incredibly stupendous image formats that have come and gone over the years; ``WI'' is merely the latest, and SVG and JPEG2000/JPEG-LS are right around the corner. Anybody remember Johnson-Grace/AOL's ART format? How about Iterated Systems' fractal format (FIF)? Or some of the quadtree-based ones, or SPIHT, or FlashPix, or JPiG, or CMP, or ePIC, or HARC-C? Heck, even JPEG with arithmetic compression is considerably better than standard DCT-Huffman JPEG, but does anybody actually use it? No. Proprietary standards are simply not tolerated very well on the Web. Even free, open standards like PNG (with free, open-source, non-GPL'd reference libraries) have a huge barrier to climb. It took ``standard'' JPEG four years to catch on; it's taken PNG four years to catch on (yes, it's really been that long, and 2.5 years since the W3C Recommendation); and, barring largish miracles--e.g., Netscape and Microsoft cooperating--it will take any other new image format just as long.

There you have it. There's lots more info on the web site, and there are a couple of mailing lists for folks who really want to get gnarly with PNG. Oh, and please buy the book. ;-)

Ben Mehling writes "Hotwired's WebMonkey/backend section is running a nice series on database connectivity crash course by Richard Dice. Richard decided to use Linux, Apache, mod_perl, and mySQL as his "backend" [ct:Exactly what Slashdot uses]. Most of the stuff is pretty elementry - getting started oriented, but its pretty cool to get some nice exposure for all these products. Overall I am pretty impressed with the WebMonkey/Backend section... I wouldn't mind seeing them do a series on PerlEX (the Win32 mod_perlish counter part). "

Ben Mehling writes "Hotwired's WebMonkey/backend section is running a nice series on database connectivity crash course by Richard Dice. Richard decided to use Linux, Apache, mod_perl, and mySQL as his "backend" [ct:Exactly what Slashdot uses]. Most of the stuff is pretty elementry - getting started oriented, but its pretty cool to get some nice exposure for all these products. Overall I am pretty impressed with the WebMonkey/Backend section... I wouldn't mind seeing them do a series on PerlEX (the Win32 mod_perlish counter part). "

Itamar S.-T. let us know that PalmPC.com has been registered by 3com and is up and running. Next is this articleabout the new P3's sent in by David Cloud. Last Drew Daniels sent in this hotwired article and This ZDNet article.

More good good press for Opera v3 today, with a good column about it's speed and ability to handle the necessary items. Who knows-could this be the dark horse of the web race?