Slashdot Mirror

occidental writes "Sanders Kleinfeld writes: In the past six years, the rise of the ebook has ushered in three successive revolutions that have roiled and reshaped the traditional publishing industry. Revolution #3 isn't really defined by a new piece of hardware, software product, or platform. Instead, it's really marked by a dramatic paradigm change among authors and publishers, who are shifting their toolsets away from legacy word processing and desktop publishing suites, and toward HTML5 and tools built on the Open Web Platform."

An anonymous reader writes "New Tesla owner and Executive DIrector of Cloud Computing at Dell, George Reese, brings the Tesla Model S REST API authentication into question. 'The authentication protocol in the Tesla REST API is flawed. Worse, it's flawed in a way that makes no sense. Tesla ignored most conventions around API authentication and wrote their own. As much as I talk about the downsides to OAuth (a standard for authenticating consumers of REST APIs—Twitter uses it), this scenario is one that screams for its use.' While not likely to compromise the safety of the vehicle, he does go on to say, 'I can target a site that provides value-added services to Tesla owners and force them to use a lot more electricity than is necessary and shorten their battery lives dramatically. I can also honk their horns, flash their lights, and open and close the sunroof. While none of this is catastrophic, it can certainly be surprising and distracting while someone is driving.'"

Lemeowski writes "Cygnus Solutions co-founder Michael Tiemann takes an in-depth look at whether music can truly ever be open source. Leaning on his personal experiences of trying to convince the market that a company that provided commercial support for free software could be successful, Tiemann argues that similar to how 'the future of software was actually waiting for the fuller participation of users ... so, too, is the future of the art of music.' In his essay, Tiemann makes a case for open source music, from licensing for quality recordings to sheet music with notes from the original composer in an easy-to-reuse format, and he offers ways to get involved in making music open source." Apropos open source music, reader rDouglass adds a link to the Open Goldberg Variations project, last mentioned on Slashdot in 2012.

An anonymous reader writes "Curious about the recently purposed NSA cuts, Courtney Nash explores a few myths about systems automation 'In the aftermath of Edward Snowden's revelations about NSA's domestic surveillance activities, the NSA has recently announced that they plan to get rid of 90% of their system administrators via software automation in order to "improve security." So far, I've mostly seen this piece of news reported and commented on straightforwardly. But it simply doesn't add up. Either the NSA has a monumental (yet not necessarily surprising) level of bureaucratic bloat that they could feasibly cut that amount of staff regardless of automation, or they are simply going to be less effective once they've reduced their staff.'"

Michael Ross writes "As a hugely popular scripting language with an 18-year history, PHP has been the topic of countless computer language books. One of the most comprehensive offerings has been Programming PHP, published by O'Reilly Media. The first edition appeared in March 2002, and was written by Rasmus Lerdorf (the original developer of PHP) and Kevin Tatroe. A second edition was released in May 2006, and saw the addition of another co-author, Peter MacIntyre. With the many changes to the language during the past seven years, the book has again been updated, to cover all of the major new features made available in version 5 of PHP." Keep reading for the rest of Michael's review. Programming PHP, 3rd Edition author Kevin Tatroe, Peter MacIntyre and Rasmus Lerdorf pages 540 publisher O'Reilly Media rating 8/10 reviewer Michael Ross ISBN 978-1449392772 summary An extensive tutorial of the PHP web programming language. This third edition was published on 22 February 2013, under the ISBN 978-1449392772, with the same three authors at the helm. At a substantial 540 pages, the information is organized into 17 chapters, each focusing on a particular area of the language and its usage. This material precedes an appendix of almost 130 pages, which serves as a reference for all of the language's built-in functions. In fact, not only could this book suffice as a reference guide, it could also serve as a tutorial, because it is accessible to programmers of all levels, including beginners who have never before worked with PHP. The preface notes that the material assumes only "a working knowledge of HTML." However, the example code seems to also assume that the reader is comfortable with fundamental programming concepts, such as conditionals and loops.

To learn more about the book, prospective readers and buyers may wish to visit the publisher's website, where they will find a description of the book, its table of contents, a free copy of its first chapter, and the example code for ten of the chapters. Speaking of formats, the book is available in print and electronic media. (This review is based upon a copy of the print version kindly provided by the publisher.)

The first three chapters explain the bedrock fundamentals of the language, including its lexical structure, data types, variables, expressions, operators, flow-control statements, code inclusion methods, and functions. All of the information appears to be valid, aside from several technical blemishes: In Example 1-1, most of the lines of code are duplicated. Example 1-5, which supposedly creates a PNG file, does not seem to work. The section on constants (page 21) should have mentioned the core predefined constants and also distinguished those from magic ones. The binary literal 0b01100000 is 96, not 1 (page 23). It is claimed that an object is evaluated as false if it contains no values or functions (page 25), and yet: "class C{} assert( new C );." The closure example code (page 29) fails because it includes a function name and no terminating semicolon. The example code in the middle of page 66 contradicts the claim that an inner function "cannot be called from code parsed after the outer function." The example code starting at the end of that page fails because $a in foo() is undefined. Nonetheless, even experienced PHP programmers could pick up knowledge not encountered before, or at least refresh what was learned years ago and since forgotten due to disuse.

The next two chapters explore in detail further essential components of PHP: strings, regular expressions, and arrays. As with the earlier chapters, readers will encounter example code that does not appear to have been tested. For instance, the print_r() output of an object is missing the class name (page 84). On the same page, print_r() and var_dump() of $GLOBALS do indicate "*RECURSION*," but do not loop infinitely or three times, respectively, as claimed. The $record on page 86 is missing its trailing tab character. For these errors and others, it is not clear whether the authors or the technical reviewers are ultimately responsible. Regardless, readers should find useful the tables summarizing regular expression character classes, anchors, quantifiers, and options. On the other hand, the treatment of conditional expressions is sorely in need of examples. Also, readers will be baffled when told that "The preg_match() function takes the same arguments and gives the same return value as the preg_match() function []" (page 112). Lastly, the callback example code is faulty (pages 131, 133, and 141). The sixth chapter, covering object-oriented programming is well-written, aside from the confusing phrase "including it to a different name" (page 160) and the anti-Nietzschean "this will fatal" (page 161).

With Chapter 7, the book shifts gears from the basic underpinnings of PHP to more applied topics, in this case, web techniques — specifically HTTP, global variables, server information, web form processing, sessions, and more. The narrative is quite clear, except when the reader is told that periods in field names are converted to underscores because periods are illegal in PHP variable names (page 178); the connection is not explained. The next chapter looks at server-side data storage, including the topics of PDO, MySQLi, SQLite, and MongoDB. Confusingly, readers are told that the sample SQL database code is available in a file, but they are not told where to find it (http://examples.oreilly.com/0636920012443/).

Chapters 9 through 11 address PHP's support for three specialized file types: graphics, PDF, and XML. The explanations are excellent, and the authors provide numerous examples. The only obvious flaws are in Example 11-1 (page 269), where the echo statement is missing the "<?" and two of the lines have mismatched single and double quotes.

The remaining half dozen chapters cover critical aspects of PHP development. The chapter on security does not attempt to be exhaustive, but instead explains the most common attack vectors and how to block them. The chapter on application techniques discusses code libraries, templating, output buffering and compression, error handling, and performance tuning. Any programmer intrigued by the idea of replacing clunky VBA code with PHP, should be interested in Chapter 14, which explores the differences in running PHP on Windows vs. other platforms, with a brief look at manipulating the contents of Microsoft Word and Excel files using PHP. RESTful web services and XML-RPC are the topics of the next chapter, whose only apparent blemish is that json_encode() does not add spaces between the array values (page 339). The penultimate chapter addresses multiple environments, manual debugging, and the use of an IDE. The last chapter briefly covers PHP support for dates and times, and thus should have been located much earlier in the book, with the other material on fundamental concepts.

Overall, this book is quite approachable. Throughout, one will find programming style recommendations, However, as with any technical work of this size, there are passages that could be made more complete or clear. Occasionally the authors will mention something explained only later — e.g., "short echo tag" (page 60) — which can be frustrating to anyone new to a language.

The concepts of PHP being taught are extensively illustrated with example code. Some of it is concise enough so as not to distract from the narrative flow, but far too many examples involve much more code than necessary. This at first glance might seem to be an advantage, but it actually makes it more difficult for the reader to see the parts of the code relevant to the topic at hand. Also, the authors underutilize whitespace in the code, instead jamming tokens and parentheses together.

In a book of this size, we should not be astonished to find some errata: "Wordpress" (page xv), "try and" (same page; should read "try to"), "tick function registered when register_tick_function()" (55; should read "with" not "when"), "eXtensible" (59), "super-global" (67; should read "superglobal"), "display_classes() function" (vs. "function displayClasses()"; 164), "$var" (294 and 295; should read "$value"), "functions of blocks" (323; should read "functions or blocks"), "retried" (337; should read "retrieved"), and "a.k.a." (350; should read "e.g.").

In terms of the production of the book, like most other O'Reilly titles, this one is nicely put together, with readable font. But sometimes words are jammed together so much that lines appear to be a single word, e.g., "codeitselfbutplentifulenoughthatyoucanusethecommentstotellwhat'shappening" (page 17). Also, the publisher should avoid splitting the function names as if they were English words, e.g., "addc" and "slashes()" on separate lines (page 91). The index is missing some obvious entries, e.g., "closures." Many code snippets are missing the "Example" numbers and captions. This may be fine if the authors do not reference those snippets, but makes it problematic for anyone else to reference them.

Even though this is arguably one of the most comprehensive PHP books on the market, it does not cover all aspects of the language. On page 1, readers learn that PHP can be utilized in three major ways — server-side scripting, command-line scripting, and client-side GUI applications; but only the first is covered in the book. The appendix consumes over 120 pages, and comprises information easily available online in the PHP manual's function reference. Those pages could instead have been devoted to at least introducing command-line scripting and GUI applications. In fact, there are two major changes the authors could take in bringing this book much closer to perfection: Firstly, retest all of the code and root out any technical snafus. Secondly, replace the lengthy appendix with full coverage of the topics of command-line scripting and client-side GUI applications.

Regardless, Programming PHP is both a tutorial and a reference resource packed with information and example code. Benefiting from the author's deep expertise in the language and its usage, the book is the most promising single source for anyone who wishes to learn this ubiquitous web scripting language.

Michael Ross is a freelance web developer and writer.

You can purchase Programming PHP, 3rd Edition from amazon.com. Slashdot welcomes readers' book reviews (sci-fi included) -- to see your own review here, read the book review guidelines, then visit the submission page.

An anonymous reader writes "Sanders Kleinfeld explains how his experiences with a Makerbot device led him to the decision that 3-D printing hasn't quite arrived as a legitimate, consumer-friendly technology. Quoting: 'Waiting five hours for your Yoda feels like an eternity; you can play approximately sixty rounds of Candy Crush Saga in that same timeframe (although arguably, staring blankly at the MakerBot is equally intellectually stimulating). To make matters worse, I’d estimate MakerBot’s failure rate fell in the range of 25%–33%, which meant that there was around a one-in-three chance that two hours in, your Yoda print would fail, or that it would finish but once it was complete, you’d discover it was warped or otherwise defective. ... The first-generation MakerBot Replicator felt too much like a prototype, as opposed to a proven, refined piece of hardware. I look forward to the day when 3D printers are as cheap, ubiquitous, and easy to use as their 2D inkjet printer counterparts.'"

simonstl writes "The IRS wasn't after just the Tea Party, Progressives, or Medical Marijuana: Open Source Software was a regular on IRS watch lists from 2010 to 2012. Did they think it was a for-profit scam, or did they just not understand the approach?"

blackbearnh writes "Most commencement speeches are long on platitudes and short on practical advice. O'Reilly blogger James Turner has tailored a speech aimed specifically at the current batch of graduating CS majors. Among the advice that the 35-year industry veteran offers are to find a small company for your first job, but not one that is going to burn you out. Also, keep learning new things, but don't fall into the trap of learning the flavor of the day technology. Quoting: 'Being passionate about software is critical to being successful, because the field is a constantly moving target. What will net you $130K today will be done by junior programmers in five years, and unless you're constantly adding new tools to your belt, you’re going to find yourself priced out of the market. ... You are rarely going to get an opportunity to have your current employer pay for you to learn things, so learn them on your own and be in a position to leverage the skills when a new project comes along. But if you have a passion for technology, you'll already be doing it, and enjoying it without needing me to tell you to."

An anonymous reader writes "Seth Ladd has an excellent write-up of Dart: 'When Dart was originally launched, many developers mistook it for some sort of Java clone. In truth, Dart is inspired by a range of languages such as Smalltalk, Strongtalk, Erlang, C#, and JavaScript. Get past the semicolons and curly braces, and you'll see a terse language without ceremony. ... Dart understands that sometimes you just don’t feel like appeasing a ceremonial type checker. Dart’s inclusion of an optional type system means you can use type annotations when you want, or use dynamic when that’s easier. For example, you can explore a new idea without having to first think about type hierarchies. Just experiment and use var for your types. Once the idea is tested and you’re comfortable with the design, you can add type annotations."

An anonymous reader writes "Andy Oram reports on the quality, security, and community driving open source adoption. 'All too often, the main force uniting competitors is the fear of another vendor and the realization that they can never beat a dominant vendor on its own turf. Open source becomes a way of changing the rules out from under the dominant player. OpenStack, for instance, took on VMware in the virtualization space and Amazon.com in the IaaS space. Android attracted phone manufacturers and telephone companies as a reaction to the iPhone.'"

An anonymous reader writes "Simon St. Laurent writes in praise of CSS selectors: 'After years of complaints about Cascading Style Sheets, many stemming from their deliberately declarative nature, it's time to recognize their power. For developers coming from imperative programming styles, it might seem hard to lose the ability to specify more complex logical flow. That loss, though, is discipline leading toward the ability to create vastly more flexible systems, a first step toward the pattern matching model common to functional programming.'"

pfignaux writes with an interesting view on the place of centralized standardization in modern browsers. From the article: "When HTML first appeared, it offered a coherent if limited vocabulary for sharing content on the newly created World Wide Web. Today, after HTML has handed off most of its actual work to other specifications, it's time to stop worrying about this central core and let developers choose their own markup vocabularies and processing." Instead, the author proposes that CSS, Javascript+DOM, the W3C's accessibility framework, and Web Components are sufficient to implement the rendering of smaller, domain-specific markups.

First time accepted submitter occidental writes in about Etsy's push to get more women engineers. "You’ve probably heard of Etsy, the bustling online marketplace for crafters and artists. You probably wouldn’t be surprised to learn that most of its customers are women, both buyers and sellers. Ditto that the Etsy team is a pretty good representation of the Earth’s gender ratio. Yet when Marc Hedlund took the helm of Etsy’s Product Development & Engineering department, 97% of the engineering department were men. Hedlund realized that in his nearly two decades in IT, he’s hired no more than 20 women for engineering positions. This began to bother him. Especially after his daughter was born."

XWWT writes "A few weeks ago Make offered to send us a sample of its Raspberry Pi Starter Kit to see if we would do a review of the product. Samzenpus asked around the engineering team to see if there was someone who would be willing to do an on-camera review of the device. With all of the buzz about Raspberry Pi, I was very excited to get hands-on time with the device so I could more closely examine the platform. At first we wanted to do this piece as a video but quickly realized that a) it would probably be boring to see some blinky lights and push buttons working on a sample project, and b) the amount of audio that would need to be bleeped to cover my frustration with parts of the kit would be annoying. On a personal note, I also wanted to document all of my experience here as I thought it would be beneficial for newcomers to the maker technology and sometimes having someone else’s experience documented can help you avoid pitfalls and mistakes. (Full Disclosure: I am the Director of Engineering for Slashdot Media. We were given a review copy of the Make: Raspberry Pi Starter Kit. We were not paid for this review but had fun doing it.)" Keep reading for the rest of Wes's review. Unpacking the Box:
The box was nicely packaged with lots of little pieces parts in baggies and was well assembled. I immediately pulled out the Pi board and all of the packaged elements to see what was included. It became apparent that the shipping box would be useless to keep all the parts together once I unpacked it and found an old small plastic tool box to keep the parts in for future use and transport.

Included in the box was the 512MB Pi unit, 1A USB charger (underpowered for big projects), Pi Cobbler kit, Pi enclosure, 4GB Class 4 SDHC card, breadboard, a trimmed down version of the Medtronics kit, short HDMI cable, jumper wires (male) and the Getting Started with Raspberry Pi book. They seemed to be packed well as subassemblies so I tried to keep them together as such until later so I wouldn’t lose or mix parts.

The Medtronics kit had LEDs, resistors, capacitors, diodes, pushbuttons, switches, jumpers and some timer chips; all fun toys. Basically it is a collection that anyone doing electronics work would need in order to do a handful of projects. Most of these parts are cheap when bought in bulk, but getting variety collections like this tend to be expensive as you are buying only a couple of parts so it was nice to see them included. I was disappointed that I didn’t see any male-to-female jumpers in the box as these are useful in connecting pins but realized that was the point of the Pi Cobbler Kit.

After I had looked over the board itself, I thought it best to actually try to follow instructions since I was supposed to review the kit. I opened the included Getting Started with Raspberry Pi book and reviewed the first two chapters to get an idea of what was actually on-board the Pi itself and to see how the “Getting Started” would work for a first timer. Typically I find that getting started books from Make try to appear like How-To manuals blended with a lab book and they don’t do well being either. That was certainly the case with this book as I progressed.

The first chapter was really helpful as it laid out what the main components were on the board and what the actual available processing power. The board is an ARM11, 32bit, 700MHz processor. We happened to get the B version so it has 512MB of available RAM. The physical size of the board is a little larger than a stack of credit cards, with all of the components it is about the total size of a mans pocket wallet (about 3”x2”x1/2”). I examined the physical joints on the board and all were machine done (expected) and seemed to be in good order. The first problem I noticed though was that the joints for the HDMI and Audio/Video jacks would not be sufficient to keep them from being broken off the board. Additionally the joints holding the power unit seemed shaky if the unit were plugged in/out too frequently (the book and blogs confirmed that suspicion). The Ethernet port on the board seemed to be in good shape as did the GPIO and Display and Camera Serial Interfaces.

I was pleased to see that there were some status LEDs on-board for simple debugging. Those of us that are used to solving boot problems with status indicators like LEDs or audible tones know that these are important when you just can’t get a board to respond.

I then examined the enclosure case assembly which still had the protective wrapping on it and stunk of cutting fluid. There were no instructions on assembly for that so I set it aside. There seemed to be small parts in that package and I didn’t want to lose them, so I left it sealed.

Setting up Raspbian:
I wanted to validate quickly that there were no problems with the board so I ran through the steps of flashing the SD card with a copy of Raspbian. I actually tried both the dd tool installation under UNIX and the Win32DiskImager to see if there were significant differences in the experience. While the dd process seemed straight-forward the Win32DiskImage was just as easy. I found the documentation here to be the simplest to follow. Some might argue that having a pre-loaded SD card would have been best but I think the point of doing this yourself helps you to better learn the process and get more comfortable with the device.

I then plugged in the HDMI cable to the Pi and dug up a USB mouse and keyboard. Next, I plugged in the USB power supply and SD card. Immediately I made a note to use a powered USB port next time as it would reduce the number of times I would have to torque the onboard USB ports. When I went to plug the HDMI into my monitor I realized that I only had DVI ports and had to scrounge around in my toolbox for a HDMI to DVI converter. (DVI converters are inexpensive and would have been a nice addition to the kit.) I also made a mental note at this point to DX the 1.5m HDMI cable for something longer. I put the board on a non-reactive surface (notebook) so the contacts would not short and then booted the device. I followed all of the default options laid out in the Getting Started book just to make it simple. All-in-all the experience in booting and setting up Raspbian for the first time was satisfying.

Make: Pi Enclosure:
When I first looked at the Pi Enclosure it was pretty easy to see how it was supposed to go together. What I didn’t realize was the amount of swearing it would take to actually get it done. There are only nine parts in the V1 assembly and it should be easy to do, but without instructions it might as well have had a million parts. There is a delicate balance between each of the parts and the tolerance is very low compared to the profile of the board. You also need to torque the enclosure parts to get them to fit together while balancing the assembly in one hand and not drop the Pi. Not at all optimal. In the end I broke a connector slot on the enclosure which required a little superglue to fix. Once I had the board in the enclosure I realized that the opening for the power port was off enough that it would require modification to accept the USB power cord. After taking the enclosure apart I used a project file to widen a couple of the openings (power, GPIO) and tried again, this time adding in the 26 pin ribbon for the Cobbler kit knowing I didn’t want to have to take this apart again just to add that in later. It was even more difficult to put the pieces together with the ribbon cable, but I got it to work. (BTW: Make sure colored ribbon is on pin 1 which is on the same end as the Pi power port.). The how-to for assembling the enclosure here seems to work fine, but doesn’t account for adding the ribbon cable. (I looked over the V2 of this box which uses bolts and nuts to hold together and I see there are problems with how you hold the nuts in place for assembly. I can only imagine the frustration with that version and the number of times nuts are dropped into the box only to have to open it and retrieve them to try again.) Better option: Make your own project box out of LEGOs.

Ada Fruit Cobbler Kit:
Basically the Ada Fruit Cobbler Kit is a simple device to connect the GPIO of the Pi to a breadboard making experimentation a little easier. The kit includes a PCB, socket, 26 pin ribbon and header pins. Assembly was pretty straightforward except for separation of the header pins. My kit came with the header pins in one stick with about 36 pins. You only need 26 (2x13) so breaking this down, while simple, still takes some care. I should have used jewelry pliers or side cutters which would have made sure I didn’t break it into 12, 13 and the balance. Adding back in one header pin is never fun and I should have known better.

Soldering was simple. First I soldered the socket to the PCB so I was working from the inside joints to the outside joints. Turning the assembly upside down worked well for this and my iron was still at a good temperature. I started from one end and worked my way down each set of pins, checked the joints and cleaned up one or two that were messy. Next I placed the 12 and 13 pins into a breadboard, set the PCB on that and worked from the middle pins out and then added the lone pin back in. (2x13 sticks would have made this much easier.) The header pin plastic melted a little because I was being aggressive, but a few tweaks and I was able level the socket with the pins so it didn’t look like my youngest child completed the work. It would have been a better experience if I had a soldering iron with temperature adjustment, smaller soldering tip and smaller diameter solder. My desolder tool helped when I found I had to reset a head pin that I accidentally pushed on with my iron.

I think assembly of the Ada Fruit Cobbler kit will be the most intimidating part of the kit for someone new to electronics. The kit calls out that you will need soldering skills and this is as basic a soldering job as you can get, but still some might shy away from it. I understand that more recent versions actually have the kit pre-assembled for those who don’t want to solder.

Good assembly instructions can be found here.

Working with the OS:
The Raspian OS is Lightweight X11 (LXDE) with Openbox. For non-Linux users this may seem a little scary but there is a whole body of work around this and outside of the scope of this review.

Configuring and setting up the OS on my home network was typical for a Linux install. I wish I had a wireless USB though so I didn’t need to rely on the Ethernet adapter and fear of having a cable pulled and dropping the device. Connectivity completed, I wanted to play with some programming on the device.

I was happy to see Python and IDLE in the install as it made writing a simple program to tinker with the system easy. Additional modules can be downloaded and installed easily. Sample programs are easy to find or write and are typical. At this point I had a working Linux desktop computer, the size of my wallet, connected to my network and a breadboard for experimenting with IO.

I have yet to run this headless but will do so at some point.

Working with IO:
After I completed the assembly items and tinkering, I picked out a project for the breakout board to see if there was something cool that I could show. I worked on the first simple IO example in the book and quickly found that the documentation is really poor for a first-timer.

The first example of GPIO work in the Getting Started book lays out that you should use male-to-female adapters, then promptly tells you that the Pi Cobbler makes it easier to experiment and then continues the experiment with mtf adapters, which aren’t included in the kit. It tries to compensate for this by using a really bad drawing of the GPIO pins that aren’t completely labeled and have caveats about versions of the board. So before wiring the board I had to do a little investigation about the version of the board which you can tell only by booting the device (a nice stamp on the board would have been nice). Fortunately I have a version 2 board making the wiring a little easier to follow. (More information on Pin IO can be found here.) I checked for errata on the book to see if some of it has been sorted out but didn’t see this addressed at the time I was setting the project up.

Note on IO projects: You should really make sure you have your circuits setup and buffered when working with external experiments. It is also important to understand how a breadboard works and which terminals are tied out. Basically if you aren’t careful and paying attention you can accidentally feed power back to your Pi and end up blowing it out. (Mixing the 3V3 and 5V will do that in an instant.) For a $35 board that isn’t too expensive of a lesson, but would probably cause a newbie to be quickly discouraged.

The ‘Hello World’ examples in the Getting Started for IO include lighting an LED and reading from a pushbutton. The setup for these circuits is pretty simple but the author of the experiment doesn’t explain well how the powerbus works on the breadboard which could easily lead to a project discouragement. Additionally, the diagrams are set for mtf jumpers so matching that to the Cobbler kit and making sure you get the correct pins there can be a problem. Reading IO in the samples was easy and was simply a matter of running as su and setting the direction of the pin and then echo or cat the value to set/read its state.

Other sample projects assume you have a PowerSwitch tail relay sitting around, which I don’t, so turning off an external device (table lamp) was out of the question in my first couple of experiments. I would have been nice to see either all of the experiments focused at what was in the kit, or to include all of the items needed for the experiments in the kit.

I tinkered with GPIO and Python to automate some of the work and it was all quite simple to do. Samples in the Getting Started were fine but as with most programming examples, there were some small typos.

I think for someone coming to this the first time the experiments in the book are pretty simple but assume some experience with electronics. For new electronics users I would recommend a copy of Make: Electronics as it does a good job of laying out Electricity 101 in straightforward terms. You will also want to start assembling some other break out tools which can be easily had from lots of sources.

I picked up a copy of Raspberry Pi Users Guide by Upton and Halfacree for more project ideas in the future and look forward to reading and working those projects. I also ended up getting a couple of other books about the Raspberry Pi to see what they have in them and will likely do a book review at some point about their content.

General Observations:
For $35 the Pi is a great buy but the problem is finding the companies who are selling it for that price; Make sells theirs for $50. The added project items needed in this kit seem to be a little pricy, causing the overall price to get it up to the $130 range. Ada Fruit Cobbler kits are running $8, Pi enclosures are running $15, USB chargers run about $7, 4GB cards run about $6, solderless breadboards about $15 and probably $10 for the extra parts in the box, $10 or so for the book. If you are already doing electronics hobby work, I would find a different sourced board and skip the extras here. If you are new and want to give this a try or want to one-stop the parts, then buy the kit.

There is a great deal of an IKEA effect by having you participate in the assembly and feel like you just made something cool. It was largely fun putting the parts together and I am thinking about project applications almost daily. One of our developers belongs to a racing club and we were thinking that these would be a cheap means of tracking and relaying car speed/vitals to a central unit. I am also curious to see if these would be a better solution for tracking car performance for those into hypermiling. In any case, I plan on trying a number of projects and continue to develop with the board.

Lessons Learned:

• A) Find a project box or assemble one of the nice Lego Pi Enclosures described out on the Internet. The project enclosure in the kit is fragile and difficult to assemble. There is a nice example made by a German Scout named Biz and can be found here. Or, if you are clever, you can make something bigger and better. As there is no heat-sink on board, I would avoid enclosures with a lid so you can vent any thermal from the board.

• B) The enclosed book is ok, but there are other resources that were more valuable in the setup.

• C) Get a powered USB device to control your mouse/keyboard, etc. There are only a couple of open slots on the Pi.

• D) An HDMI to DVI adapter is helpful.

• E) Get a longer HDMI cable to make this practical for experimenting.

Ian Lamont writes "The Economist writes that self-publishing threatens the existence of the International Standard Book Number (ISBN) regimen, which is used to track and distribute printed books. Self-publishing of e-books has experienced triple-digit growth in recent years, and the most popular self-publishing platforms such as Amazon's Kindle Direct Publishing don't require ISBNs (Amazon assigns its own reference number to these titles). But Bowker, the sole distributor of ISBNs in the United States, sees an opportunity in self-publishing. The packages for independent authors are very expensive — Bowker charges $125 for a single ISBN, and $250 for ten. It also upsells other expensive services to new and naive authors, including $25 barcodes and a social widget that costs $120 for the first year. Laura Dawson, the product manager for identifiers at Bowker, insists that ISBNs are relevant and won't be replaced anytime soon: 'Given how hard it is to migrate database platforms and change standards, I wouldn't expect to replace the ISBN, simply because it is also an EAN, which is an ISO standard that forms the backbone of global trade of both physical and digital items. There are a lot of middlemen, even in self-publishing. They require standards in order to communicate with one another.'" It seems like a lot of programs/services just use ASINs (despite being controlled by a single private entity), probably indicating some deficiency with the current centralized registration regime. Back in 2005, Jimmy Wales suggested we needed something (culturally) similar to wikipedia for product identifiers. The O'Reilly interview indicates that the folks issuing ISBNs think DOIs are DOA too.

Michael Ross writes "Upon hearing the name "Enyo," one may wonder if the speaker is referring to the Greek war goddess, or if it is the name of some Celtic New Age music with a Latin twist. In the world of front-end software development, Enyo is a cross-platform open-source JavaScript framework that can be used to build HTML5 web applications for the desktop and for mobile devices, including those powered by iOS and Android. The project website bills it as "an object-oriented JavaScript application framework emphasizing modularity and encapsulation." Any programmer interested in learning Enyo — or at least exploring what it is capable of — can consult the online documentation and the forums, but a more time-efficient approach might be to read a book focusing on the topic, such as Enyo: Up and Running, written by Roy Sutton, a contributor to the project." Read below for the rest of Michael's review. Enyo: Up and Running author Roy Sutton pages 74 pages publisher O'Reilly Media rating 6/10 reviewer Michael J. Ross ISBN 978-1449343125 summary A brief introduction to a mobile-inspired JavaScript framework. The book was published by O'Reilly Media on 6 February 2013, under the ISBN 978-1449343125. (My thanks to O'Reilly for providing a review copy.) On the publisher's page, visitors can find limited information about the book and its author, some reader reviews, links to purchase the electronic and print versions, and a page for errata (of which there are none, as of this writing). At 74 pages in length, this title comprises seven chapters, plus an appendix explaining how to set up a local development environment for working with Enyo, with a few options to choose from. Thus, the appendix is where most readers should and likely will start. In the preface, the author notes that the book assumes "some familiarity with HTML, CSS, or JavaScript"; that "or" should be an "and," since all three technologies are foundational to Enyo.

The first chapter introduces Enyo by examining a fairly simple web app — one that displays a traffic light on the web page. Naturally, in a black-and-white book such as this, the red/yellow/green colors are all in grayscale. Much more importantly, of the seven jsFiddle-hosted code examples provided in the chapter, the first five do not work (as of this writing), apparently because in each case there exists in the code some sort of control character, displayed as a red dot. (In the third example, the dot character is in the JavaScript and not the HTML.) Readers should delete that character and click the "Run" button, to see the intended results. The last two examples work only because the dot character comes after the closing </script> tag. It is baffling how these flaws could have gone undetected by the production staff and technical reviewers.

The material seems to raise as many questions as it answers. Assuming that the code printed in the book works (no downloadable code archive is offered), readers will probably be left pondering questions such as: Is create: function() some sort of constructor? Why isn't a new color passed through the call this.colorChanged()? Why is oldValue apparently not used? Where is setColor() defined? While it is a good idea to entice the reader to try a new technology by showing its capabilities, if that reader is expected to understand the example code presented, then it should be fully explained; otherwise, it should not be presented. As an alternative, the author could have limited the discussion to what functionality Enyo provides to the programmer, without listing source code in print or on jsFiddle. This would have provided the reader with greater motivation to invest the time and effort in learning what can be a challenging subject.

As a result of these early problems, this first chapter does not get the book off to a promising start. The second chapter, "Core Concepts," is perhaps the one that should have begun the book, because it describes many of the core ideas critical to Enyo: kinds, encapsulation, published properties, events, signals, inheritance, constructors, and statics. However, the pace is too fast for beginners, and more examples are needed to explain the concepts, step-by-step. By the bottom of page 11, countless readers will likely be bewildered with the terse discussion of getter and setter functions, "changed" functions, construction, and passed values (which are properties or not). Also, readers will again encounter the aforesaid problem of the red dot character breaking the example code on jsFiddle. (Further instances in the book will not be documented here.) The third chapter continues the discussion, focusing on components, menu and form controls, and functions, as well as some components for animation and making web requests. All of the information looks correct. The only puzzling aspect is why break tags are used (on page 22) instead of a CSS display: block; declaration.

User interface is addressed in the next two chapters, the first of which presents layout components commonly needed for Enyo apps — scrollers, repeaters, fittables, lists, and panels. The second one explores CSS styling of an Enyo app, performance considerations of apps on handheld devices, debugging, common mistakes, jsFiddle, internationalization, and localization. With these chapters, the narrative in the book becomes noticeably more comprehensible.

The penultimate chapter — essentially comprising two pages — delineates some options that the Enyo developer has for deploying a newly-built app to any one of the supported platforms. This chapter, like all the earlier ones, ends with a summary that is so brief, and applicable to so few pages, that each one seems pointless. Why do publishers feel obligated to include these useless chapter summaries in almost every technical book? The final chapter is a one-page conclusion, in which the author encourages readers to learn more and become involved in the Enyo community.

This book is more of an introduction, although no reason is provided as to why it was not instead made a more extensive treatment of the subject. Upon completing the book, the average reader will probably conclude that she did not absorb enough knowledge of the Enyo core to begin immediately developing apps using this framework, and the best course of action might be to start over again on page 1, or perhaps seek out a second source, before optionally returning to this one for a second run-through. The material could have been structured so all information is presented sequentially — so the reader does not encounter concepts yet unseen — with more step-by-step explanations.

Rather than presenting the reader with code snippets that have no relation to one another, it would have been much more interesting and motivating if the author had devised and explained code that incrementally builds into a nontrivial app. Furthermore, the example source code should have been made available on the publisher's website, so readers could avoid typing it from the text or extracting it from jsFiddle if they wished to try it in their local development environments.

In terms of typography, the font size of this book is a bit too small, especially for extended reading, and for people with subpar vision. This is even more true for the code snippets, which are in an even smaller font. In many of the lines of prose, the words are too close to one another — a problem exhibited in a few other recent O'Reilly titles. Did the production team feel it necessary to further compress a 74-page book?! In fact, proper names, such as those of components, are oftentimes broken between two lines in the text — sometimes nonsensically, e.g., "FittableR" followed by "owsLayout" (page 32). The book contains several errata: "This is [not] to say" (page viii), "such as [a] local installation" (viii), "url" (27), "we might modify add" (34), "woud" (35), "one [of] the most" (35), and "allow you [to] easily debug" (56). For such a slender volume, the production quality seems to have received less attention than it deserved.

Overall, this offering does not reach O'Reilly's usual high standards. It's a shame, because it seems like such a promising topic — one that could be more thoroughly explored in a larger volume. Perhaps this feedback, and that of other readers, could be folded into a second edition. This is a real possibility, given that the author notes in his conclusion that he considers the book an active project, and intends to keep it up-to-date with the changes to Enyo itself. In the meantime, this is a promising start that can give readers a taste of Enyo's potential for building modern web apps for desktop and mobile platforms.

Michael Ross is a freelance web developer and writer.

You can purchase Enyo: Up and Running from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

jones_supa writes "The classic hacker book publisher O'Reilly is releasing their book Open Government for free as a tribute for Aaron Swartz. The book asks the question, in a world where web services can make real-time data accessible to anyone, how can the government leverage this openness to improve its operations and increase citizen participation and awareness? Through a collection of essays and case studies, leading visionaries and practitioners both inside and outside of government share their ideas on how to achieve and direct this emerging world of online collaboration, transparency, and participation. The files are posted on the O'Reilly Media GitHub account as PDF, Mobi, and EPUB files."

jones_supa writes "The classic hacker book publisher O'Reilly is releasing their book Open Government for free as a tribute for Aaron Swartz. The book asks the question, in a world where web services can make real-time data accessible to anyone, how can the government leverage this openness to improve its operations and increase citizen participation and awareness? Through a collection of essays and case studies, leading visionaries and practitioners both inside and outside of government share their ideas on how to achieve and direct this emerging world of online collaboration, transparency, and participation. The files are posted on the O'Reilly Media GitHub account as PDF, Mobi, and EPUB files."

destinyland writes "O'Reilly and Associates just announced that they're offering a 50% discount on every ebook they publish for Cyber Monday. Use the code CYBERDAY when checking out to claim the discount (which expires at midnight). Amazon has also discounted their Kindle Fire tablets to just $129. Due to a production snafu, they've already sold out of the new Kindle Paperwhite, and won't be able to ship any more until December 21"

An anonymous reader sends this quote from the NY Times Bits blog: "If you want to know just how broken the patent system is, just look at patent D670,713, filed by Apple and approved this week by the United States Patent Office. This design patent, titled, 'Display screen or portion thereof with animated graphical user interface,' gives Apple the exclusive rights to the page turn in an e-reader application. ... Apple argued that its patented page turn was unique in that it had a special type of animation other page-turn applications had been unable to create." The article doesn't really make it clear, but this is for the UI design of showing a page being turned, not the actual function of moving from one page to another. That said, the patent itself cites similar animations in Flash from 2004.

theodp writes "U.S. tech talent shortage discussions tend to focus on getting more young people to go to college to become CS grads. Nothing wrong with that, writes Anil Dash, but let's not forget about education which teaches mid-level programming as a skilled trade, suitable for apprenticeship and advancement in a way that parallels traditional trade skills like HVAC or welding. Dash encourages less of a focus on 'the next Zuckerberg' in favor of encouraging solid middle-class tech jobs that are primarily focused on creating and maintaining tech infrastructure in non-tech companies. Dash also suggests 'changing the conversation about recruiting technologists from the existing narrow priesthood of highly-skilled experts constantly chasing new technologies to productive workers getting the most out of widely-deployed platforms and frameworks.'"

theodp writes "Blogger Floopsy complains that he would love to RTFM, but can't do so if no one will WTFM. 'You spend hours, days, months, perhaps years refining your masterpiece,' Floopsy laments to creators of otherwise excellent programming language, framework, and projects. 'It is an expression of your life's work, heart and soul. Why, then, would you shortchange yourself by providing poor or no documentation for the rest of us?' One problem with new program languages, a wise CS instructor of mine noted in the early look-Ma-no-documentation days of C++, is that their creators are not typically professional writers and shy away from the effort it takes to produce even less-than-satisfactory manuals. But without these early efforts, he explained, the language or technology may never gain enough traction for the Big Dogs like O'Reilly to come in and write the professional-caliber books that are necessary for truly widespread adoption. So, how important is quality documentation to you as a creator or potential user of new technologies? And how useful do you find the documentation that tech giants like Google (Go), Twitter (Bootstrap), Facebook (iOS 6 Facebook Integration), Microsoft (Windows Store apps), and Apple (Create Apps for IOS 6) produce to promote their nascent technologies? Is it useful on its own, or do you have to turn to other 'store-bought' documentation to really understand how to get things done?"

Michael Ross writes "Of all the open source content management systems used for building websites, Drupal has a reputation for being one of the most flexible and powerful available, but not the easiest for web designers to use. Drupal version 7 has made some strides in alleviating those flaws, but there is still much progress to be made. During the past few years, a number of books have been published that explain how Drupal designers can do custom theming, but they tend to focus on the technical details of the theme layer, and not the practice of web design when using Drupal as a foundation. That rich yet neglected subject area is the focus of a new book, Drupal for Designers: The Context You Need Without the Jargon You Don't." Keep reading to see what Michael has to say about the book. Drupal for Designers author Dani Nordin pages 328 pages publisher O'Reilly Media rating 8/10 reviewer Michael J. Ross ISBN 978-1449325046 summary How to design and manage Drupal projects. The book's author, Dani Nordin, is a Massachusetts-based web designer and the founder of The Zen Kitchen, a UX design business. The book was published by O'Reilly Media, on 1 August 2012, under the ISBN 978-1449325046. The publisher's page offers a description of the book, the table of contents, an author bio, and some free sample content (the first chapter). This publication is a compilation of three previously-released short guides — Planning and Managing Drupal Projects, Design and Prototyping for Drupal, and Drupal Development Tricks for Designers — with additional material. All of these books were written by Dani Nordin, and comprise the "Drupal for Designers" series by O'Reilly Media. (My thanks to the publisher for a review copy of this particular title.)

The book's material spans 328 pages, and is organized into seven parts, which do not include the introduction or the first chapter. The seven parts — each comprising at least two chapters — are largely presented in the same order that a typical reader would want to learn and implement the recommendations: Discovery and User Experience; Sketching, Visual Design, and Layout; Setting Up a Local Development Environment; Prototyping in Drupal; Making It Easier to Start Projects; Working with Clients; and Sample Documents.

Unlike most introductory Drupal books, this one wisely begins with a helpful dictionary of Drupal terminology. The first chapter also discusses the phases that compose a typical Drupal project lifecycle. Sandwiched in between is some guidance on where to place custom code in a Drupal directory system. The author advises that "Any module, theme, or other customization that you create for your site should always reside in sites/all" (page 2, and also reflected on pages 1 and 5). That may be true of contrib modules and themes, but certainly not custom ones, which are better located in sites/default or sites/[domain name]. She states that a child theme should be "stored separately in sites/all/<client_name>" (page 4). Actually, they should be placed in "sites/default/themes" or the themes subdirectory of a domain name directory. Finally, she recommends that for a multisite installation, one should keep "everything in sites/all" (page 5). Lumping everything into the "all" subdirectory would defeat the fundamental mechanism of multisite, which allows one to host multiple sites on a single Drupal installation, with their custom files and settings separated by domain name.

The first part of the book is loaded with valuable counsel on how to conduct the discovery phase of a website project, including coverage of project goals, user experience (UX), mockup tools, user personas, wireframes, prototypes, and the key components of a short-form project brief. It is evident from the narrative that the author is drawing upon a great deal of real-world experience, as well as lessons learned from other veteran web designers. The only blemish is where the author refers to "the project brief in Section 8" (page 45, repeated on page 254), and yet there appears to be no such section in the book. Perhaps she means Appendix A, which has an example project brief.

Once a design team has completed and received sign-off on a project brief — as well as any wireframes and other helpful preliminaries — a logical next step is to build the initial visual design. In the second part of the book, the author demonstrates how she uses sketches, style tiles, layout elements, greyboxing, grid systems, and Fireworks templates for crafting a visual design for a website. Throughout these chapters, she uses a redesign of her own personal website to illustrate the material. Both this part and the previous part of the book contain little information that is specific only to Drupal; thus, it could be useful to designers building websites using other CMSs.

Some readers of the book may already have up-to-date Drupal environments installed and configured on their development web servers. For those who do not, Part III will likely be appreciated, especially if the reader is using a Mac machine, because that is the environment to which the text and screenshots are geared. The author contends that "Windows seems to add an annoying layer of complexity to most of the command-line stuff" (page 102). Yet from my own experience, installing and using Git and Drush on a Windows PC is largely the same as in a Linux environment. Most developers complain that the main hurdle is Git's unintuitive workflow, which is independent of the operating system. The author touches upon some other tools, such as LESS and phpMyAdmin. Chapters 9 and 10 focus on Drush and Git, respectively. The last chapter in this section steps the reader through installing MAMP and Drupal. The discussion is generally comprehensible, except for the first paragraph on page 132, which is arguably the most confusing in the entire book. For instance, echoing a misstep seen earlier, it advises that all changes to your Drupal site should be stored in the sites/localhost directory, which contradicts the advice on the previous page, that all customizations to the site should be located in the sites/all directory.

The fourth part of the book covers prototyping in Drupal: gleaning from the client the information needed to define the content types for the website; choosing the appropriate modules for implementing the desired functionality; using views for displaying data; improving the HTML generated by views; creating custom Drupal themes; and using LESS to better manage the CSS within a theme. The advice is on target, except for the recommendation to use the Submit Again module, which does not have a Drupal 7 release, and has been replaced by the Add another module. Readers who are having difficulty locating the User Reference module mentioned by the author (page 187), can find it as a submodule in the References project. Lastly, the author instructs the reader to enable any base theme used (page 217), but actually it does not need to be enabled; installation alone is sufficient.

Part V, the briefest of them all, explains how to utilize the Features module, as well as Drush Make and installation profiles. Part VI comprises three chapters which offer guidance on how to propose an estimate for new projects, how to push back on unreasonable client requests, and how to learn from and document a finished project. This material is so closely related to that presented in the first part of the book — project discovery, planning, project briefs, etc. — that these final three chapters should have been incorporated into that earlier part. In fact, the first paragraph of this part states that it describes a phase of the discovery process that should be conducted prior to the phase described in Part I. Nonetheless, the author provides smart tips on some of the more difficult aspects of project management. The last part of the book comprises three appendices with sample documents — specifically, a project brief, a work agreement, and a project proposal.

On the publisher's page for the book, no errata have been reported, at this time. That is likely because the book appears to contain remarkably few errata: "What if there was" (pages 81 and 245; "was" should be "were"); "get familiar [with] the command line" (page 108); "a couple of" (page 172; should be "a few," as it is referencing three bullet points); ".less" (page 208, twice; should be "LESS"); "carpal tunnel[s]" (page 231); "original code [for] a feature" (page 242); and ".tpl" (page 266; should be ".tpl.php"). This is certainly a low number of errata for a technical book of this size. Kudos to the author and the O'Reilly editing team.

Overall, the book's style is clear and conversational, with only a few rough patches. Incidentally, the terms "directory" and "folder" are synonymous, but newbie readers who do not understand this could be confused when the two terms are used interchangeably, especially within the same sentence (e.g., page 109). Interspersed at various points in the text are interviews with people involved in web design, entitled "From the Trenches," which add perspective from designers other than the author. The reader will also find some natural humor and humility, which is always welcome in a technical work.

The author and publisher have made good use of the many screenshots, showing sample designs, Drupal user interface pages, etc. Unfortunately, for the Drupal pages, the admin theme used is the default, Seven, which results in black text on a gray background — a poor choice for such wide screenshots being compressed into small images on the page. Consequently, much of the text is barely legible, especially for anyone with imperfect eyesight.

From a technical point of view, the information provided is accurate and worthwhile. The only serious problem is the misleading advice, noted above, concerning the placement of custom modules and themes within the directory structure of a Drupal project — which was undoubtedly unintentional. The reader will encounter some HTML markup, a lot more CSS, and a minimal amount of PHP code. All of it is neatly formatted, and the only apparent problem is where a snippet of example code includes invalid nested "<?php" tags (page 188).

Despite these minor blemishes, this is one of the better-written Drupal books on the market. Web designers who will be working on Drupal projects, should be well rewarded in choosing this book as a solid starting point for their studies.

Michael J. Ross is a freelance web developer and writer.

You can purchase Drupal for Designers from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Michael J. Ross writes "Veteran computer programmers — adept with languages such as PHP, Perl, and JavaScript — typically have no trouble learning an additional language, often just by reading online tutorials and stepping through sample code. But for those new to programming, that approach can prove difficult and frustrating. Yet nowadays there appears to be growing interest among such people for learning how to write programs in Python, especially as it is seeing increasing use by Google and other organizations, and is often chosen as the primary teaching language in schools. For such budding programmers, one possible starting point is the book Head First Python." Read on for the rest of Michael's review. Head First Python author Paul Barry pages 494 pages publisher O'Reilly Media rating 6/10 reviewer Michael J. Ross ISBN 978-1449382674 summary A gentle introduction to the Python programming language. The book's author Paul Barry, is an experienced programmer and lecturer in computing science. The book was published by O'Reilly Media, on 7 December 2010, under the ISBN 978-1449382674. On the publisher's page, visitors can learn more about the book and its author, see feedback from readers, peruse the discussion forum, and read the errata identified so far (many of which appear to be fixed in the most recent printing). The book's example code is available from Head First Labs. It is packaged into a large Zip archive file, containing directories for all the chapters. Additional resources — such as test data — can be downloaded from a page on the website of the school where the author teaches, The Institute of Technology, Carlow in Ireland. (But don't bother trying to use the site's search functionality to find more information about him, because no search results are returned, as of this writing. Hmm, what is the Goidelic word for "fail?")

The book's material, spanning 494 pages, is organized into 11 chapters, plus a supplementary section for more advanced topics. In the first chapter, "Everyone loves lists," the author briefly explains how to find and install on one's computer Python and its IDLE development environment, if necessary. Oddly, he assumes that if it is already present on a Windows machine, then it will be found in the directory c:\Python31; this is repeated in the second chapter as well. More importantly, he mentions the (command-line) command to start the Python shell, but then immediately begins describing IDLE, without mentioning how to get it running. Most of the chapter explains how Python supports lists and functions, although the coverage of the latter topic is much too brief, considering how critical functions are in any programming language, including Python.

The second chapter shows how to bundle up functions into modules, and make them available to other developers through distribution utilities. This may be an important topic, yet it is inconceivable to me as to why, so early in the book, the author dives into the rather involved details of registering with PyPI and distributing a Python module to the rest of the world, before even introducing such language basics as input/output, files, and objects. (More on that later.)

If the reader perseveres through all the flaws in the first two chapters, then she will likely find that the quality of the narrative gradually improves throughout the rest of the book. An example of this is the third chapter, which addresses basic file access and exception handling. The following chapter, which focuses on data persistence, extends the discussion of how to save in-memory data in files, including the use of the standard library "pickle" for dumping and later loading any type of data. The next few chapters introduce the reader to increasingly complex ways of representing data, in the form of lists (again), dictionaries, and (object-oriented) classes. The discussion is fairly clear, except readers familiar with other object-oriented languages may be confused on page 194, where class attributes are being defined in the constructor only. In fact, the author does not fully explain where attributes can be defined, and what their resultant scope would be.

The seventh chapter roughly marks the halfway point in the book, and also is where the focus shifts from developing Python programs to releasing them to the public. The author begins by incorporating techniques presented earlier in the book, to build a Python-based web app, using the popular Model-View-Controller (MVC) paradigm. The next chapter shows how to port those web apps to mobile devices. Chapter 9 extends these topics, by demonstrating how to query for and accept user data in HTML forms on web pages, and in similar dialogs on Android phones — as well as how to store that data in an SQLite database. The next logical step is to learn how to make Python apps scalable, as well as cloud-based, in this case using the Google App Engine (GAE). Readers are also introduced to Django, for form validation. The final chapter, and the appendix, cover a number of miscellaneous topics, such as how to spreadsheet-type data storage, development IDEs, variable scope, unit testing, and regular expressions.

Most readers should find that the relaxed writing style makes this book approachable, though fairly lengthy. Like most if not all of the other books in the Head First lineup, this one is made more palatable with a sense of humor that is not silly — a welcome improvement to any programming book. The best parts are in the "fireside chats," which are fictional and sometimes pointed conversations, oftentimes between two similar components of Python.

Yet the narrative is far from perfect. Some of the phrasing is ponderous, e.g., "Perform the edits to your code" (page 113); why not just "Edit your code?" A few concepts are used without explanation, e.g., the Python value of "None" (page 80). Some technical slang terms are used quite early in the book, yet with no definitions or explanations for the neophyte — e.g. "app" and "webapp" (both on page xvi). Admittedly, most readers of this book will be programmers or other techies, who feel comfortable with those terms. But the Head First series is intended for people just getting started, and thus all such terms should be explained up front, if only briefly.

There are numerous errata not found in the aforesaid list (on the O'Reilly Media website): "tools to for accepting" (page xviii), "design user-friendly websites" (xxvii; should be "learn Python"), "try and" (14, 84 twice, and 188; should be "try to"), "As your plan to" (37; "you plan" was likely intended), "utilities" (49), "users that" (50; should be "users who"), "argument" (52), "a iterable" (53 and 54), "the the list on screen" (64), "the your latest version" (65), "a argument" (71), "gives you with a chance" (88), "file's contents" (123; incorrect plural possessive), "the facilities pickle" (134; probably should be "the pickle facilities"), "your were" (170), "it's new mode" (185; no apostrophe in "its"), and at this point I ceased recording errata. These flaws, plus the 119 already reported, suggest that the author and O'Reilly copy editors did not perform enough quality control when working on the manuscript.

One glaring problem with the material is the dramatic unevenness in the depths of coverage, from one topic to the next. For instance, even though the book is aimed at readers who already know another programming language (page xxiv), an entire section is devoted to establishing the (obvious) non-scalability of having to use an individual print() statement for every item in a list, and the superiority of instead iterating through the list (page 15). Yet in the same chapter, only a couple pages (29-30) are provided to explain functions, with no discussion of topics important to any programmer, such as whether one can set parameters to default values, or how to pass arguments by reference, if it's even possible. Some of this unevenness is due to the book's goal of presenting technical information in an easily-digestible form; for instance, in Chapter 7, the reader is told how a web server returns a web page, but later faces the minefield of invoking Python's built-in web server on a CGI server.

A second significant problem is the often-illogical ordering of material — both high level and low. Chapter 2 illustrates both cases: The author recommends that the reader release her code to the public (in the form of modules) after presumably having learned only one chapter's worth of material, when such code would undoubtedly be unready for public consumption. At a much lower level, the first "There Are No Dumb Questions" entry in the chapter briefly discusses various editors that could be used for writing Python code. This information should have been presented at the beginning of the first chapter, or even in the introduction — not after the reader has already chosen whatever editor seemed most convenient at the moment, and probably saved at least one file of code. A third variation of this problem, is when the reader is tested on concepts not previously presented — such as the "Who Does What?" section on page 53, which quizzes the reader on half a dozen functions, none of which the reader has presumably even seen before, and some of which are never mentioned again in the book!

In terms of the typography of the print version of the book, I found the font face and size to be problematic, especially on those pages where the text appears to have been printed a bit too lightly — at least in the review copy of the book kindly provided to me by the publisher. For instance, all of the code on page 227 is practically unreadable. Throughout the book, in the snippets of code, the comments are a light gray making them quite difficult to read — probably because these are screenshots of IDLE sessions, in which comments are by default displayed in green, which apparently does not translate well to grayscale illustrations.

Overall, this book appears to have received less oversight and editing than is typical in the Head First series. If you are a veteran programmer searching for a concise treatment of Python, then this book is not optimal for you. Yet if you seek an engaging and highly visual introduction to an elegant programming language, then Head First Python can be a worthwhile place to start your journey.

Michael J. Ross is a freelance web developer and writer.

You can purchase Head First Python from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

blackbearnh writes "Everyone these days knows that you have to double- and triple-check your code for security vulnerabilities, and make sure your servers are locked down as tight as you can. But why? Because our underlying operating systems, languages, and platforms do such a crappy job of protecting us from ourselves. The inevitable result of clamoring for new features, rather than demanding rock-solid infrastructure, is that the developer community wastes huge amounts of time protecting their applications from exploits that should never be possible in the first place. The next time you hear about a site that gets pwned by a buffer overrun exploit, don't think 'stupid developers!', think 'stupid industry!'"

theodp writes "If you're a COBOL programmer, you're apparently persona non grata in the eyes of the nation's Chief Information and Chief Technology Officers. Discussing new government technology initiatives at the TechCrunch Disrupt Conference, Federal CIO Steven VanRoekel quipped, 'I'm recruiting COBOL developers, any out there?,' sending Federal CTO Todd Park into fits of laughter (video). Lest anyone think he was serious about hiring the old fogies, VanRoekel added: 'Trust me, we still have it in the Federal government, which is quite, quite scary.' So what are VanRoekel and Park looking for? 'Bad a** innovators — the baddest a** of the bad a**es out there,' Park explained (video), 'to design, create, and kick a** for America.' Within 24 hours of VanRoekel's and Park's announcement, 600 people had applied to be Presidential Innovation Fellows."

SWroclawski writes "Well known hacker and hackerspace advocate Mitch Altman has decided to temporarily part ways with Maker Faire over their involvement with DARPA (as reported on Twitter and Facebook). This public parting of ways raises the question of what role government, especially the military, should play in working alongside hackers and educators."

Michael J. Ross writes "Prior to Google+, the company's previous attempts at social networking — Orkut, Dodgeball, Jaiku, Wave, and Buzz — were largely failures, and tended to frustrate users who had devoted time and effort to contributing content and establishing connections with other users, only to see the services wither on the vine. In contrast, Google+ appears to be receiving far more nurturing by the Internet behemoth, and as a result has arguably better chances of not just surviving, but expanding to the point of eventually challenging Twitter and Facebook. Like its rivals, Google+ offers online help information to explain to newcomers the basics of how to use the service. But there is little to no advice on how to make the most of its capabilities, and even the basic functionality is not always clearly explained. That is the purpose of a new book, Google+: The Missing Manual." Keep reading for the rest of Michael's review. Google+: The Missing Manual author Kevin Purdy pages 232 pages publisher O'Reilly Media rating 7/10 reviewer Michael J. Ross ISBN 978-1449311872 summary An introduction to Google's social networking service. Authored by Kevin Purdy, the book was published by O'Reilly Media, on 30 December 2011, under the ISBN 978-1449311872. The publisher's page has a brief description of the book, its table of contents, some comments on the book from customers and reviewers, a couple errata (as of this writing), and links for purchasing the print version (such as the one kindly provided to me by the publisher) and/or the e-book versions (in EPUB, MOBI, and PDF formats). The "missing CD" page has links to most if not all of the online resources mentioned in the text.

Like the other entries in the Missing Manual series, this one starts with the basics, and builds upon that foundation. It does not assume any knowledge of Google+, or even possession of a Google account.

The book's material is organized into nine chapters, for a total of 232 pages. The first chapter, "Getting Started," explains exactly how to join Google+, invite friends to your new network, and configure your profile, including your privacy settings and a photo (even tweaking it online). The second chapter, "Managing Contacts with Circles" covers how to create new circles, edit and organize existing ones, share them with other Google+ users, and find people to add to your circles. But, oddly, the information is not presented in that logical order. The author explicates the advantages of using more than the default four circles provided by Google. Some points are repeated, but briefly enough that it is inconsequential.

While the first two chapters lay the foundation for joining Google+ and setting up your account and circles, the next three chapters explore the details of using this service — starting with "Streams, Sharing, and Privacy," which explains the various types of streams (main, circle, Notification, and the now-defunct Incoming stream), as well as the user interface elements for those streams and the individual posts they comprise. The author also demonstrates how to write your own posts, specify who gets to see them, edit your posts, and interact with the posts submitted by other users. The next chapter explores the important topic of notifications, which are sent as e-mail messages, smartphone messages, etc. Helpfully, the author discusses the differences between the user interfaces of the Android and iPhone notification apps. The subsequent chapter fully explains how to share photos and videos with other Google+ users, as well as how to upload and perform basic editing of images. However, it may have been more logical to present the latter information before the former.

For people who want the capabilities previously only provided by commercial web conferencing services, hangouts might be the most welcome feature of Google+. Chapter 6 explains how to set up and participate in these videos/audio meetings online, as well as how to incorporate Google Chat, YouTube videos, and Android devices. The subsequent chapter, "Searching and Sparks," has plenty of advice on how to search for other Google+ users and the content they contribute. The penultimate chapter dives into the differences you may encounter when using Google+ on small screen devices — specifically, Android and Apple smartphones and tablets. The last chapter, which is the briefest of the bunch, is also likely to prove the least useful to most readers, as it covers how to get started playing the games built into Google+.

The book does not cover Google+ Pages, which was likely introduced after the final draft of the book was submitted to the publisher. Readers are directed to an untitled 14-page PDF file that covers the essentials of Google+ Pages. Oddly, the publisher's page links to that file with the text "Download Example Code"; but there is no example code for this book. The supplement contains a few flaws: "box pop-up box" (page 4), "using a promoting your Page" (9), and "his her name" (11).

Speaking of which, given the relatively modest number of pages in this book, and the limited amount of text on each one, this book contains far too many errata: "works different" (page xiii; echoes of Apple's infernal "Think Different" marketing campaign?), "If typing web addresses by hand that isn't" (page 3), "a different a social networking site" (4), "she's added you [to] her" (54), "added to [the] +Add box" (58), "even if [you] just" (79), "and the[n] click the" (79), "settings that lets you can choose" (83), "modicum [of] more fuss" (105), "share its photos [with] specific circles" (117), "where [the] photo" (124), "just like [the] lightbox view" (126), "and or" (147; should read "and/or"), "an job" (148), "how to [use?] Google+ running" (169), "search find" (170), "bring up to the same list" (180), "The form exact" (185; should read "The exact form"), "you can't get start" (191), "in in" (193), and "a box let you know" (194).

Some of the statements in the narrative are odd — for instance, "Halloween right around October 31" (page 7; when else would Halloween occur?). Other phrases are poorly worded — for instance, "whenever you feel irked or like something must be broken" (44), "maybe an extra like a link" (60), and "select an item from the menu that appears to see only circle-related notifications" (80). Lastly, at least one pair of verbs have inconsistent form ("start" and "mentioning" on page 62). All of these blemishes should have been caught by the copyediting crew. But for the most part, the narrative is straightforward. It is occasionally livened up with a bit of humor, which is good, because portions of the text begin to sound the same, as a result not so much of the author's writing, but more the Google+ interface itself.

Only a few technical errors are immediately evident — for instance, on page 61, the author refers to a for-loop in computer code incorrectly: "+1 is a common way of making a program run over and over again." But it is not a program that is being repeated, but rather a code block.

Scattered throughout the text are numerous text boxes — most of which are labeled "Note" or "Tip." Unfortunately, they are set in a font that is a bit too small for comfortable reading. Also, there does not appear to be any difference among these types of information sections, yet there are at least half a dozen different names for them.

All of the key topics are nicely illustrated with sample screenshots, in grayscale, oftentimes with relevant controls circled or otherwise indicated. The only weakness is that the author typically does not mention which figure is being referenced in the text — not that that would help much anyway, since none of them have figure numbers. It's usually clear from the context, but not always.

Yet the very existence of this book may give readers some pause: If a book of this size is required to explain how to use a social networking service aimed at the general public, perhaps the Google+ user interface needs to be overhauled and made more intuitive? Yet that process is probably underway, because Google+ is under constant revision. Thus there will be portions of the text and screenshots that differs somewhat from the current incarnation of the user interface and its features. But for most of these instances, it is easy enough to determine how what you read in the book correlates with what you might see on the screen.

The primary weakness of this book is that it does not attempt to explain how Google+ might be integrated into a business's online marketing strategy, nor how it compares against Facebook or Twitter in terms of its advantages and disadvantages. In fact, as noted above, the book addresses Google+ Pages only in a supplementary document. Such information would have made this entry in the Missing Manual series far more valuable.

However, one forte of this book is that the author has clearly put effort into learning and explaining the privacy implications of the various Google+ features — critical in this era of evaporating privacy and data breaches on an unprecedented scale.

On balance, he largely achieves his objective. Google+: The Missing Manual is an informative and approachable introduction to Google's social network.

Michael J. Ross is a freelance web developer and writer.

You can purchase Google+: The Missing Manual from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Michael J. Ross writes "Prior to Google+, the company's previous attempts at social networking — Orkut, Dodgeball, Jaiku, Wave, and Buzz — were largely failures, and tended to frustrate users who had devoted time and effort to contributing content and establishing connections with other users, only to see the services wither on the vine. In contrast, Google+ appears to be receiving far more nurturing by the Internet behemoth, and as a result has arguably better chances of not just surviving, but expanding to the point of eventually challenging Twitter and Facebook. Like its rivals, Google+ offers online help information to explain to newcomers the basics of how to use the service. But there is little to no advice on how to make the most of its capabilities, and even the basic functionality is not always clearly explained. That is the purpose of a new book, Google+: The Missing Manual." Keep reading for the rest of Michael's review. Google+: The Missing Manual author Kevin Purdy pages 232 pages publisher O'Reilly Media rating 7/10 reviewer Michael J. Ross ISBN 978-1449311872 summary An introduction to Google's social networking service. Authored by Kevin Purdy, the book was published by O'Reilly Media, on 30 December 2011, under the ISBN 978-1449311872. The publisher's page has a brief description of the book, its table of contents, some comments on the book from customers and reviewers, a couple errata (as of this writing), and links for purchasing the print version (such as the one kindly provided to me by the publisher) and/or the e-book versions (in EPUB, MOBI, and PDF formats). The "missing CD" page has links to most if not all of the online resources mentioned in the text.

Like the other entries in the Missing Manual series, this one starts with the basics, and builds upon that foundation. It does not assume any knowledge of Google+, or even possession of a Google account.

The book's material is organized into nine chapters, for a total of 232 pages. The first chapter, "Getting Started," explains exactly how to join Google+, invite friends to your new network, and configure your profile, including your privacy settings and a photo (even tweaking it online). The second chapter, "Managing Contacts with Circles" covers how to create new circles, edit and organize existing ones, share them with other Google+ users, and find people to add to your circles. But, oddly, the information is not presented in that logical order. The author explicates the advantages of using more than the default four circles provided by Google. Some points are repeated, but briefly enough that it is inconsequential.

While the first two chapters lay the foundation for joining Google+ and setting up your account and circles, the next three chapters explore the details of using this service — starting with "Streams, Sharing, and Privacy," which explains the various types of streams (main, circle, Notification, and the now-defunct Incoming stream), as well as the user interface elements for those streams and the individual posts they comprise. The author also demonstrates how to write your own posts, specify who gets to see them, edit your posts, and interact with the posts submitted by other users. The next chapter explores the important topic of notifications, which are sent as e-mail messages, smartphone messages, etc. Helpfully, the author discusses the differences between the user interfaces of the Android and iPhone notification apps. The subsequent chapter fully explains how to share photos and videos with other Google+ users, as well as how to upload and perform basic editing of images. However, it may have been more logical to present the latter information before the former.

For people who want the capabilities previously only provided by commercial web conferencing services, hangouts might be the most welcome feature of Google+. Chapter 6 explains how to set up and participate in these videos/audio meetings online, as well as how to incorporate Google Chat, YouTube videos, and Android devices. The subsequent chapter, "Searching and Sparks," has plenty of advice on how to search for other Google+ users and the content they contribute. The penultimate chapter dives into the differences you may encounter when using Google+ on small screen devices — specifically, Android and Apple smartphones and tablets. The last chapter, which is the briefest of the bunch, is also likely to prove the least useful to most readers, as it covers how to get started playing the games built into Google+.

The book does not cover Google+ Pages, which was likely introduced after the final draft of the book was submitted to the publisher. Readers are directed to an untitled 14-page PDF file that covers the essentials of Google+ Pages. Oddly, the publisher's page links to that file with the text "Download Example Code"; but there is no example code for this book. The supplement contains a few flaws: "box pop-up box" (page 4), "using a promoting your Page" (9), and "his her name" (11).

Speaking of which, given the relatively modest number of pages in this book, and the limited amount of text on each one, this book contains far too many errata: "works different" (page xiii; echoes of Apple's infernal "Think Different" marketing campaign?), "If typing web addresses by hand that isn't" (page 3), "a different a social networking site" (4), "she's added you [to] her" (54), "added to [the] +Add box" (58), "even if [you] just" (79), "and the[n] click the" (79), "settings that lets you can choose" (83), "modicum [of] more fuss" (105), "share its photos [with] specific circles" (117), "where [the] photo" (124), "just like [the] lightbox view" (126), "and or" (147; should read "and/or"), "an job" (148), "how to [use?] Google+ running" (169), "search find" (170), "bring up to the same list" (180), "The form exact" (185; should read "The exact form"), "you can't get start" (191), "in in" (193), and "a box let you know" (194).

Some of the statements in the narrative are odd — for instance, "Halloween right around October 31" (page 7; when else would Halloween occur?). Other phrases are poorly worded — for instance, "whenever you feel irked or like something must be broken" (44), "maybe an extra like a link" (60), and "select an item from the menu that appears to see only circle-related notifications" (80). Lastly, at least one pair of verbs have inconsistent form ("start" and "mentioning" on page 62). All of these blemishes should have been caught by the copyediting crew. But for the most part, the narrative is straightforward. It is occasionally livened up with a bit of humor, which is good, because portions of the text begin to sound the same, as a result not so much of the author's writing, but more the Google+ interface itself.

Only a few technical errors are immediately evident — for instance, on page 61, the author refers to a for-loop in computer code incorrectly: "+1 is a common way of making a program run over and over again." But it is not a program that is being repeated, but rather a code block.

Scattered throughout the text are numerous text boxes — most of which are labeled "Note" or "Tip." Unfortunately, they are set in a font that is a bit too small for comfortable reading. Also, there does not appear to be any difference among these types of information sections, yet there are at least half a dozen different names for them.

All of the key topics are nicely illustrated with sample screenshots, in grayscale, oftentimes with relevant controls circled or otherwise indicated. The only weakness is that the author typically does not mention which figure is being referenced in the text — not that that would help much anyway, since none of them have figure numbers. It's usually clear from the context, but not always.

Yet the very existence of this book may give readers some pause: If a book of this size is required to explain how to use a social networking service aimed at the general public, perhaps the Google+ user interface needs to be overhauled and made more intuitive? Yet that process is probably underway, because Google+ is under constant revision. Thus there will be portions of the text and screenshots that differs somewhat from the current incarnation of the user interface and its features. But for most of these instances, it is easy enough to determine how what you read in the book correlates with what you might see on the screen.

The primary weakness of this book is that it does not attempt to explain how Google+ might be integrated into a business's online marketing strategy, nor how it compares against Facebook or Twitter in terms of its advantages and disadvantages. In fact, as noted above, the book addresses Google+ Pages only in a supplementary document. Such information would have made this entry in the Missing Manual series far more valuable.

However, one forte of this book is that the author has clearly put effort into learning and explaining the privacy implications of the various Google+ features — critical in this era of evaporating privacy and data breaches on an unprecedented scale.

On balance, he largely achieves his objective. Google+: The Missing Manual is an informative and approachable introduction to Google's social network.

Michael J. Ross is a freelance web developer and writer.

You can purchase Google+: The Missing Manual from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

You asked Carl Malamud about his experiences and hopes in the gargantuan project he's undertaken to prod the U.S. government into scanning archived documents, and to make public access (rather than availability only through special dispensation) the default for newly created, timely government data. (Malamud points out that if you have comments on what the government should be focusing on preserving, and how they should go about it, the National Archives would like to read them.) Below find answers with a mix of heartening and disheartening information about how the vast project is progressing.

LoC?
by an Anonymous Reader

So how many GB/TB is a Library of Congress? :)

Or, more seriously, how big are you estimating? Are you using raw scans or some sort of compression (JPG, PNG, etc)? What resolution are you using? Do you vary the resolution depending on the document?

What sort of meta data are you putting in?

CM: The reason John Podesta and I suggested a Federal Scanning Commission in our letter at YesWeScan.Org is we really don't know how big the holdings of the government are. I can tell you that the Library of Congress is about 32 million cataloged books (a significant increase from the 6,487 books Thomas Jefferson donated to get them started). But, this is about more than books, it is about paper records, microfilmed technical papers, video, audio, photographs, and much more.

The scale is fairly vast. The Smithsonian has 137 million objects, including about 13 million images. David Ferriero, the Archivist of the United States estimates he has over 10 billion pages of text documents, 7.2 million maps, and 40 million photographs including everything from past census records to presidential dinner menus, and that includes about 7.5 million motion pictures and sound recordings. The Government Printing Office distributes their documents to the Federal Depository Library Program, and that includes over 60 million pages of collections including the Official Journals of Government such as the Federal Register. That's just scratching the surface, and we recommended a Federal Scanning Commission to begin the process of understanding what we have (and what is worth digitizing).

As to standards? There are lots of pretty good standards on how to digitize. NARA, Library of Congress, GPO all spec out document scans at 400 dpi, for example. For photographs, moving images, and other objects, there are some pretty good and pretty detailed standards at www.digitizationguidelines.gov. I know Brewster Kahle's operation and my own tend to work off those specifications (in fact Brewster does quite a bit of scanning for the government).

As to compression? Well, I've found people tend to overcompress things. That said, sometimes the initial quality isn't that great, so a 600 dpi uncompressed scan would be silly in some cases. But, for photographs I try very hard to keep the TIFF images around and not rely on JPEG. Likewise, for audio it is really nice to keep a nice 48 khz version of your file around if you can simply because if you screw up the compression maybe somebody else can do a better job in a few years. Disk space is relatively cheap, so that isn't the barrier it used to be. For video, I rip MPEG2 at whatever it is on a DVD, when I'm actually digitizing I try to get the video bitrate up to 8-10 mbps when ripping a Betacam or Umatic. Some people think that is overkill, but I'd rather be safe than sorry.

Metadata? Well, you got to have it or you're not going to get very far when it comes to access. Many librarians have made perfect the enemy of the good when it comes to metadata and have resisted any attempt at digitization because we don't have the very best metadata we might have. I'm more in the camp of scan what you have and get as much of the metadata as you can into it. For example, we have 3,200 1000-page volumes of briefs from the 9th Circuit of the U.S. Court of Appeals. We didn't have good metadata, but we had the Internet Archive scan them anyway. Then, after we got our PDF files, I shipped those off to a double-key team in India and they broke the briefs up into individual documents and typed the metadata into a spreadsheet for me, which we hope to release soon.

My point is that sometimes you can shoehorn the metadata in after the fact or you can use a variety of techniques to pull the metadata out of the documents (e.g., smart OCR). In theory, you can use crowdsourcing to get the metadata, but so far I've not had a lot of luck persuading thousands of people to spend their time doing that kind of work. A captcha is a quick thing to do and is between you and something you want, whereas entering metadata in for videos or documents is one of those civic duty things that everybody thinks everybody else should be doing.

Total size? Brewster says a book is about 400 Mbytes (though he's very quick to point out that you could put the words in all the books in the library into a terabyte and if you're distributing PDFs, you can easily throw 130,000 full-color, searchable PDFs onto a 4 TB drive). But, you were probably asking about raw data. Here's some raw numbers:

32 million books at 400 Mbytes each is 12.8 petabytes 50 million photos at 150 Mbytes each is 7.5 petabytes 10 billion pieces of paper ("records") at 100 Kbytes each is 1 petabyte 20 years of video at 8 mbps is only 630 Tbytes.

(Somebody check my math?)

If you're talking a decade-long federal digitization initiative, we're looking at well south of 50 petabytes, which seems pretty doable in this day and age!



Can the rare books collections be digitized?
by autophile

Three closely related questions about the rare books collections at the Library of Congress:

1. I know there is some kind of effort going on to digitize the rare books collections, but can it be sped up? There are many high-quality low-cost archival book scanners out there (such as the ones developed at diybookscanner.org).

2. It gets really annoying to have to receive paper copies of books when copies are requested. Why not DVDs of high-quality images?

3. Why is there no outreach by the LoC to smaller, cheaper book scanning efforts? The Internet Archive, DIYBookscanner.org, and Decapod all come to mind.

CM: In reverse order. I don't know why we aren't distributing and decentralizing our scanning efforts. The Internet Archive is a heavy-duty production shop and they do an amazing job, as do folks like Google Books and the folks digitizing things the Mormon Church. But, there are a bunch of DIY solutions and it would be really nice if we could get more people pitching in. The biggest problem on distributing the digitization efforts is quality control. I know when it comes to ripping video, I can easily teach other people how to grab an MPEG2 off a DVD, but when it comes to things like digitizing a Betacam, that takes some training. But, we're all trainable and I wish we could all do more.

Getting back paper copies of books and papers when they're doing a copy anyway is just plain dumb. Likewise with things like FOIA results. John Podesta testified before the Senate about FOIA and said if an agency answers a FOIA request, they should also post their result online so others can see it. That seems pretty obvious.

As far as digitizing rare book collections, there are some amazing pockets throughout the government but there is no real coordination and there certainly is no effort to scan at scale or to come up with a realistic national digitization strategy. That is why we called on the White House to lead the effort. Within the Library of Congress there are some amazing collections, but if you look around to places like the National Agricultural Library or the National Library of Medicine or the libraries in the service academies you'll find lots more. Some have argued that digitizing rare books is silly because the audience is just a few academics, but I can tell you from my own experience helping host the network site for the Archimedes Palimpsest that when you make this kind of information available, there is an amazing long tail.

If you scan it, they will come. And, to answer your question, if we all scan it, they will come much sooner.



Real time legislation drafting
by kerskine

Would it be possible to implement a system that would allow real-time and continuous review of legislation while it's being drafted? Much has been made over the past three years about legislation being available for review before voting by the House or Senate. The final draft for review usually is huge PDF that makes it near impossible for citizens, interest groups, and the media to thoroughly analysis in time.

CM: You want to see the sausage being made not just buy the hot dog! I'll comment on the U.S. Congress since that's the system I know best. Thomas is a pretty good system if you happen to be stuck in 1994. It does have all the amendments and the actions and the various stages that legislation go through. But, it isn't real time, more like "pretty quick." As Van Jacobson once quipped, "Same day service in a nanosecond world." And, Thomas isn't really machine processable, it is final form, usually formatted ASCII text (shades of NROFF!). People like Josh Tauberer who built GovTrack.US have spent considerable time crawling those systems and trying to get the data into regularized formats and make it available to others to reuse via APIs, but that isn't the same as exposing the inner working of the sausage factory.

Majority Leader Cantor's staff has been pushing a system to make the raw data all available in XML from the Clerk's office and I think that is a very promising initiative which hopefully will bear fruit. (They're having a February 2 conference to discuss their plans if you are interested. I have no idea if it will be streamed for those of who aren't Inside the Beltway and I don't know their schedule for moving past conferences and into production.)

Congress is a pretty complicated beast. I know some folks like Sean McGrath have had better luck with some of the state legislatures. The problem is you need to dig deep into the inner working of a legislature. In the Congress, that means you're changing things like authoring tools that are used in the Clerk's office and by all the staff members, so you have to be careful or you get a bunch of really angry Congressman yelling at you because their staff can't crank out the flavor-of-the-week in the form of a bill or amendment.

There's also a bit of an issue of will. My work with the Congress to put hearings on-line showed that you could take the official transcripts of a hearing and use those to generate closed captions on the video. All you need is the official transcript of the hearing, but in order to get those I had to execute a special Memorandum of Understanding with the House Oversight Committee. Other committees guard their transcripts jealously and won't let them out for several when. When I started processing a bunch of historical videos we purchased from C-SPAN, I went to the Government Printing Office and found that many committees never deliver their transcripts, even a decade after the fact!



How to keep track of legislative activity about open access?
by oneiros27

Recently in the federal register, there were two calls for comments about access to data and research from federally funded research:

http://federalregister.gov/a/2011-28623 [federalregister.gov] http://federalregister.gov/a/2011-28621 [federalregister.gov]

I didn't hear about these until ~4 weeks after the original announcement, and with the holidays, it was too late to try to get the societies I'm involved with to prepare and vote on official statements. Are there any places where people can get/post notices of these sorts of things so that we can stay informed and try to help influence policies?

CM: The Federal Register is getting a lot better now that it is a much more open system. The idea of "Federal Register 2.0" was a paper I wrote for the Obama transition, so it is an issue I've tracked pretty closely and frankly, I've been amazed at how much better it is now. What they did is instead of selling the raw data feed for the Federal Register for $17,000/year, they went from SGML to XML and then released the data in bulk for free. A few guys out in San Francisco were looking for something to do to enter a contest and they took that bulk data and dreamed up GovPulse.US. That was such a better version of the Federal Register that the Office of the Federal Register switched the official site over to their open source platform. My point is the tools are there to do better notification mechanisms, and I'm sure the government would welcome somebody grabbing the GovPulse.US code out of Github and making it even better.

That's the technical answer. But, the substantive answer is that there is a huge boatload of stuff in the Federal Register and it is pretty hard to figure out what to pay attention to. I also missed that particular call for comment, and I've even missed several Requests for Information coming out of places I try and pay attention to, like the White House's Office of Science and Technology Policy. And, I do this stuff full-time! Perhaps better targeted notification mechanisms are the answer. Maybe it is a social media solution, where you pay attention to things your friends are paying attention to. I hope the answer is not that the only way to pay attention is to be employed with a beltway bandit which can afford hundreds of minions that do nothing but pay attention to Washington. Indeed, there are some very fancy for-pay services from folks like Congressional Quarterly and Bloomberg that cost an arm and a leg, but I can't help but think there has to be a better way that is also open.



What do you think of corporate partnerships?
by mhh5

I'd like to know what you think about corporate partnerships in the process to get public data released. (I'm not sure if Google Patents existed before the USPTO released its databases.) Do corporations that get involved in the process tend to make the process better without question, or are there tradeoffs in some areas because the corporations always want to help but then try to retain a proprietary version of the data for themselves?

CM: The theory is that the government gets some kind of valuable service (like digitization) that the government wouldn't get otherwise so it is a "win-win." But, the reality is all too often the government gets snookered and what we do is give some corporation exclusive access to some pot of data and the government doesn't get much of anything. The deal between Amazon and the National Archives was a good example of that kind of a private fence around the public domain. With a help from Boing Boing, I started systematically purchasing those public domain videos and re-releasing them in the wild. I have no problem with Amazon selling public domain video, I just hate it when they get a de facto or a contractual exclusive. (My testimony before Congress on this subject is here.)

There are lots of other examples of government getting snookered. For example, the Government Accountability Office let Thomson West get access to 60 million or so pages of federal legislative histories. At great cost to the government, they were all packed up and dispatched to West which digitized them all and then sent them back to the government. West now sells access to his amazing database. What did the government get for it's trouble? A few logins for GAO staffers. Even members of Congress need to pay to access the database! (We have an interesting paper trail on this issue.)

I'm glad you brought up the Google Patent system because I was personally involved in making that happen and I can tell you that this one is totally legit. Jon Orwant is the lead developer on this for Google and I played a small part in helping convince the White House and the Patent Office they ought to give Jon access to their data (the heavy lifting on that deal was by Beth Noveck who was the Deputy CTO at the time). Google makes all the data they got from the Patent Office available for bulk access with no strings attached. I can vouch for that because I did a mirror of their system. Last I heard Google was sending out anywhere from 1 to 10 terabytes of data PER DAY to external sources and even normally very critical folks who work in this arena have been really happy.

The big problem in the Patent Office is their computing infrastructure is a real catastrophe. Their power plant is over 95% capacity (e.g., plug in a computer, bring the building down!) and even though the Under Secretary knew that selling DVD subscriptions was silly, he wasn't able to switch over to an FTP service. He cut the deal with Google Patent and it worked out well for the government, for Google, and for everybody else.

What's the difference between the Google deal and the Amazon deal? In the case of the Amazon and GAO/West deals, the government lawyers did all the negotiating and they were totally outsmarted by some sharks in industry. But, when government has people like Under Secretary Kappos and Beth Noveck doing the negotiating, these things can work out just fine. The key is government should partner with people who want to do public service, not people who want to service the public.

Encouraging Governments?
by theNAM666

In a city such as Nashville, things as basic as business ownership and property records are not available online. In states such as New Jersey, public records such as basic corporate filings (officers, operating address/address for service of process) are accessible only for a fee.

What concrete actions can citizens confronting such situations, take to encourage accessibility and accountability?

CM: I find you need a carrot and a stick to make this stuff happen, especially at the local level. Folks like Everyblock.Com and CodeForAmerica.Org have done great working prying some of these databases loose, but there is still lots to do.

The first thing you should do is pick up the phone (or pick up your email client) and write/call the people who run the system. Ask them if you can have access to the data. Sometimes, it is as simple as that.

Other times, though, it isn't quite as simple since they want the money (or they want the control or they think this should be done by "private industry" by which they mean some buddy who is a contractor). The nice thing about any government system is somebody usually has oversight responsibilities. So, the next step is to find a city council member of state legislator who has oversight on the agency in question and ask them.

Again, life isn't usually that simple, but sometimes you win! If you can't get anywhere that way, what I usually end up doing is basically competing with the government system. Build a proxy system like RECAPtheLaw.Org did to recycle paid documents. Or, get a sponsor and buy a reasonable number of docs and build a web site that looks like it is going to be a real production system.

Then, go back again and ask. Maybe if you have eyeballs or at least have a nice web site, that is enough to get the government moving. But, if that doesn't do the job, you may have no choice but to compete with them for real, which of course requires a big commitment in time and energy and not everybody can do that. I know in the case of the Patent Office, I started pestering them in 1993, including several times when I spent 6-figure sums purchasing their data, and it still took until 2011 to crack that nut.

The real trick is focus/obsession. Pick one thing you really care about and just keep pestering them until you crack it open. If you're surfing from one opengov problem to another, showing up for a 1-day hackathon then moving on to something else, you're not going to get anywhere. Pick something real and make it your thing.



Privately Owned, Copyrighted Law
by AdamnSelene

I think I have read that the law itself cannot be copyrighted and it should be possible to make it available available to everyone. But as a techie who drafts standards and specifications, I was wondering about how far this goes--especially since Congress recently proposed enacting some of our standards into law. (They decided not to, but they read some parts into the committee records as they debated.) Can you still accomplish your project if a governmental body adopts (or considers adopting) a privately owned, copyrighted technical reference manual or set of safety standards as administrative law (or regulations that carry the force of law)? Or would such obstacles keep you from being able to digitize all of the government's laws (and archives of proposed laws)?

CM: The idea that the law has no copyright is a fundamental part of the American system of government. That applies to states and municipalities as well. The basic decision is Wheaton v. Peters from 1834 but that decision has been reaffirmed over and over. The law is sacred in the American system. You can't have equal protection under the law or due process under the law if there is a poll tax on access to justice.

When we get to a privately developed standards however, it turns into a very interesting issue. The basic mechanism is called Incorporation by Reference. The government will take some external document (such as a model building code) and incorporate the entire text to make it the law of the land. A guy named Peter Veeck was responsible for a landmark decision in 2002 when he published the Texas Building Code which was an incorporation of a privately-developed and very expensive model code. The court ruled that while the model code had copyright, the law of the land did not.

Based on the Veeck decision, my group went and posted many of the public safety codes enacted by the states. We started by purchasing model codes, finding the incorporating legislation, and concatenating the two pieces together and posting the resulting PDFs. More recently, we've done some extensive reworking of the California public safety codes, known as Title 24, converting the entire text into valid XHTML, recoding the graphics as SVG graphics, the formulas as MathML, and regenerating the PDF documents as nicely typeset documents instead of low-quality scans. You can see this work on the web but it is also available as Google Code project.

The federal government also uses this mechanism intensively, with over 2,000 standards incorporated into the Code of Federal Regulations. This is non-trivial stuff, things like all the OSHA safety regulations. The issue was recently considered by a federal group called the Administrative Conference of the U.S. which basically rolled over and endorsed the idea that it is ok for important parts of the law to cost money. (Read EFF's protest letter if you want a good critique of what they did.)

I'm not necessarily saying that government should be able to appropriate any privately-developed standard and make it available. And, I'm not necessarily saying you want OSHA bureaucrats drafting the standards. But, I do think the big standards establishment and the government regulators have cut a deal that results in the law not being available and the costs forked off on private citizens and small business with extortionate monopoly prices. I just paid $847 for a 48-page safety standard from Underwriters Labs and $60 for 2-page safety standard from the Society of Automotive Engineers, both of which are mandated by law in the CFR. They do need money to run their operations, but let me just point out that in 2009 the 501(c)(3) nonprofit Underwriters Labs paid their CEO $2,138,984 and the nonprofit SAE paid their CEO $412,578.

Ancestry.com
by An Anonymous Reader

What is your opinion about websites like Ancestry.com which make use of public records and charge a subscription fee for access? What is the incentive for the government to migrate old documents into digital form when services like these exist? Do you think Ancestry.com should be a 501(c)(3)?

CM: I'm not a big fan of for-profit corporations that have a business model of monetizing the public domain. I'm fine if they exist and fine if they make billions of dollars, but if they are the only game in town they've taken something that belongs to all of us and and turned it into their private property.

The government got snookered on the Ancestry.Com deal. They could have insisted that the raw data be available in bulk for anybody else to use. The folks that approach the government to cut these sweetheart deals argue that is unreasonable because they need a "return on investment" and the argue that if they don't get the return on investment they won't do the deal (and by extension nobody else will do the deal).

But, government can argue much harder! For example, instead of negotiating some exclusive thing with Ancestry.Com, how come they didn't ask the Internet Archive to grab the data? Or put together something creative with a couple of foundations that would pay for the digitization in return for the kind of payback the foundations like to see (e.g., good press, photo opportunity with the President, or other tools of the trade)?

You asked if Ancestry.Com should be a 501(c)(3)? Not all nonprofits do something that I think which should be an essential part of their mission, which is allow others to compete with them. I believe providing open access to all data ought to be a precondition to getting nonprofit status (an idea that Gil Elbaz has been pushing for quite some time). A good example of a nonprofit that builds walls is Guidestar which wants to be the place where you go for all your nonprofit information. The IRS should be making all Form 990 returns of nonprofits available in bulk for anybody to use, which would knock the bottom out of Guidestar's attempts to build walls and force them to stay innovative and provide value.

Pacer Problems
by onyxruby

How much difficulty do you anticipate in getting and publishing records in Pacer? If there's one system that should be free it the decisions that our courts make and yet you are charged by the page just to view the results. Are you concerned about a court taking an unkind view on your archiving what is in Pacer?

CM: PACER is an abomination. Do they take a dim view of our efforts? Well, the Administrative Office of the U.S. Courts reacted so strongly to our efforts to make their data available that they called the FBI on Aaron Swartz and cancelled the only meaningful public access system they had, which consisted of one terminal in each of 17 public libraries around the country. In this era of rapidly decreasing costs, they just boosted their access charges from 8 cents a page to 10 cents a page, arguing that this is a bargain compared to 25 cents a page for a copy machine.

What I find so disturbing about PACER is that when we did get 20 million pages of docs, we were able to conduct a comprehensive analysis of privacy violations in the courts, an analysis that led to a nice thank-you letter from the Judicial Conference and changes in their privacy rules. In other words, only when public interest groups got access to the data did we begin to address privacy issues. Public access is not just about pro se prisoners defending themselves from a jail cell, which is the view of many in the Administrative Office of the Courts. Public access is about attempts like ours (and many other folks) to make our system of justice function better. When we say we are "an empire of laws not a nation of men" that means we write down what we are doing in our courts so that it is no longer the arbitrary decisions of individuals. The paper trail is there so we can make sure the system is functioning properly. When you limit that access to those that only have a Gold Card, you pervert democracy and you pervert justice.

This principle that access to justice shouldn't hide behind a cash register goes back to the Greeks. Theseus in Euripedes' Suppliants said "when there are no public laws, one man holds power by keeping the law all for himself, and there is no more equality. But when the laws are written, the weak man and the rich man have equal justice." The PACER system is justice for the rich man.

Steve Schultze and the team at Princeton did a lot of the heavy lifting on this issue, including the very nice RECAPtheLaw.Org system they built. They've also done a lot of financial analysis that shows that the courts are not only recovering their costs for operating the expensive PACER system, they're making a huge profit (to the tune of $100 million/year) and using their excess profits to do things like buy big-screen TVs in direct violation of the E-Government act.

The basic problem on PACER is the Judicial Conference has delegated the issue to a few techie judges who think what they've built is something great. But, PACER is a hairball of bad PERL code and the result has not served the judges, the bar, or the American people very well. My only hope is that eventually, the Judicial Conference will see that their information technology is 30 years behind the rest of the Internet and feel ashamed at the travesty they have wrought. Until then, we have RECAP.

If you're interested in the issue, a couple of resources to look at are the PACER paper trail and a bit of a rant that I delivered at the Gov 2.0 summit.



How to visualize opened data?
by hardwarejunkie9

The amount of information you're trying to free is entirely staggering and consists, largely, of tables of numbers. These numbers are incredibly significant, but people generally can't see them.

After you free all of this information and make it available to the public (as it should be), then what? What do you expect for the public to do with these numbers? Tables of information are not nearly as useful as graphs. This data needs to be seen, but, more importantly, it needs to be understood.

Do you have any ideas for how to disseminate this information? Perhaps a team-up with someone like gapminder.org's Hans Rosling might be particularly valuable for all of us.

CM: Actually, most of the data I'm looking at is not tables of numbers, it is video, images, textual documents, technical papers, maps, and books.

But, I definitely get what you're saying and there are a lot of numbers. For example, the IRS Form 990s should be structured data instead of PDF documents, so extracting the data from the mass of paper is the initial challenge. There are lots of other examples of this kind of initial extraction, getting what were printed paper docs into structured data. There are some interesting tools, such as OCRopus which does layout analysis, but there needs to be much more. One of the reason we called for a Federal Scanning Commission is that we think there is a lot of directed R&D that could not only scale up mass digitization but could also work on the important value-added of extraction of structured data and handling some of the tricky issues like detecting the presence of Social Security Numbers.

Once you have the data, as you say, then what? I'm a big fan of the idea that the government starts by providing bulk data, then they provide an API, and then maybe they also build web sites and apps and other things along with everybody else out there. That's a 3-part hierarchy that Ed Felten and some of his students developed and it should be a law that applies to all government information systems that are externally facing.

The issue here is that all too often people look at a problem like "digitize all government information" and they want to see the whole stack of the solution from one place. But, I think you can do a layered approach and count on the fact that there is always somebody smarter out there and our job is to reduce the barriers to entry. So, how would I visualize the data? I have no idea, but I'd make damned sure that folks like Martin Wattenberg at Many Eyes and Hans Rosling at Gapminder knew the data was out there and then I'd sit back and be amazed at whatever they come up with. How's that for pushing the problem downstream?



Why is data access so hard?
by CanHasDIY

Can you provide any explanation as to why it is so difficult and cost-prohibitive to obtain records from the government, especially considering the abundance of laws requiring government compliance with requests for information (AKA "Sunshine Laws")?

Is it simply a matter of government employee ineptitude, or have you found evidence of a more nefarious rationale?

CM: I get that question a lot. Why would a member of Congress take deliberate steps to stop public hearings from being available? Why would a court administrator deliberately restrict access to public court documents? Usually the answer is, as Heinlein said, "you have attributed conditions to villainy that simply result from stupidity." When I'm explaining why something is so broken on a big government system, my usual answer is that there are a lot of people still stuck in the 1970s and 1980s, when information dissemination was really, really hard and it took men in white lab coats and computers the size of freight trains to process data. In other words, the problem with a lot of folks who are government gatekeepers is they just don't get the Internet and they don't get computers. In fact, usually when some senior bureaucrat is throwing stones at me, you can find younger staffers working for them rolling their eyes.

That's an optimistic view, and if I'm right things will get better. But, I'm often wrong on my predictions of the future. (I was the guy who saw TimBL demo the web in 1992 and thought to myself "interesting, but it won't scale.")

But, there is also some more nefarious stuff happening, often the accumulation of power by being able to cut exclusive deals with contractor buddies. If your life in government consists of receiving emissaries from Lockheed Martin, maybe you think you're making everybody happy by letting them build you a $1 billion computer system. Often, you think your problems are so unique that the $1 billion solution is the only answer.

And, in some cases, as we've seen from numerous GAO reports, Inspector General reports, Congressional hearings, and newspaper articles, there are some really evil people out there who think the public domain and the government is their personal business opportunity. Looting the federal government is the kind of civic crime that ranks right up there in my book with stealing cookies from Girl Scouts and selling fake medicines to sick people.



Who is the worst?
by TheBrez

Which government agency is the worst to get information from?

CM: I don't know who the worst are (there's a lot of competition for that slot), but the ones that piss me off the most are the ones that should know better.

Public.Resource.Org is a really small operation. I'm the only staff member. My part-time sysadmin is @mdkail who is pretty busy with his day job as CIO at NetFlix. My ISP is Jim Martin and his team at ISC who are kind of busy running the F-Root. My office net is supported by the amazing systems team at O'Reilly which rents me office space at below-market rates.

I'll grant you government would have a tough time getting that kind of help. But, I'm a one-man shop and we run the 4th most popular U.S. government video channel on YouTube, we're the source for a lot of the on-line presence of the U.S. Court of Appeals, and we've supported efforts for the U.S. Congress, the White House, and the National Archives. If we can do this out of Northern California, couldn't the vast resources of the federal government in Washington, D.C. do a whole lot better than they're doing now?

For me, my current bete noir is the U.S. Congress. We got half-way through processing their archives of video from congressional hearings, publishing about 31 terabytes of data. Then, a couple of staffers decided this was a bad idea and pulled the rug out from under us. They actually decided it was a bad idea to publish video from public congressional hearings.

Like any agency, Congress is a mixed bag. We had tons of support from Darrell Issa, for example, and ran a very successful pilot project for him for a year. We talked to all sorts of people on committees and in the various agencies that support the Congress. But, at the end of the day, a couple of staff members were able to decide that the public archive shouldn't be public and they terminated our project. (If you have some time, you might like to read our rather surreal paper trail.)

So, rather than the worst, I think we need to look for the most shameful, the ones that have the privilege and the power and could easily do better. I know it is in vogue to throw stones at government in general and Washington in particular, but there are times when government can be so useful and so awe inspiring it takes your breath away. Government can be that shining city on the hill but we all have to take an active part in our government to keep those lights shining bright.

PerlJedi writes "The future of innovation in America is the Maker movement. A new project being announced on the Makezine blog aims to bring low cost innovation and alternative manufacturing processes to schools in hopes of turbo-charging the next generation of inventors in the U.S. From the announcement: 'The new Makerspace program, developed by Dale Dougherty of MAKE and Dr. Saul Griffith of Otherlab, will integrate online tools for design and collaboration with low-cost options for physical workspaces where students may access educational support to gain practical hands-on experience with new technologies and innovative processes to design and build projects. The program has a goal of reaching 1000 high schools over four years, starting with a pilot program of 10 high schools in California during the 2012-2013 school year.'"

tomhudson writes "AllThingsD is reporting that Facebook has agreed to share users' private data with Politico. Quoting: 'Most notably, the Facebook-Politico data set will include Facebook users' private status messages and comments. Every post and comment — both public and private — by a U.S. user that mentions a presidential candidate's name will be fed through a sentiment analysis tool.' Yes, they claim it will be anonymized, but we've seen that doesn't really work in real life."

If you've ever tried to look up public records online, you may have run into byzantine sign-up procedures, proprietary formats, charges just to view what are ostensibly public documents, and generally the sense that you're in a snooty library with closed stacks. Carl Malamud of Public.Resource.Org has for years been forging a path through the grey goo of U.S. government data, helping to publicize the need for accessible digital archives — not just awkward, fee-per-page access. (Mother Jones calls him a "badass.") Malamud has (with help) been making it easier to get to the huge swathes of data in government sources like PACER, EDGAR, and the U.S. Patent Office. He's got a new initiative now to establish a "Federal Scanning Commission," the task of which would be to assess the scope and outcomes of a large-scale effort to actually digitize and make available online as much as practical of the vast holdings of the U.S. government. ("If we were able to put a man on the moon, why can't we launch the Library of Congress into cyberspace?") Ask Malamud below questions about his plans and challenges in disseminating public information. (But please, post unrelated questions separately, lest ye be modded down.)

Michael J. Ross writes "Web designers and developers alike are increasingly enthused about the capabilities offered by HTML5, which is generally considered the combination of the latest version of the Web's primary markup language and its related technologies. Consequently, publishers have rushed to market a wide variety of books that purport to explore the inner mysteries of HTML5, even as the standards — and how browsers implement them — are still in flux. In characteristic fashion, O'Reilly Media took the time to wait for some of the dust to settle, and attempted to create a resource more approachable and solid than those thrown together quickly. The final result is Head First HTML5 Programming." Read on for the rest of Michael's review. Head First HTML5 Programming author Eric Freeman and Elisabeth Robson pages 608 pages publisher O'Reilly Media rating 7/10 reviewer Michael J. Ross ISBN 978-1449390549 summary A heavily-illustrated introduction to building web pages and web apps with HTML5. The release of this book is quite timely, given the current developments in web technologies. As one of the underpinning components, HyperText Markup Language (HTML) has undergone tremendous change during its two-decade history — with new element names and attributes being added to try to keep up with the latest multimedia formats, design techniques, and other factors in the Internet's evolution. Even though this newest major revision, HTML5, is still not completely supported by most browsers, much of its capabilities are already available, to one extent or another. Also, forward-thinking designers and developers are not waiting for the final blessing by the W3C to begin learning what they can do with it now and in the future.

This book was written by Eric Freeman and Elisabeth Robson, both of whom possess a lot of experience with the subject matter. This title was released on 18 October 2011, under the ISBN 978-1449390549. Its considerable size, 608 pages, is partly due to the extensive use of humorous pictures, actors, scenarios, clever drawings, and a generous use of whitespace — characteristic of other titles in the Head First series. At first glance, these elements might seem like cartoonish gimmicks, meant only to boost the page count or keep graphics employees busy. Actually, these methods are intended to help readers retain the new knowledge, and make the learning process more pleasant. This approach is covered in more detail in the book's introduction.

The material is organized into ten chapters, followed by an appendix. The only technical prerequisite, for prospective readers to get the most out of the book, is a solid understanding of HTML and CSS. Some JavaScript knowledge would be helpful, but is not necessary. On the publisher's page, visitors will find more details about the book, a couple reader reviews, some brief author bios, links to purchase the print and electronic versions (PDF is the only format), and the reported errata (of which there are eight, as of this writing). The example code and other files for the book can be obtained from WickedlySmart.

The first chapter introduces HTML5, at a high level and a fast pace, focusing on the new features that it offers, such as the new JavaScript APIs: embedded video and audio (without the use of plug-ins), client-side data storage, off-line web apps and caching, geolocation, canvases, sockets, Web Workers, and advanced capabilities for forms and drag-and-drop. JavaScript is also introduced, with some simple example code. Much more detail is presented in the subsequent chapter. The only confusing point is, on page 53, when the authors state that there are three different ways to add JavaScript code to a web page, but the figure shows four permutations. The third chapter explains how to work with events and handlers, using a simple music playlist app to illustrate the ideas. In the subsequent chapter, functions and objects are explored in much greater detail, and the presentation is quite methodical and comprehensible.

With Chapter 5, "Geolocation," the authors shift from establishing a foundation of basic JavaScript knowledge, to showing how to apply it for constructing web applications. In the case of geolocation, readers are stepped through the process of building a simple web app that detects the user's current position, displays it on a Google map, and tracks any changes in the position. The next chapter shows how to make one's code work with web services, using the JavaScript communication APIs, and why JSONP bypasses the problems with XMLHttpRequest requests being blocked for security reasons by the JavaScript same-origin policy. The presentation is solid, except for the claim on page 257 that the callback receives an object, when actually it receives an array of objects. Chapter 7 explicates the new canvas element, which offers capabilities encroaching upon the realm of Adobe's Flash. The next chapter, titled "Video," is a logical continuation of the discussion on the canvas element, because the latter allows one to do a lot more with the video API. The authors demonstrate how to do that, after discussing the different video formats and techniques for writing robust HTML to accommodate as many brands and versions of browsers as possible.

HTML5 has taken the venerable browser cookie, and extended its storage capacity tremendously, in the form of the local storage API (a.k.a. "Web Storage"), which is addressed in the penultimate chapter. Sadly, no troubleshooting information is provided in case the reader finds that the example code does not work in Firefox, even when using a web server (i.e., "http://" instead of "file://") — and instead fails quietly with an error message "localStorage is null" in the JavaScript error console. (For those who are interested, one source of the problem is when the Firefox configuration preference "dom.storage.enabled" has somehow been set to "false.") The tenth and final chapter, "Web Workers," shows how to utilize multithreading in JavaScript code to improve its performance, when possible. Readers using Firefox 8.0 (the latest version as of this writing) will likely find that the example code does not work on a localhost, throwing a "Could not get domain" error message, as a result of a known bug. The appendix briefly covers ten additional topics not discussed in the chapters, including Modernizr, the audio element, jQuery, XHTML, SVG, and more.

With a book this size, it is inevitable that it will contain various blemishes. Some of them are a result of the book production process: In the text, JavaScript tokens are not distinguished from English words in any manner (such as a monospace font or bolding), which can trip up the reader. On some of the two-page spreads, the portions of the images and arrows get lost in the book's gutter. In the many illustrations involving one or more persons saying something, their statements are shown in thought bubbles, which is mildly but invariably disconcerting. Other flaws are results of the writing and/or editing: Commas are oftentimes used where semicolons or periods were called for, or just missing altogether — especially in the mock interviews. Sometimes the conversational style — characteristic of the Head First series — becomes a bit too casual, and in some places the authors are trying too hard, such as the repeated use of "skool."

The example code is generally of good quality, but not always consistent; for instance, is employed in some places, but elsewhere — leaving the reader to wonder why. Also, there's at least one case of (incorrect) curly quotes in the code (page 454). It is helpful to have the example code available for download, although it would have been decidedly better had the root directory of the archive file contain an index.html pointing to all of the included apps, so readers could bookmark that single starting point, rather than having to modify their browser's URL each time. In addition, it is oftentimes not obvious as to which chapter subdirectory corresponds to any given location in the book.

However, the main problem with this book is the sloppy editing, evidenced by the notably high number of errata: "pin point" (page xiv), "test editor" (xxii; should read "text editor"), "iPhone" (xxv; should read "HTML5" or something similar), "folks that" (xxxi; should read "folks who"), "get [a] sense" (1), "on the page 2" (3), "can you get a long way" (21), "assign it [the] empty" (26), "you can also thrown in" (40), "its got" (46), "Your job is the act like" (57), "lets concentrate" (58), "get [the] length" (68), "Go ahead an open up" (90), "What you can" (129; should read "What can you"), "a object" (142), "an new object" (147), "to to" (158), "you [are] saying" (158), "users location" (166), "south" (167), "three properties" (177; should read "four properties"), "google" (186), "including [the] last two methods" (192), "give it a try it" (218), "will use" (220; should read "we'll use"), "take a 90 milliseconds" (221), "the this code" (249), "with out with" (268), "HTML =" (271; should read "HTML5 ="), and "an drawable region" (285). These are just the errata found in the first half of the book. Fortunately, they are in the narrative, and not the example code, which would have had a much more negative impact upon the reader.

This book is definitely an introductory tutorial, and by no means a reference. Not all of the new HTML5 elements are covered, nor is CSS3 provided full coverage. The repetition of concepts may aggravate experienced or impatient programmers: For people with some experience with these technologies, and for people who readily glean information from technical books upon first exposure to the given concept, the frequent repetition in this book would border on tiresome, if it weren't presented so pleasantly, oftentimes with humor. On the other hand, the Head First books are predicated on the approach of presenting information in different formats, to maximize learning. Any newbie should appreciate this volume's clear explanations, even if they are presented multiple times, but differently. Also, there is plenty of testing of one's knowledge, to reinforce what has been learned.

Head First HTML5 Programming is an entertaining yet instructive and compelling tutorial on how beginners can learn to use many of the advanced new techniques in HTML, CSS, and JavaScript.

Michael J. Ross is a freelance web developer and writer.

You can purchase Head First HTML5 Programming from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

mikejuk writes "Today we celebrate Dennis Ritchie Day, an idea proposed by Tim O'Reilly. Ritchie, who died earlier this month, made contributions to computing that are so deeply woven into the fabric that they impact us all. We now have to remark on the elephant in the room. If Dennis Ritchie hadn't died just after Steve Jobs, there would probably have been no suggestion of a day to mark his achievements. We have to admit that it is largely a response to the perhaps over-reaction to Steve Jobs which highlighted the inequality in the public recognition of the people who really make their world work."

Michael J. Ross writes "For decades, programmers have written computer code in one language, and then programmatically translated that code into another, lower-level form (typically machine code that can be run directly by a microprocessor, or some sort of bytecode that can be interpreted by a virtual machine). For instance, source code written in C or C++ is compiled and assembled into machine code. In web programming, there are emerging languages and other tools for translating code into JavaScript. For instance, Google Web Toolkit allows the programmer to create web apps in Java. The latest addition to this category is CoffeeScript, a language that can be compiled into JavaScript, and is intended to reduce source code size and clutter by incorporating some of the best operators from other Web scripting languages, particularly Ruby. It is also the topic of a new tutorial, CoffeeScript: Accelerated JavaScript Development." Read on to learn what Michael thinks of this book. CoffeeScript: Accelerated JavaScript Development author Trevor Burnham pages 138 pages publisher Pragmatic Bookshelf rating 5/10 reviewer Michael J. Ross ISBN 978-1934356784 summary A fast-paced tutorial of CoffeeScript. This book is authored by Trevor Burnham, who is credited as one of the early contributors to the project by Jeremy Ashkenas (the creator and project lead of CoffeeScript) in his foreword to the book. Published by Pragmatic Bookshelf on 3 August 2011, under the ISBN 978-1934356784, CoffeeScript: Accelerated JavaScript Development fills only 138 pages, which is certainly a change of pace from the majority of programming tomes now being released. This book's material is grouped into six chapters, plus four appendices — aside from a preface, which introduces CoffeeScript as well as a word game, which is used as the example project throughout the book. Oddly enough, the preface mentions jQuery, but not as one of the well-known attempts to streamline JavaScript code.

The first chapter, "Getting Started," begins by briefly explaining how to install Node and npm (Node Package Manager). These instructions assume that you are following along in a Linux environment or some emulation thereof. They also seem to assume that nothing goes wrong in any of the steps, because no troubleshooting guidance or references are provided. Given the number of moving parts required to get CoffeeScript running, as well as the technical pitfalls that could ensnare a Windows or Mac user, the author should have provided more clear and detailed installation instructions. Also, readers unfamiliar with Linux/Unix may be puzzled by some of the instructions. For instance, page 3 appears to state that the way to check that those two aforesaid packages are on your path, is to simply type in "PATH" (whereas what is needed is "echo $PATH"). From that point forward, the narrative gradually becomes more opaque, with cursory coverage of text editor plug-ins, the "coffee" command line compiler, REPL, "the soak" (an existential chain operator), and the limitations of trying to debug CoffeeScript code. It is quite possible that by the end of this chapter, many readers will decide to not bother trying to learn CoffeeScript, and instead to stick with plain JavaScript, possibly supplemented with jQuery (which is not to say that jQuery code is any easier to read).

In the next three chapters, the author presents the basics of CoffeeScript, including how to: define and use functions and their arguments; test conditionals; throw and catch exceptions; understand variable scoping and context; create arrays using splats; accept input from the console; create objects, arrays, and soaks (in more detail than before); iterate over collections; match patterns; define namespaces using modules; and create prototypes and classes. He makes extensive use of examples, which thankfully are concise (unlike some programming books whose example code span far too many lines, and sometimes even multiple pages — forcing the reader to dig through the code, trying to find the important lines). Also, the brevity of CoffeeScript syntax is undoubtedly a factor. However, his concise style extends to the narrative as well, and will likely cause newbies to have to read the material several times — and even then wonder whether they fully grasp the concepts. It seems that the author understands CoffeeScript extremely well, but is not always able to communicate that knowledge to the reader in a patient and comprehensible manner.

Chapter 5 is a primer on jQuery, and is apparently included in the book so that the example application (the word game) can be made to work in a web browser — since none of the code or narrative (aside from the example app) appears to be related to CoffeeScript. It would have been more efficient to simply point the reader to an online jQuery tutorial, and then present only the CoffeeScript-specific differences — or just briefly explain how to load CoffeeScript files in an HTML file, which could have been done in a sidebar. The last chapter demonstrates how to run CoffeeScript on a web server, utilizing Node.js, and also explores how the lack of threads in JavaScript can impact Node programming. The example project is made multiplayer using Node, Connect, and WebSocket.

The appendices provide answers to the end-of-chapter exercises, alternative methods of running CoffeeScript code, a JavaScript cheat sheet, and a list of a half dozen bibliographic references. This book concludes with a suspiciously-short index, at less than three pages long, which appears to provide only the first or earliest occurrences of the major terms. Consequently, anyone who tries to use this book as a reference work for looking up key terms quickly — or for finding their later occurrences — will likely need to obtain an electronic version of the book, since all e-readers have search functionality. Furthermore, the index is missing some key terms used in the text, such as "function callbacks" and "arbitrary expressions" — heck, it's even missing "expressions," a fundamental concept in any programming language.

Prospective readers who wish to learn more about the book, can visit Pragmatic Bookshelf's page, which offers brief descriptions of the book and its author — as does O'Reilly Media's page. But, as of this writing, only the former makes available an e-book version, pre-publication reader comments, a discussion forum, the example source code used in the book, and a link to a page for reporting errata, which already has more than half a dozen items listed. More are present in the text: "add [a] multiplayer capability" (page xx); a lone ")" missing its matching "(" (in Exercise 6, page 34); "in a lot in functions" (page 107; should read "in a lot of functions"); "a[n] overhead" (page 110); "everyone and their dog is" (page 116).

The author's writing style is sometimes quirky, which in most cases adds a bit of levity, but occasionally leads to the misuse of terms, e.g., array ranges usage described as "fantastical" (page 43). "BDFL" (page xiii) will prove puzzling at first to most readers. On page xvi, the reader is told that JavaScript "contains multitudes." — multitudes of what? And nothing can excuse the groan-inducing "automagically" (page 100).

In terms of the ordering of the topics, one of the most exasperating aspects of this book is the way that many language concepts — such as chained comparisons, and variables being true or false (or "truthy" or "falsy") — are not presented up front, on their own, but mixed in with discussions of other topics, including development of the game application, and even in the answers to the chapter questions (Appendix 1). This makes the book generally unsuitable as a reference, especially when combined with a disappointing index.

One might assume that the modest size of this book is a result of the small size of the language itself. But another factor is surely the pithy presentation style for even some of the most important concepts in the language. Perhaps worst of all — especially from the perspective of someone relatively new to programming — some basic concepts are not addressed, or the example code does not address common use cases. For instance, in CoffeeScript, how does one create a block consisting of multiple lines of code? On page 17, indentation is briefly mentioned, but the sample code shows single-line blocks only. Other important ideas are "saved as an exercise" (which may induce flashbacks to exasperating technical college textbooks). Some readers may conclude that the author didn't want to make the effort of fully describing the language, in a more canonical fashion, which would have resulted in a much longer, but more valuable book.

It is unclear as to how much of the likely mystification and frustration of the average reader will be due to the writing choices made by the author, and how much can be blamed on the sometimes cryptic syntax of CoffeeScript, evident in the discussion of topics such as function binding (Chapter 2) and keywords (e.g., from page 106, "what.x and @x are, of course, equivalent if and only if what is this." Of course!). Readers are told in the introduction that they do not need to be experts in JavaScript to understand the book's material, and can be amateurs (page xviii). But there are several places in the book where intermediate-level knowledge, at a minimum, would be needed. That sort of difficult material may be another point in the CoffeeScript journey where some readers will decide to eschew learning the language.

The production quality of the book is fine, except that the chosen font's ratio of height to width is more than what is usually found in books nowadays; when combined with inadequate spacing among the words within many of the sentences, it makes it difficult for the reader to rapidly scan the material. The e-book version reflects the same minor problem. Yet it makes excellent use of color for syntactically highlighting the code — a feature not seen in the print version.

So if you would like to do some JavaScript programming, but without writing any JavaScript, then one possible place to start your journey is CoffeeScript: Accelerated JavaScript Development. As of this writing, it is the only CoffeeScript book on the market. Yet should the language continue growing in popularity, then more substantial and recommendable books will probably become available.

Michael J. Ross is a freelance web developer and writer.

You can purchase CoffeeScript: Accelerated JavaScript Development from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

OverTheGeicoE writes "The Wall Street Journal reports that Google+ has added 20 million users in just 3 weeks. According to the article, no other site has recorded such high growth in such a short time period. Twitter did something similar once, but in months, not weeks. It's especially surprising considering that access to Google+ is by invitation only. Why is Google+ growing so quickly?" A recent article at O'Reilly Radar offers a possible answer to this, calling Google+ "the rapidly growing seed of a web-wide social backbone," but one that requires openness from Google to really flourish and supplant Facebook. The growth of Google+ will be helped by their acquisition of Fridge, a startup company focused on group sharing. Meanwhile, recruiters and marketers are already eyeballing the growing social network and licking their chops.

theodp writes "Are you or your kid intrigued by Python, but not quite ready to purchase an in-depth O'Reilly book? Zed A. Shaw's 2nd edition of Learn Python The Hard Way may be a friendlier option. Shaw's path to Python programming is simple: 1. Go through each exercise, 2. Type in each sample exactly, 3. Make it run. If $60 for the hardcover is too much to ask, or $15.99 for paperback, you can spend a measly buck for the PDF/ePub download. Still too steep? OK, there's even a free online HTML edition. After completing the 52 exercises, Shaw's concluding Advice From An Old Programmer says, 'Which programming language you learn and use doesn't matter. Do not get sucked into the religion surrounding programming languages as that will only blind you to their true purpose of being your tool for doing interesting things.'"

Michael J. Ross writes "Released during the early days of the Web, in 1995, JavaScript has come a long way: Initially a client-side scripting language typically (mis)used for decorative effects, it is now an essential part of countless major websites. Its increasing capabilities and popularity are due to several factors, including the development of libraries that resolve earlier stumbling blocks that held the language back (such as inconsistencies among the implementations in different vendors' browsers). JavaScript: The Definitive Guide, authored by David Flanagan, was first published just one year later, in 1996, with several significant updates made since then." Read below for the rest of Michael's review JavaScript: The Definitive Guide, 6th Edition author David Flanagan pages 1100 pages publisher O'Reilly Media rating 9/10 reviewer Michael J. Ross ISBN 978-0596805524 summary The most comprehensive treatment of JavaScript yet published. The book is now in its sixth edition, under the ISBN 978-0596805524, and was published on 10 May 2011 by O'Reilly Media (who kindly provided me with a review copy). At 1100 pages, it certainly feels heavier than its advertised 2.6 pounds — but that may only be a side effect of the thought of wading through over a thousand pages of technical explanations and example code. Yet one could argue that the size is justified, considering the amount of information the book conveys, and its obvious aim to be a comprehensive treatment of the language. The material is organized into four parts, including 22 chapters. On the publisher's Web page, visitors will find a brief description, the complete table of contents, a few consumer reviews, reported errata (seven as of this writing, and none confirmed), the example code used in the book, some free content (the first chapter), and links to purchase the print and e-book versions.

The book commences with a multipart introduction, which begins with the sentence "JavaScript is the programming language of the Web." Even though that statement is not true — since there are many other Web programming languages — it does hint at the importance of the language in the mind of the author, and his willingness to put so much effort into creating such a detailed monograph. The introduction is also the first point in the book where one sees the clear demarcation made by the author between core JavaScript (i.e., the language definition, regardless of its runtime environment) and client-side JavaScript (i.e., usage of the language within Web browsers, including the use of libraries). Both areas are covered in great detail in the first two parts of the book, in quasi-tutorial format, while the last two parts cover the same areas, but in a purely reference format.

Specifically, the first part of the book, "Core JavaScript," offers almost a dozen chapters that explicate the basics of the language: its lexical structure; types, values, and variables; expressions and operators; statements; objects; arrays; functions; classes and modules; regular expressions; JavaScript subsets and extensions; and server-side JavaScript. At almost 300 pages, this part alone could form its own volume. The manner in which the author dives into the technical details, and the amount of example code, immediately make it evident that the book is intended for readers who have experience programming, although not necessarily in JavaScript. In fact, some readers — especially newbie programmers — may become frustrated with those places in the narrative where the explanation is not entirely clear. For instance, on page 7, the "points array from above" refers not to any code on that page, but instead refers to an array defined two pages earlier. Fortunately, such stumbling blocks are infrequent. For experienced JavaScript programmers, these chapters could provide a comprehensive review. For readers new to JavaScript, the material may seem overly dry, but the illustrative code should be quite helpful.

The ten chapters that compose the second part of the book, "Client-Side JavaScript," show how to work with the language within a Web browser. This includes learning how to embed JavaScript code in HTML files; differences among browsers and the versions thereof; the security of JavaScript code; the Window object; how to access and manage the elements within the Document Object Model (DOM); scripting CSS styles; events, and methods of handling them; scripting HTTP, and its use in Ajax (reflected in this edition's subtitle, "Activate Your Web Pages"); the jQuery library; techniques for storing data on the user's computer; how to use JavaScript to dynamically create and manipulate graphics, audio, and video content, as well as charts and drawings; and, lastly, the use of several HTML5 APIs. Speaking of that last topic, probably the most significant changes in this edition, versus the previous one, is the coverage of ECMAScript 5, as well as the new objects and methods introduced with HTML5. Naturally, some of these enhancements do not work in any version of Internet Explorer but the most recent, so the author discusses workarounds, if available.

As noted earlier, the third and fourth parts of the book constitute the purely reference material, with the first part focusing on core JavaScript, and the latter on the client-side aspects of the language. Every chapter is organized into a series of entries, each devoted to a particular class or object, ordered alphabetically. For each entry, the reader is given a brief synopsis, description, and in some cases example code and references to other entries. Each class entry also includes information on its properties and methods, where applicable. Each single method entry includes information on its arguments and any return value. The book concludes with what is arguably the longest and possibly most valuable index I have ever seen in a computer book.

There are only a few immediately-evident weaknesses of this book: Firstly, there are some phrases that may be clear to the author, but likely will prove baffling to the typical reader — e.g., "nonlinear cross-reference problem" (page 8) and "the jQuery gives a synopsis of each method" (page 523). Secondly, some of the example HTML code could have been written better, such as the use of an HTML table for defining the layout of a simple form, with labels and fields (page 13). Finally, despite the claims of the marketing copy that this title is suitable as both "an example-driven programmer's guide or a complete desk reference," it would serve better as the latter, and not as a tutorial for learning the language. Clearly, the more comfortable one feels with computer programming — especially JavaScript itself — the more that one could get out of this book.

On the other hand, there are far more pluses than minuses. One of the real strengths of the book is how the author does not hesitate to use (sometimes lengthy) blocks of code, with explanatory comments for almost every line, to clarify the language — as opposed to paragraphs of text, which could have easily doubled the length of the first two parts (which comprise roughly the first two thirds of the book). Also, in conjunction with the narrative and code fragments, the author makes effective use of figures whenever needed — particularly in Chapter 21, in demonstrating how to work with graphics and multimedia content.

Evolving with the language itself, and again brought up to date, JavaScript: The Definitive Guide still retains its crown as the ultimate reference resource for JavaScript programmers.

Michael J. Ross is a freelance website developer and writer.

You can purchase JavaScript: The Definitive Guide, 6th Edition from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

blackbearnh writes "We all know by now that Test Driven Development is a best practice. And so is having 100% of your code reviewed. And 70% unit test coverage. And keeping your CCN complexity numbers below 20. And doing pre-sprint grooming of stories. And a hundred other industry 'best practices' that in isolation seem like a great idea. But at the end of the day, how much time does it leave for developers to be innovative and creative? A piece on O'Reilly Radar argues that excessive process in software development is sucking the life out of passionate developers, all in the name of making sure that 'good code' gets written. 'The underlying feedback loop making this progressively worse is that passionate programmers write great code, but process kills passion. Disaffected programmers write poor code, and poor code makes management add more process in an attempt to "make" their programmers write good code. That just makes morale worse, and so on.'"

An anonymous reader writes "Much has been written about the recent EC2/EBS outage, but Keir Thomas at PC World has a different take: it's shown how much cutting-edge Internet infrastructure relies on Amazon, and we should be grateful. Quoting: 'Amazon is a personification of the spirit of the Internet, which is one of true democracy, access to the means of distribution, and rapid evolution.'" An article at O'Reilly comes to a similarly positive conclusion from a different angle.

An anonymous reader writes "FCC.gov just launched a public beta which aims to take the agency into the future. The $1.35 million site follows the WhiteHouse.gov lead to Drupal. Agency director Steven Van Roekel spoke with O'Reilly about the agency's push for an open platform: 'It's not breakthrough stuff, but it's breakthrough for government.'"

databuff writes "The April 4 launch of the $3 million Heritage Health Prize has been announced by the Heritage Provider Network, a network of doctors. The competition challenges data hackers to build algorithms that predict who will go to the hospital in the next year, so that preventative action can be taken. An algorithm might find that somebody with diabetes, hypertension and high cholesterol is a 90 per cent risk for hospitalization. Knowing this, it might be cheaper for an HMO to enroll them in an exercise program now rather than pay the likely hospital bill. The competition takes the same approach as the $1 million Netflix Prize, but solves a far more significant problem."

macslocum writes "Ambiguity surrounds the real impact of digital book piracy, notes Brian O'Leary in an interview with O'Reilly Radar, but all would be better served if more data was shared and less effort was exerted on futile DRM. 'The publishing industry should be working as hard as we can to develop new and innovative business models that meet the needs of readers. And what those look like could be community-driven. I think of Baen Books, for example, which doesn't put any DRM restrictions on its content but is one of the least pirated book publishers. As to sales, Paulo Coelho is a good example. He mines the piracy data to see if there's a burgeoning interest for his books in a particular country or market. If so, he either works to get his book out in print or translate it in that market.'"

blackbearnh writes "An article over on O'Reilly Radar makes the argument that, just as inexpensive or free software development environments have led to a cornucopia of amazing Web and mobile applications, the plummeting cost of 3D fabrication equipment could enable myriad new physical inventions. The article was prompted by a new Kickstarter project, which if funded will attempt to produce a DIY CNC milling system for under $400. Quoting: 'We're already seeing the cool things that people have started doing with 3D fab at the higher-entry-level cost. Many of them are ending up on Kickstarter themselves, such as an iPhone 4 camera mount that was first prototyped using a 3D printer. Now I'm dying to see what we'll get when anyone can create the ideas stuck in their heads.'"

rheotaxis writes "Many professional web developers have spent years building dynamic, database-driven web applications, but some of us, like myself, want to make the user experience more interactive and instantaneous. The book Building the Realtime User Experience, by Ted Roden, is an introduction to some new techniques making that happen now. New web servers like Cometd and Tornado power solutions that keep HTTP connections open until data is available for the clients requesting it, a technique called 'long-polling.' This means web developers can provide a real-time user experience using HTTP for all sorts of client devices now connecting to the Internet, not just web browsers, but mobile devices as well." Read below for the rest of rheotaxis's review. Building the Realtime User Experience author Ted Roden pages 320 publisher O'Reilly Media rating 8/10 reviewer rheotaxis ISBN 0596806159 summary Shows you how to build realtime user experiences by adding features on your site without making big changes to the existing infrastructure This book covers SUP and PubSubHubbub syndication, messaging with Bayeux protocol and Cometd, and asynchronous Python using Tornado, contrasting these with well-known client-side JavaScript methods. It then demonstrates how long-polling can implement and integrate chat, IM, SMS, and analytics. The last chapter wraps up with an example using all these technologies, a multi-user, real-time, interactive game using geo-location with mobile clients. Ted's writing style is concise and to the point, focused exclusively on the challenges presented and solved in each chapter, including just enough details for experienced programmers to download and setup the software tools being used, including the Google App Engine. The code samples are straight-forward, but be forewarned, it will be easier for readers with some experience building server-side scripts like PHP, Python, or Java, and a database server like MySQL. On the other hand, even if you never used Google App Engine before, that's OK, because Ted covers that in enough detail to get you started quickly. The sample code wasn't yet available on the O'Reilly web site, so you'll need to type in the code samples to try them. Check the O'Reilly errata page for the book to get a head start making the code work. (Full disclosure: I posted some of the errata.) The sample code for Cometd and Tornado ran easily on my laptop (HP 2.2 GHz with Windows Vista), and should be fine on Linux or Mac. Everything you need is open-source and easily downloaded.

The author explains that real-time web development puts the user at the center of all web interactions, and that developers have struggled with solving the push versus pull problem. The pull method requires multiple, periodic queries for updates from server information feeds, something that wastes server CPU and bandwidth when no changes have occurred, and is compounded by the number of different users making these queries. The push method allows the servers to contact the clients when information feeds have been updated, saving CPU and bandwidth.

RSS was designed for easy syndication of information feeds, but it suffers from the limitations of the pull methodology. While several push technologies have been proposed to solve this problem, only Simple Update Protocol (SUP) and PubSubHubbub are covered in detail here. Both of these are demonstrated with PHP code, so they should be easy to implement on hosted web account with PHP and MySQL. The author explains that while SUP isn't a real push methodology, it does address some the CPU and bandwidth issues. PubSubHubbub, a true push methodology when compared to SUP, is described with an equal amount of detail.

Next, the book covers techniques already familiar to JavaScript programmers who have experience building AJAX enabled web pages. Skim the text and glance at the code and diagrams in Chapter 3, if you already have this experience. The subtitle for this chapter is "Widgets in Pseudorealtime", and the key take away from this chapter is that client-side JavaScript can be used with pull or push technologies, depending upon the server-side implementation. If you don't yet have experience with AJAX, then be sure you can follow these code examples, because AJAX will be used in all the other chapters.

Have you ever wished your blog could send live updates to your readers the moment you post them? You'll learn how, using Bayeux protocol, Java, Cometd, and the Jetty web server. The sample code allows you to grasp how long-polling works with modern browsers. Once a client browser opens an HTTP connection to a web server using a POST method, the server leaves this connection open until it has data to deliver to the client. This chapter suggests using Firebug, a Firefox plugin for debugging web applications from the client side, to discover and track long-polling seasons.

Do you need to handle a large amount of incoming data, and then redisplay it on client browsers with almost no delay? Tornado, the Python web server, provides a solution. Tornado was created by FriendFeed, and made open source after being acquired by Facebook. Kudos to Facebook for making Tornado available. Please read Chapter 5 and 6 together, since they both explain how the Tornado server works. The sample code starts with Python threads that cache a Twitter feed, process and filter it, then send it out to web browsers already connected to Tornado using long-polling and asynchronous callbacks. Tornado is then used to implement a peer to peer chat system using long-polling. Again, each client stays connected to the Tornado server until messages are ready to deliver to each chat participant. Taken together, Chapter 5 and 6 lay the groundwork for more advanced Tornado web applications covered later in the book.

This is followed by two chapters using the Google App Engine to support real-time user experiences even though the Google App Engine does not support long-polling. If you have never used Google App Engine before this, don't worry. The author spends 10 pages explaining how to sign up. Then you build an application in the cloud and connect with your IM client, instead of the web browser. You can make your IM server accept commands and respond with information from other web services. The section, "Setting Up an API", gives you a tantalizing glimpse of possibilities explored later in the book. After adding Python code from the next chapter, you have SMS capabilities. Why would you want to do this? Because it allows users to keep informed while they're away from the web, making SMS another part of the real-time user experience.

Once you have implemented and deployed your real-time application, you can add analytics that give you immediate feedback about user interactions with your site. Instead of paying for a service, you can build your own custom web analytics using Tornado and client-side JavaScript. I especially like the authors approach to summarizing all the incoming web usage data into a single, super-simple, HTML template that is immediately updated as web usage changes. It should satisfy your curiosity to watch users interact with your web site in real-time, and you can make it track IM and SMS traffic connecting to your server too. Finally, the last chapter demonstrates how all the know-how you learn from the rest of the book can be combined in new and highly imaginative ways. The author provides all the details you need to setup a location-based, multi-user, real-time, interactive, game played by users with mobile web devices.

This book would be good for anyone that needs to quickly learn how to use Tornado and integrate it with other web services. It's also helpful for people who want to integrate the Google App Engine with other web services. Whether you're going to build a real-time web experience from the ground up, or just add a few more dynamic features to an existing site, the lessons you can learn from this book will help you.

You can purchase Building the Realtime User Experience from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

macslocum writes "John Graham-Cumming is launching a project to finish Charles Babbage's dream and build an Analytical Engine for public display. The goal: inspire future generations of scientists to work on their own 100-year leaps."

Thinkcloud writes "Speculation abounds that Amazon is planning their own storefront for selling Android apps, one in which they, not the developers, will set the price and decide which apps to feature (and which apps to exclude from the store all together). It's a shrewd move and smart strategy for Amazon, though its impact on app sellers is less certain."

macslocum writes "If Twitter is so inclined, the company could turn the new t.co shortening service into a powerful analytics tool that solves the marketing and tracking issues of off-site engagement."

macslocum writes "Map APIs took off in 2005, and during the ensuing years the whole notion of maps has changed. Where once they were slick add-ons, map functionality is now a necessary — and expected — tool. In this piece, Adam DuVander looks at the current state of mapping and he explains how mobile devices, third-party services and ease of use are shaping the map development world."