Slashdot Mirror

NPM Inc. added a feature to JavaScript's package manager this spring letting users type npm audit fix to replace old, insecure project modules -- and the Register asked them how it's going? Since April, according to the company, npm users have run 50 million automatic scans and have deliberately invoked the command 3.1 million times. And they're running 3.4 million security audits a week. Across all audits, 51 per cent found at least one vulnerability and 11 per cent identified a critical vulnerability. In a phone interview with The Register, Adam Baldwin, head of security at NPM, said he didn't have data on how many people are choosing to fix flagged flaws. "But what we've seen from pull requests suggests it's gaining traction," he said.

Incidentally, npm's thinking about security is finding similar expression elsewhere in the industry. Earlier this year, GitHub began alerting developers when their code contains insecure libraries. During a recent media briefing, GitHub's head of platform Sam Lambert said he hoped that the process could be made more automated through the mechanized submission of git pull requests that developers could simply accept to replace flawed code.

Baldwin said NPM might implement something similar, an intervention rather than a simple notification. "Currently it's not proactive policy enforcement," he said. "But it's something we're considering." That would appeal to NPM's growing enterprise constituency. "Enterprises for sure want the compliance and control," said Baldwin. "They want that ability to know the open source they're bringing in is safe or meets a certain set of criteria."
Wednesday NPM added "Report a Vulnerability" buttons to every NPM package web page, and also started checking new passwords against the "Have I Been Pwned?" database to spot already-compromised passwords. "The tools for avoiding problems and fixing them are getting better," writes the Register. But it'd be interesting to hear from Slashdot readers.

How do you feel about code repositories automatically offering replacements for insecure libraries?

There was some trouble last weekend at the world's largest package repository. An anonymous reader quotes the official npm blog: On Saturday, January 6, 2018, we incorrectly removed the user floatdrop and blocked the discovery and download of all 102 of their packages on the public npm Registry. Some of those packages were highly depended on, such as require-from-string, and removal disrupted many users' installations... Within 60 seconds, it became clear that floatdrop was not a spammer -- and that their packages were in heavy use in the npm ecosystem. The staffer notified colleagues and we re-activated the user and began restoring the packages to circulation immediately. Most of the packages were restored quickly, because the restoration was a matter of unsetting the deleted tombstones in our database, while also restoring package data tarballs and package metadata documents. However, during the time between discovery and restoration, other npm users published a number of new packages that used the names of deleted packages. We locked this down once we discovered it, but cleaning up the overpublished packages and inspecting their contents took additional time...

In cases where the npm staff accepts a user's request to delete a package, we publish a replacement package by the same name -- a security placeholder. This both alerts those who had depended on it that the original package is no longer available and prevents others from publishing new code using that package name. At the time of Saturday's incident, however, we did not have a policy to publish placeholders for packages that were deleted if they were spam. This made it possible for other users to publish new versions of eleven of the removed packages. After a thorough examination of the replacement packages' contents, we have confirmed that none was malicious or harmful. Ten were exact replacements of the code that had just been removed, while the eleventh contained strings of text from the Bible -- and its publisher immediately contacted npm to advise us of its publication.
They're now implementing a 24-hour cooldown on republication of any deleted package names -- and are also updating their review process. "As a general rule, the npm Registry is and ought to be immutable, just like other package registries such as RubyGems and crates.io... However, there are legitimate cases for removing a package once it has been published. In a typical week, most of the npm support team's work is devoted to handling user requests for package deletion, which is more common than you might expect. Many people publish test packages then ask to have them deprecated or deleted. There also is a steady flow of requests to remove packages that contain contain private code that users have published inadvertently or inappropriately."

JavaScript's npm package manager reports 4 million users, doubling every year, leading to an interesting question from tech industry analyst James Governor: Just how many developers are there out there? GitHub is very well placed to know, given it's where (so much) of that development happens today. It has telemetry-based numbers, with their own skew of course, but based on usage rather than surveys or estimates. According to GitHub CEO Chris Wanstrath, "We see 20 million professional devs in the world as an estimate, from research companies. Well we have 21 million [active] users -- we can't have more users than the entire industry"...

If Github has 21 million active users, Wanstrath is right that current estimates of the size of the developer population must be far too low... Are we under-counting China, for example, given its firewalls? India continues to crank out developers at an astonishing rate. Meanwhile Africa is set for crazy growth too... You certainly can't just count computer science graduates or software industry employees anymore. These days you can't even be an astronomer without learning code, and that's going to be true of all scientific disciplines.
The analyst attributes the increasing number of developers to "the availability, accessibility and affordability of tools and learning," adding "It's pretty amazing to think that GitHub hit 5 million users in 2012, and is now at 20 million." As for the total number of all developers, he offers his own estimate at the end of the essay. "My wild assed guess would be more like 35 million."

Michael Ross writes "Since its introduction in 1994, JavaScript has largely been utilized within web browsers, which limited JavaScript programmers to client-side development. Yet with the recent introduction of Node.js, those programmers can leverage their skills and experience for server-side efforts. Node.js is an event-based framework for creating network applications — particularly those for the Web. Anyone interested in learning this relatively new technology can begin with one of numerous resources, including Sams Teach Yourself Node.js in 24 Hours." Keep reading for the rest of Michael's review. Sams Teach Yourself Node.js in 24 Hours author George Ornbo pages 464 pages publisher Sams Publishing rating 7/10 reviewer Michael J. Ross ISBN 978-0672335952 summary An introduction to the Node.js framework. This book, authored by George Ornbo, was released by Sams Publishing on 15 September 2012, under the ISBN 978-0672335952. The recent publication date is promising, because Node.js is evolving rapidly, thus gradually obsoleting books written not that long ago. On the publisher's page, visitors will find a brief description of Ornbo's book, a few customer reviews, the table of contents, a sample chapter (the 14th, "A Streaming Twitter Client"), and links to purchase the print and electronic versions of the book. There is also a link to the companion site, which offers some of the same content as Pearson's page, but also has a link to download an archive file containing all the example code, nicely organized.

The book's material spans 464 pages, and is organized (shoehorned) into 24 "hours" (chapters), grouped into six parts. The first two chapters in "Getting Started" explain how the reader can download Node.js, create a "hello world" web server program, install new modules using npm (Node Packaged Modules), search for modules, locate documentation on them, and indicate module dependencies for an application. Unfortunately, the blocks of source code presented in the first examples (Listing 1.1 and Figure 2.2) are not explained in the narrative (until the fifth chapter) or even commented. Readers would likely appreciate some clues as to the nature of http.createServer, req, res, the "underscore" module, etc. — especially at the beginning of their journey. If readers are not expected to understand these details at this point, then they should be told so, to avoid any concerns that such an understanding is assumed in the subsequent chapters. The author does not explain where Node.js is installed or what changes it makes to the terminal's default path variable. On page 18, the term "project folder" is unclear: should the "underscore" module end up in hour02/example01/node_modules, or nodejs/node_modules, or nodejs/node_modules/npm/node_modules? Only later is this (partially) answered.

Chapter 3 demonstrates the complexity that arises from concurrent input/output in networked applications. This material should arguably have been presented at the beginning of the book, to better establish the purpose of Node.js, and the value to the reader of studying it. The next chapter summarizes jQuery and JavaScript callbacks, and then provides a helpful discussion of how Node.js uses the latter. The author contends that the asynchronous paradigm of Node.js is unsuitable for long-running processes, but does not explain why this is true, which would have provided some substantiation for the claim.

The second part of the book, "Basic Websites With Node.js," encompasses four more chapters. The first one discusses how to: create a simple server (using the core HTTP module), examine the response headers (generated for web pages, in different browsers and on the Linux command line), execute 301 redirects, respond to different types of requests (using the URL module), and create a simple client. Oddly, the author does not explain or even mention the sizable JSON output — the first line of which is "{ domain: null," — displayed in the reader's server terminal when the web page pointing to that server is refreshed or when the "curl -I" command is run. The next two chapters cover how to build websites using the Express framework, and are likely the first point where the reader will see some of the real-world complexity of Node.js. The eighth chapter explains how to persist data between calls to the application, including files, environment variables, and MongoDB.

Debugging, testing, and deploying are all critical topics for any application development, and are covered in the third part of the book. The author illustrates three methods of debugging: STDIO, a core module, is a lightweight method for debugging Node.js code; it allows one to output messages to the console, check the value of any variable or literal, and track function calls and responses from third-party services. Node.js provides access to the more powerful debugger of V8 (the Google Chrome JavaScript engine), which supports breakpoints and code stepping. Node Inspector, compatible with WebKit-based browsers, provides all of the above functionality, and more. The next two chapters present several modules that ease the important process of creating full-coverage tests, and demonstrate how to deploy applications to any one of three Node.js-capable cloud hosting providers (Heroku, Cloud Foundry, and Nodester).

Having covered the basics of Node.js, the author begins the fourth part of his book with two chapters that show how to use Socket.IO, WebSockets, and Express to build real-time web applications. These techniques are illustrated in the development of a chat server as well as a nickname management and messaging system. The aforementioned sample chapter extends these techniques further in working with the Twitter API to consume its real-time data, push it to the browser, and show results in a dynamic graph. This section is wrapped up with coverage primarily of JSON — specifically, how to create, consume, and send JSON-structured data.

APIs were addressed briefly in the previous section, but are explored much more deeply in the subsequent five chapters. Readers may initially conclude that the discussion of processes is elementary, but the author then shows how one could utilize that knowledge to interact with Node.js scripts, including detecting script exits and errors, sending signals and arguments to a script, generating child processes if needed, and sending messages among them. In the 18th chapter, the author goes into greater detail about Node.js's Events module, best practices, and how to generate event listeners dynamically. The buffer API may be low-level, but it is essential for storing raw binary data, as opposed to the Unicode-encoded strings that JavaScript uses within a browser. The Buffer and Stream modules are presented with plenty of helpful examples.

The last part of the book addresses miscellaneous topics, starting with CoffeeScript (a JavaScript precompiler). While CoffeeScript affords numerous benefits, it is not clear why it would deserve an entire chapter in a book dedicated to Node.js. In the next chapter, readers learn how to verify their Node.js code, add command-line executables, and then package it all up into portable modules that can be contributed to the npm registry or GitHub. The last two chapters explain how to create and configure middleware using the Connect module, and how to use Backbone.js (a front-end JavaScript framework) in conjunction with Node.js to build browser-based web applications.

Each chapter concludes with a summary (invariably a waste of space), a Q&A section, a workshop comprising quiz questions (with the answers presented immediately below it, for almost instant spoiling), and several exercises for the reader.The index at the end is missing several of the important topics discussed in the text.

The book contains many errata: "EBay" (page 1; should read "eBay"), "OSX" (page 9; presumably Mac OS X), "yaml" (page 15; should read "YAML"), "irc" (19), "led to [a] great deal" (27), "to solve Concurrency" (37), "process" (54; should read "processes"), "try and" (55; should read "try to"), "This goal" (56; should read "The goal"), "how [a] class" (56), "You will [see] the" (62), "status of [a] web server" (70), "javascripts" (77), and "then [the] name" (87). At this point, less than 20 percent into the book, it was clear that the copyeditors had done a sloppy job, so I stopped recording these flaws that should have been caught. Those first four errata suggest that "textese" is even pervading the world of technical publishing. (Strangely, there does not appear to be a place on the publisher's website for reporting errata.)

The production team should have been looking for places to cut down on the heft of the print edition. The "Try It Yourself " sections sometimes duplicate what is found in the regular text nearby — especially in the third and fourth chapters. For instance, three sets of HTML markup are repeated, as well as the surrounding discussion (pages 42 through 47).

In general, the text does not appear to have been carefully scrutinized by technical reviewers and copyeditors. Occasionally the reader is given critical information later than would be optimal, e.g., the "Watch Out" warning on page 18, provided after the reader installs a module. The writing style is noticeably awkward in countless places in the book, including several run-on sentences. (Technical authors should not be bashful in using commas when doing so would help readability.) Also, the text is littered with too many exclamation marks — as if that is going to make any narrative more exciting.

In terms of the production quality of the book, a lay-flat binding would have made it much easier to read when using both hands on the keyboard. Also, in my review copy (kindly provided by the publisher), a disappointingly large number of the pages had small black splotches of ink; fortunately, none made the text unreadable.

On the other hand, Node.js is certainly not a simple subject area, and this book is able to convey a lot of information about it. This book's forte is the extensive use of example code to illustrate the concepts being presented. Incidentally, kudos to the author for inviting the reader to contribute to the Node.js community, such as adding new modules to GitHub or updating the documentation of existing modules. Overall, readers new to Node.js would certainly benefit from working their way through this volume.

Michael J. Ross is a freelance web developer and writer.

You can purchase Sams Teach Yourself Node.js in 24 Hours from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.