Domain: monkey.org
Stories and comments across the archive that link to monkey.org.
Stories · 779
-
Hubble Reinforces Planet Formation Theory
eldavojohn writes "Physorg is running an interesting article on the most recent of Hubble's accomplishments. It has provided us evidence supporting that which Emmanuel Kant proposed over 200 years ago — that planets do indeed form from disks of gas and dust that surround stars. The trick, apparently, was observing many cases where a star's planet forms on the exact same circumstellar disk as the dust and gas. Hubble also aided the researchers in determining the weight of many extrasolar planets. Some had contended that these were not planets but rather brown dwarf stars — which is determined by measuring their weight." Update: 10/12 23:08 GMT by T : That's not the only theory Hubble's recent observation's have supported: read on below for a bit more. somegeekynick writes "Hubble has spotted a bunch of little galaxies, nicknamed Spiderweb, over 10 billion light-years away in the process of merging. This observation supports the so-called 'bottom-up' theory of galaxy formation, according to which smaller clumps of matter collided and merged with each other to form larger galaxies during early stages of the universe's evolution." -
BBC Reports UK-U.S. Terror Plot Foiled
j823777 was one of several readers to point out a BBC report that "A terrorist plot to blow up planes in mid-flight from the UK to the U.S. has been disrupted, Scotland Yard has said. It is thought the plan was to detonate up to three explosive devices smuggled on aircraft in hand luggage. Police have arrested 21 people in the London area after an anti-terrorist operation lasting several months. Security at all airports in the UK has been tightened and delays are reported. MI5 has raised the UK threat level to critical — the highest possible." spo0nman adds a link to the Associated Press's coverage. Update: 08/10 12:57 GMT by T : Several readers have pointed out new restrictions imposed as a result of this plot on passengers' carry-on luggage. In the UK, nearly all possession (including laptop computers) must be carried in the cargo hold; while their rules don't yet go quite as far, U.S. airlines are stepping up their enforcement of carry-on-restrictions, including banning substances like toothpaste. -
AOL Releases Search Logs of 657,427 Users
An anonymous reader writes "AOL has released the search logs of over 650,000 users for research purposes. This looks like it may become a public relations disaster for AOL, as well as a privacy nightmare for the users involved as Michael Arrington of TechCrunch notes: "AOL has released very private data about its users without their permission. While the AOL username has been changed to a random ID number, the ability to analyze all searches by a single user will often lead people to easily determine who the user is, and what they are up to. The data includes personal names, addresses, social security numbers and everything else someone might type into a search box." This is also being covered on The Paradigm Shift and Oh My News." fantomas adds " Looks like they've just taken it down but it's still available on The Pirate Bay; not sure why but some of the academic researchers are going crazy musing the ethical aspects of letting the world know who's searching for how to kill their wives ..." Update: 08/07 21:32 GMT by T : amromousa writes "AOL is now apologizing for the release ..., calling it a "screw-up," which they're upset and angry about." -
Cell Phone Reception Hack
New Breeze writes "Has this ever happened to you? Just when you need to make a phone call, the bars of reception are scant to none. But Graeme, who writes a blog called 'Earth: Mostly Harmless,' gives us hope. Succeeding where most would quit, he chronicled his ingenuity in a post titled 'How I got mobile phone reception where there was no signal.'" Update: 08/01 14:31 GMT by T : Note: Credit for this story belongs to Mike Yamamoto, who wrote it for CNET's News.com. -
Extensive Coverage of Ottawa Linux Symposium 2006
cdlu writes "LWN and NewsForge both extensively covered the goings-on at this year's OLS. NewsForge: day 1, day 2, day 3, and day 4. LWN (subscription required for most): article 1, article 2, article 3, and article 4." I especially enjoyed the description of reverse engineering a USB device from cdlu's coverage of day 3; one day wireless USB devices will really work with out-of-the-box Linux! Update: 07/25 04:57 GMT by T : Eric Preston, who delivered that talk on reverse engineering USB devices, kindly linked to both his slides and the accompanying screenshots. -
Robots Coming to Intro Computer Science Classes
BlueCup writes "Two colleges are hoping to make computer science courses more attractive by including personal robots with the textbooks. Looking to boost enrollment in introductory computer science classes, Microsoft Corp. is working with Bryn Mawr College and Georgia Tech on developing new ways to bring robotics technology into the classroom. Douglas Blank, a computer science professor at Bryn Mawr, said the goal will be to start incorporating the robots in introductory courses at the suburban Philadelphia college next spring. Georgia Tech hopes to start during that term as well. The idea behind the program, Blank said, is to make computer science more hands-on and practical, rather than simply about debugging programs." Update: 07/13 15:52 GMT by T :Professor Blank wrote in with some clarification on one of his statements — read on below.dougblank writes
"Note to self: when talking to the press, don't use complicated technical jargon, like 'debugging' :) I think what I actually said was 'rather than debug a program to make it give the right answer, the students must debug the program to make the robot behave the way they want it to.'
I think many of you will actually like the hardware, software, and curriculum that we are designing. Check out roboteducation.org/ and pyrorobotics.org. The new version of the software will be based on Pyro, Python Robotics. We think of the hardware as something like an iPod on wheels. The software is also being developed with an open source license. This project is not what many of you guess it might be.
The CS1 and CS2 that we are developing won't be watered down, but also won't be just the standard 'intro to programming, using robots.' It's a complete rethinking of the intro courses."
-
Windows Genuine Advantage Makes Few Friends
Readers left more than 800 comments on yesterday's report (based on the say-so of a Windows tech-support provider) that Microsoft may be turning off copies of Windows without WGA installed, as of this fall. (WGA is Microsoft's "Windows Genuine Advantage," a program using software of the same name installed on Windows users' computers intended to verify that the OS is correctly licensed.) Many suggested reasons that this sounds like no more than a rumor, while others took the opportunity to critique WGA as it currently operates on Windows machines, or to describe what they see as opportunities for the users and makers of operating systems other than Windows if (or perhaps when) Microsoft actually does shut down copies of Windows which it suspects are being used out of license. Read on for the Backslash summary to see some of the comments which defined the conversation. Update: 06/30 21:28 GMT by T : A cut-and-paste mishap gave the word "people" one too many Ps; now corrected.With a reminder to "not get silly here," ocbwilg joins several others in casting doubt on the source of the rumor about a mass turn-off:
"A 'front-line tech-support drone' who gets paid $12 an hour to read the support script is somehow going to know what sort of top-secret plans Microsoft has for the next six months? I highly doubt it. It sounds more like the sort of thing that a help desk drone would say to try to persuade a clueless computer user to do things their way.
Then, of course, there's the fact that if you install WGA today on a pirated copy of Windows, all you get is the notification message that pops up. You don't get shut down, and you don't even get cut off from Windows security updates (which are truly the only updates that matter, and even they aren't that good). I find it very difficult to believe that Microsoft is going to go from 'Hey, your copy of Windows doesn't look genuine, but you can still install our security updates' to 'I don't know if your system is pirated or not because you haven't installed WGA, but even if it is a legitimate copy I'm just going to shut you down simply because I have no way of verifying it.' Especially not in the span of 6 months."
Along the same lines, another reader asks "Why are we making all this fuss over what could just be a rumor unwittingly spread by a clueless help desk worker? Since when did help-desk techs become privy to future, unannounced plans for a company, let alone ones as sensitive as this one?"
Besides the dubious source, the sheer scale of such an action convinces reader Willith that it's not going to happen — he promises to eat his hat if it does:"The thing to look it is how this might affect legitimate corporate versions of XP — and by that, I mean VLK versions actually being used in an enterprise setting.
The company for which I work has more than 100,000 copies of XP running in offices on six continents, participating in one of the largest Active Directory installations in the world. Every system's load is tightly controlled and managed, and I can tell you that there are no copies of WGA anywhere on any of those desktops (I've seen the SMS reports). Nor will there ever be.
People say to 'vote with your dollars,' but your dollars, and my dollars, don't matter. Large corporate dollars matter — like the kind of dollars that can outfit a company's world-wide IT needs. WGA has no place on a configuration-controlled and managed enterprise desktop, and MS would never risk upsetting their real customers — corporate Windows & Office sales — to emplace something like this."
Working machines matter to smaller users, too, though, and Kremit mentions reports spotted online of "Dell desktops, valid CDs, and other licensed systems having problems with WGA," writing "When these systems stop working, people are going to flip. To them, this will be akin to the computer crashing and taking their data along with it."
Other readers had some specific gripes about the way WGA currently misfires in their own experience; Jnaujok maintains that it hasn't worked well for him:
"What about my two perfectly legitimately licensed machines at home that fail the 'Windows Genuine Advantage' test every time they update WGA? Considering that one of them is my copy of Advanced Server 2003, I won't be exactly happy when it gets killed this fall. (Hey, I just use it for the mail server program because I can't stand sendmail.)
And I'm just a little bitty guy with one server running. What happens when this hits some company's server farm and they all shut down? How much liability is Microsoft going to have when that happens?
And every time they 'fix' my copy after the new WGA comes out, I have to make manual registry changes. Can you imagine having to do that on a 500 machine server farm?"
Not everyone objects to the idea of harsher treatment for unlicensed copies of Windows; several readers welcomed the idea of more active license revocation by Microsoft as beneficial to the world of free software; WhiteWolf666 described a turn in that direction on Microsoft's part as a "solution to the Linux pricing problem," writing
"35 percent of PC software is pirated. I'm guessing that Windows XP is highly represented in that group (of pirated software; i.e. at least 30% of worldwide Windows installs are not legal). If even 10% of that user base decides to switch to Linux rather than pay the Windows tax, it'll be a substantial marketshare boost.
Reader soren42 lays out what this might mean: "If you suddenly force all the non-legal users off your platform, you're forcing them to use something else. Which means, in turn, more demand for OpenOffice, games on Linux, GAIM, ad infinitum — until there is a more, better, complete Linux end-user software stack to seriously compete with Windows."And the remaining 90%? They might decide that the MSRP cost of Windows is too close to the MSRP of a brand-new dual-core Mac.
I'm thrilled. MS has ridden on piracy marketshare for far too long. I hope they do every thing they possibly can to stamp out software piracy, and I hope they succeed."
Other readers share that sentiment, with a twist: on the basis that remote turn-off really is in the near future of Windows, some, like reader ewhac, say they're through with Microsoft: "I just built a brand new machine, primarily for gaming. Oblivion has been fairly sweet. But it looks like I won't be playing those games anymore — not unless the entire game industry decides to support Linux. ... This is morally and ethically reprehensible, and Microsoft knows it, and apparently doesn't care. Well, I do care. I do not, and shall not, grant consent to Microsoft to remotely snoop on my machine, regardless of their ostensible reasons. If my copy of Windows stops functioning as a result, I will take that as a maliciously incorporated product defect, and respond accordingly."
Most people won't be doing the same, in the eyes of RightSaidFred99, who scoffs "Give me a break, people won't be moving to Linux. They'll find a hack for Windows, they'll buy Windows, or more than likely they'll just buy a new PC that comes with Windows legally bundled. Nobody is moving to Linux because the games aren't there, the thousands of cheesy little Windows applications people love aren't there, it's different (read: scary), and it's a pain in the ass for most joe schmoes to install."
Large corporations running Windows are in a more delicate position. Reader lynx_user_abroad doubts that many corporate users are likely to go seek out either free or illegal alternatives to updated Windows licenses. To the suggestion that many users would do just that, he writes"In a contest between you and them, I'd suspect Microsoft is in the better position to understand the nature of the addiction they have created. And I'd feel safe saying that even if you yourself had succeeded in completely breaking your addiction to Windows, which I suspect you haven't.
Most people, most businesses are so hopelessly addicted to Windows that they literally can't even conceptualize their own survival without it. I'm always amused when I read the latest rant about a Windows vulnerability on an IE-only site, or read about some program manager publishing their 'Linux Strategy' document as a PowerPoint chart.
Think of all the hundreds of thousands of Microsoft Office documents the average business has, or the potential millions of dollars worth of intellectual property and business intelligence those documents represent. Now, even if they have the skill and determination to propose leaving Windows behind, think of the complexity of dealing with a customer base which might not be as skilled, or determined."
Several readers say WGA's phone-home capability doesn't affect the users who Microsoft would be expected to target, anyhow. GenericJoe says "Forget that," writing "I am a legitimate user of Windows. I know I am, because I bought a licensed copy from a reputable dealer. Thus, I figure, I don't need the WGA to tell me if I have a legitimate copy. I do have a legitimate copy. ...And Microsoft doesn't get to know anything else about anything I do, or affect me. The idea that I can be held hostage because I don't want to trust software from Microsoft. Well, that's kind of crazy."
Reader riptide_dot offers similar sentiments, asking "What if I did pay for [Windows] and I don't want the WGA software installed? I'm not allowed to use the software I paid for because I don't want to add on to it? That's like selling me a car and telling me that if I refuse to put a spoiler on the back that I won't be allowed to drive it."
As to actually unauthorized users, Akaihiryuu asserts that
Based on the common-sense arguments made above, unless Microsoft manages to not only flatten wrinkles in WGA as it currently operates, but also convince more users that check-ins with Redmond are close enough to their best interest to be worth accepting, mass turn-offs for Windows XP users seem unlikely. Thanks to the readers whose comments helped inform this discussion, especially those quoted above:"[P]eople who knowingly run illegal copies of Windows won't be affected by this in the slightest. These people have been cracking WGA since it came out, first with Javascript, then later with cracked DLLs. I'm sure there will be a crack for this within 24 hours of it being released (there always has been in the past), and these people will able to get it very easily. The only people that this will affect are
- People who think they have a legal copy of Windows but really don't because whoever they bought it from screwed them, and
- People with legal copies who either don't want to run WGA for some reason, or
- People with legal copies who run WGA and it mistakenly identifies their machine as 'not legit.'"
-
Linux Hackers Reclaim the WRT54G
An anonymous reader writes "The world's most ubiquitous wireless access point is free to run Linux again, thanks to a brilliant hack by db90h, aka Jeremy Collake. No soldering is required, as Collake's 'VxWorks Killer' nixes the WRT54G's VxWorks bootloader and installs a normal Broadcom one, allowing Linux to be installed easily. One distribution small enough for the series five WRT54G's 2MB of Flash and 8MB of RAM is the free DD-WRT project's "micro" edition. It lacks some of the fancier Linux router packages, such as nocat and IPv6, but does support PPPoE, and could be more stable than the VxWorks firmware, which seems to have generated mixed reviews." Update: 06/26 22:52 GMT by T : Note that the project's name is DD-WRT, not (as it was mistakenly rendered) WR-DDT. Check out the DD-WRT project's site. -
BumpTop, Pushing the Desktop Metaphor
Alranor writes "BumpTop is a new way of manipulating your GUI desktop with a graphics pen. Documents can be moved and piled (among other actions) as if they were real pieces of paper on a physical desktop. Simulated real physical interactions, such as documents pushing others out of the way as you move them around, are intended to increase the intuitiveness of the layout tool. Given the messiness of my desks at work and home, I'm not so sure this will work for me, but it's an interesting idea." There's a neat video demo linked from the site (and a "hip-hop overview") if you want to see BumpTop in action; unfortunately for Linux users, BumpTop seems to be Windows-only. As reader idangazit describes it, this is "not just another "me-too" alternative UI; a lot of effort and polish has been put into the (pen-based) interaction, resulting in a very natural way of interacting with collections of information. Less sci-fi than Minority Report, but far more likely to hit a desktop near you in the next few years."
Update: 06/22 16:55 GMT by T : As zdzichu reader points out in the comments below, a visually similar project called lowfat, with an equally impressive video demo, is being developed — with enough sponsorship, lowfat will go open source. -
Beginning Python: From Novice to Professional
nazarijo writes "Python seems to be devouring everything these days, with more and more people using it for serious projects. It's quickly supplanting Perl in some circles, and with good reason. It's a powerful, richly featured language with boatloads of extensions. And, unlike Perl, it's very easy to do complicated things in simple, legible code. Python books are still only a small part of the shelf at your local bookstore when you compare it to the popularity of Perl, but which ones are the gems and which ones are fluff? Having looked at a lot of Python books in the past couple of years, I think that Beginning Python: From Novice to Professional is the one that I'll most recommend to people." Read on for the rest of Jose's review. Beginning Python: From Novice to Professional author Magnus Lie Hetland pages 604 publisher Apress rating 8/10 reviewer Jose Nazario ISBN 159059519X summary Tour the Python language, from basics to advanced modules
Beginning Python is loosely grouped into three main sections. The first deals with Python fundamentals, all the goodies that are inherent to the language and the modules that it ships with. It's surprising to see how rich the language is out of the box, especially when compared to some other scripting languages. The second section would be the chapters covering popular extensions for a variety of services. These include network and web programming, SQL objects, and even GUI programming. And finally the third section is a set of 10 projects in Python, which bring everything together in a concise fashion.
I like this book a lot because it is very clear in its delivery, both the prose and the code examples used, and is consistently Pythonic. The Python language lends itself to a powerful programming style and, unlike Perl, many Python developers I know don't bother with a dozen ways to perform a simple action, they get it done and move on. What you wind up with is clear code that's easily understood by someone new to the language.
Unlike what the title would suggest, Beginning Python isn't only for the first few weeks with the language. The book is large and in depth, and the coverage of material is fantastic in many ways. You get a quick tour of the basics and then you move on to an overview of the language and then its common features. The inclusion of the 10 projects is another benefit to the intermediate user. She can refer back to this book for additional information and pointers from time to time, it wont sit still on her shelf.
That said, there are a few things in the book that I tend to disagree with. For example, the author dissuades you from using destructors in your code, but in my experience they're far more reliable, and a better place to do some cleanup, than he states. A few chapters are also a bit skimpy when they didn't need to be. For example, Chapter 18, which covers packagers like the distutils component from Python, needed to be fleshed out a lot more. This is a powerful feature in Python and sound docs on it should just be there. There's no reason to hold back on something so vital. The section on profiling in Chapter 16 is also a bit thin around the middle when it needn't be. While this seems like a minor point, having a reference to speeding up code (and measuring the improvements) is always nice. And finally, Chapter 17, which covers extending Python, is simply too short for its own good. A more in depth example would have been appreciated.
I have begun recommending this book to people I know that are smart and program in other languages, but aren't very familiar with Python. Many beginners books only take a person so far before they become a useless item on the shelf. This means that he $30 or more that was spent is now gone, so I've grown to be observant of how long I expect a book to be useful. I anticipate the useful shelf life of Beginning Python will be longer than average for most general purpose programming books for a single language. What's more is that it's not a dry reference book. Couple this to a Python cookbook for recipes and you have a two volume "mastering Python" series.
If you've been curious to learn Python and haven't yet found the book that speaks to you clearly, this may be the one. I'm pleased with the quality of the writing, the examples, and the quick pace of the book. While it's nearly 30 chapters in length, most of them are short and focused, making them easily digestible and highly useful. Overall probably the best Python books I've had the good fortune of reading."
You can purchase Beginning Python: From Novice to Professional from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page. -
Intrusion Prevention and Active Response
nazarijo writes "The security world has been taken by storm by intrusion prevention system (IPS) products in the past couple of years. After all, a typical intrusion detection system (IDS) only alerts you that something malicious may have happened, and an IPS reacts to it and can prevent the attack. Action in this scenario is obviously preferred to a passive bystander. Still, the IPS solution space is confusing to many." Read on for the rest of Nazario's review of a book designed to erase that confusion. Intrusion Prevention and Active Response: Deploying Network and Host IPS author Michael Rash, Angela D. Orebaugh, Graham Clark, Becky Pinkard, and Jake Babbin pages 424 publisher Syngress rating 7 reviewer Jose Nazario ISBN 193226647X summary An overview of host- and network-based IPS solutions
The June, 2003, report from Gartner on the death of IDS set off a lot of security industry activity. Everyone was busy trying to either defend the IDS product space, reposition their products as IPS devices, or trying to dismiss the Gartner position. Many security engineers had to suddenly evaluate the IPS products on the market and make purchase and deployment decisions, as well. However, there's been a lack of understanding of this marketspace for some time. If you've been curious about this technology, you may want to look at Intrusion Prevention and Active Response: Deploying Network and Host IPS to help you understand these solutions.
It would have been relatively easy to write a book that simply covered one facet of the IPS product space, such as network IPS systems. However, the authors have chosen to try and write a comprehensive overview of the tools currently available for both the network and the host, as well as ways in which they can be attacked and the scenarios they work in. While the book focuses on open source tools, including the Snort IPS extensions, the techniques apply to closed source, commercial tools as well.
In general I found Intrusion Prevention to be a decent first book on the subject, although a bit unfocused in its delivery. At times it seems to try and bite off more than it can chew, or go off on a tangent for too long (such as the many pages of nmap options), but in general the book does a fair job of delivering its promise. Through it you'll get a good overview of many of the technologies present in the IPS marketspace and what they offer. If you're up to it, you'll even learn a few ways to test the tools and weed out the snake oil vendors.
The book is heavy on actual system output and configuration examples. I like the explicit packet captures and snort rules, I think they go a long way towards illustrating the premise of an IPS system. As is somewhat common with Syngress press books, the formatting is a bit off at times (sometimes it's too wide or slips over the page boundary at the wrong time), but if you can work past that you're rewarded with a useful example.
For host-based IPS solutions, the book covers a number of approaches that aren't always evident as IPS techniques. Various stack protection mechanisms, including LD_PRELOAD techniques like Libsafe, GCC modifications such as StackGuard, and kernel modifications like LIDS, PaX, RBAC and GrSecurity are all described.
By now you can see that the book is pretty Linux and open source centric. This isn't too bad at all, since the basic functionality is present in most of the commercial tools, as well. These can include inline network data modification and reactions or application integrity checking tools. The open source versions, while they sometimes have fewer features, are excellent representatives of this technology.
The book really comes together in chapter 8, 'Deploying Open Source IPS Solutions.' Several vulnerable systems are set up, deployed in a fictitious network, and protected through a variety of IPS solutions which work together to create a layered security model. If the network can detect the attack, it's dropped or modified to remove the offending bits. If the malicious data gets through to the host, the host-level IPS tools remediate the problem. All in all a nice example chapter.
The discussion on how to evade IPS devices was a bit lacking, unfortunately. It seems squeezed in, and doesn't have the same level of detail as other chapters on similar topics. Detailed descriptions of the layer 3, 4 and application layer obfuscation techniques would have been useful to help explain this complex topic.
Before you begin thinking that the authors are entirely gung-ho on IPS technologies, they spend a long time discussing how they can be fooled and how they are fundamentally prone to false positives. This tempered stance is valuable, and they recommend that you take a limited set of functionality from your IDS system and make it reactive in your IPS.
There are only a couple of books that cover IPS technologies to any significant degree, and this appears to be the only one solely devoted to discussing IPS approaches for both the host and network. To that end, the authors have done a pretty good job of introducing the reader to what an IPS can give them, how to evaluate it, and what to expect in the real world. While the book itself has some production and layout problems, the material is worthwhile and will give the reader much-needed advice.
You can purchase Intrusion Prevention and Active Response: Deploying Network and Host IPS from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page. -
File System Forensic Analysis
nazarijo writes "The field of investigative forensics has seen a huge surge in interest lately, with many looking to study it because of shows like CSI or the increasing coverage of computer-related crimes. Some people see a career opportunity there, and are moving toward computer forensics, marrying both law enforcement and investigations with their interest in things digital. Central to this field is the study of data storage and recovery, which requires a deep knowledge of how filesystems work. Brian Carrier's new book File System Forensic Analysis covers this topic with clarity and an uncommon skill." Read on for the rest of Nazario's review. File System Forensic Analysis author Brian Carrier pages 600 publisher Addison Wesley Professional rating 9 reviewer Jose Nazario ISBN 0321268172 summary The standard for digital filesystem forensics
It's easy to think that computer filesystems are relatively simple things. After all, if 'dir' or 'ls' don't show what you're looking for, maybe an undelete program will work. Or will it? To be a decent, trustworthy expert in forensics (a requirement if you plan to participate in any criminal investigations), you'll have to learn how filesystems really operate, how tools like undelete and lazarus work, and how they can be defeated.
Carrier's book isn't a legal book at all, and it doesn't pretend to offer much insight into the law surrounding forensics. Instead it focuses on technical matters, and is sure to be the gold standard in its field. This is important, because it comes at you expecting you to have some knowledge, even if only informal, of what a filesystem contains. With a basic understanding of data structures, you'll get a wealth of information out of this book, and it will be a good reference long after you've first studied it.
File System Forensic Analysis is divided into three sections. These are arranged in the order that you'll want to study them to maximize the benefit you can hope to achieve, namely an understanding of how to examine filesystems for hidden or previously stored data. The first three chapters cover a fundamental series of topics: Digital Investigation Foundations, Computer Foundations, and an introduction to Hard Disk Data Acquisition. While they start at a basic level (e.g. what hexadecimal is), they quickly progress to more developed topics, such as the types of interfaces (SATA, SCSI, IDE), the relationship of the disk to the computer system as a whole, and how data is stored in a file and filesystem at a basic level. A lot of examples given use Linux, due to the raw, accessible nature of UNIX and UNIX-like systems, and the availability of tools like 'dd' to gather data.
Part 2 covers "Volume Analysis," or the organization of files into a storage system. This introduces the basics of things like partition tables (including how to read one). The next few chapters cover PC-based partitions (DOS and Apple), server-based partitions (BSD, Solaris and GPT partitions), and then multiple disk volumes like RAID and logical volumes. With this introduction, the final chapter of the section covers how to use these filesystem descriptions in practice to look for data during analysis. Filesystem layouts, organization, and things like journals and consistency checks are covered with a clarity and exactness that's refreshing for such a detailed topic.
Having covered the basics of filesystems, Part 3 covers the bulk of the book and material. Several chapters follow that specifically show you how to analyze particular filesystems by using their data structures to direct your reads. A range of filesystems are covered, including FAT, NTFS, EXT2 and EXT3, and the BSD types UFS1 and UFS2. Each filesystem has two chapters, one devoted to concepts and analysis, another entirely about data structures. Dividing each filesystem type like this lets Carrier focus first on the theory of each filesystem and its design, and then the practical use of its design to actually understand how to pull data off of it.
The real strength of File System Forensic Analysis lies in Carrier's direct and clear descriptions of the concepts, the completeness of his coverage, and the detail he provides. For example, a number of clear, well-ordered and simple diagrams are peppered throughout the book, explaining everything from allocation algorithms to NTFS alternative data streams. This use of simple diagrams makes the topics more easily understood, so the book's full value can be appreciated. This is the kind of thing that sets a book apart from its peers and makes it a valuable resource for a long time.
Finally, Carrier brings it all together and shows us how many aspects of filesystems can be examined using his "sleuth kit" tools, freely available and easy to use. Without appearing to hawk this tool at the expense of other valuable resources, you get to see how simple and direct filesystem manipulations can be done using a direct approach. This kind of presentation is what makes File System Forensic Analysis a great foundation.
Overall I'm pleased with File System Forensic Analysis, I think that Carrier has achieved what few technical authors do, namely a clear explanation of highly technical topics which retains a level of detail that makes it valuable for the long term. For anyone looking seriously at electronic forensics, this is a must have. I suspect people who are working on filesystem implementations will also want to study it for its practical information about NTFS. Overall, a great technical resource.
You can purchase File System Forensic Analysis from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page. -
Host Integrity Monitoring Using Osiris and Samhain
nazarijo (Jose Nazario) writes "When you arrive to work one morning, you find that your coworker's workstation is acting funny. A quick forensic examination reveals it's been compromised and used to scan the network for more vulnerabilities. When did this happen, and where else is this going on in you domain? With a host integrity monitoring solution, you'll be a lot further along at answering those questions than piecing it all together after the fact. And you can accomplish this with two freeware tools, as described in Host Integrity Monitoring Using Osiris and Samhain, a new book from Syngress Publishing." Read on for the rest of Nazario's review. Host Integrity Monitoring Using Osiris and Samhain author Brian Wotring, with Bruce Potter and Rainer Wichmann pages 450 publisher Syngress rating 8 reviewer Jose Nazario ISBN 1597490180 summary Use freeware tools to ensure your site's security is intact
Host integrity monitoring is the process by which system and network administrators validate and enforce the security of their systems. This can be a complex suite of approaches, tools, and methodologies, and it can be as simple as looking at loggin output. In the past, tools like Tripwire were used to check the configurations on hosts. The freeware version of this tool was limited in its manageability, which was available mainly in the commercial version.
Tools like Osiris and Samhain came along to fill the gap and have since evolved into mature projects themselves. Like any existing software tool out there, any new book should be evaluated not only on its own but also in he context of the existing documentation. Both Osiris and Samhain have decent amounts of documentation available already (Samhain seems to have a larger user documentation repository online than the Osiris tool does), and the book contributes to these docs quite well.
Host Integrity Monitoring shows you how to set up these tools and put them into production on Windows, UNIX, and OS X. Wotring's writing is fairly good, and his examples are usually pretty clear. The pace of the material is good, and there's not a whole lot of domain-specific expertise beyond system administration skills required to make use of the book. At times some of the formatting of the text gets in the way, but that's trivial compared to the quality of writing (which is pretty good).
Overall the material in the book is decent. The book opens with an overview of what host integrity monitoring is, why you should use it, and some of the basic premises. Then it goes on to discuss Samhain and Osiris, starting with their basic installation and then on to their advanced usage. They differ enough that each project merits its own pieces of documentation, even though they're similar in spirit. You'll learn how to schedule scans, integrate with other tools like Swatch, and in general administer a site installation.
The author of the book, Brian Wotring, is more familiar with Osiris than he is with Samhain, and it shows. More material (100 pages) is devoted to using Osiris than is given to Samhain (60 pages), which is to be expected. The coverage of both is sufficient, though, and fills the major parts of the book.
There are three major strengths to this book over the existing docs. The first is seeing not just the tools themselves covered but also the threats they cover in place. The second is having the two tools covered side by side, allowing you to see how to accomplish the same task with each. And thirdly, there are two appendices that are true gems of this book. The first covers how to get your Linksys Linux based AP device monitored using the Osiris tool, which isn't a small feat. The second is how to write your own modules for Osiris and Samhain, for which this appears to be the only documentation for Osiris (Samhain's website has a How To on writing modules). Again, these add value to the book over the freely available documentation.
I would have liked to have seen the chapters devoted specifically to Osiris and Samhain, chapters 6 (Osiris) and 7 (Samhain) broken up into two or three chapters covering their installation and use. The length of these chapters can make finding some material difficult at times. I would have also have liked to see the use of the "bold is input, normal text is output" technical book convention. In many examples finding the user input text can be challenging.
Host Integrity Monitoring Using Osiris and Samhain is not only about these tools but about how to accomplish host integrity monitoring on the cheap (since the code is freely available). While you can find docs on each project, this book complements those docs nicely and provides a nicely wrapped package about how to get the most out of each tool. If you've been thinking about how to ensure that no one is tampering with your system, these tools, and this book, should definitely make your solutions list.
You can purchase Host Integrity Monitoring Using Osiris and Samhain from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page. -
Rootkits: Subverting the Windows Kernel
nazarijo (Jose Nazario) writes "A group of people out there, let's call them 'elite hacker d00ds,' are able to skillfully craft Windows rootkits that evade almost any known detection system. Some people want to know how this is done, be they aspiring elite hackers, security professionals who have to try and find these rootkits, or just interested parties. If you're one of them, Grog Hoglund and James Butler's new book, Rootkits: Subverting the Windows Kernel is for you. It's focused like a laser on how to defeat detection at various levels in the Windows OS once you're in." Read on for the rest of Nazario's review. Rootkits: Subverting the Windows Kernel author Grog Hoglund and James Butler pages 352 publisher Addison-Wesley Longman rating 9 reviewer Jose Nazario ISBN 0321294319 summary A highly technical tour of how to develop and detect Windows rootkits
Some may wonder if Hoglund and Butler are being irresponsible by writing a book that shows you how to bypass detection. If you look closely, however, you'll see that all of the methods they outline are detectable by current rootkit revealing mechanisms. And they also show you how to detect many new rootkits in the process. I consider this book to be a responsible contribution to the community, professionals and amateurs alike, in the finest tradition full disclosure.
The book is organized into three major sections, even if it's note explicitly marked as such. The first section serves as an introduction to the topic and some of the high level concepts you'll need to know about Windows, control mechanisms, and where you can introduce your code. The second part is a highly technical tour of the techniques used to hook your rootkit in and hide it, And the third section is really one chapter covering detection of rootkits.
The first few chapters, which serve to introduce the topic, get technical right away. Chapter 2, for example, shows you some basic mechanisms for hooking in your rootkit. If you're getting lost at this point, you'll want to probably augment your reading with a Win32 internals book. The resources listed by the authors, though, are great. By this point you can also see that the writing is clear and the examples contribute perfectly to the topic. Hardware hooking basics are covered in chapter 3, which should give you some indication of the book's pace (quick!).
By the time you get to chapter 4 and discussing how to hook into both userland and the kernel, you're getting at some very valuable material. Although the book focuses on kernel hooking, a brief description of userland hooking is provided. Chapter 5 covers runtime patching, a black art that's not well known. This is almost worth the full price of admission, but the material gets even better.
In chapters 6-9 you get into some serious deep voodoo and dark arts. In these chapters you'll learn the basics of direct kernel object manipulation, layered device drivers (which can save you a lot of work), hardware manipulation, and network handling. All of these are techniques used by rootkit authors to varying degrees and effect, so you should become familiar with them. The code examples are clear and functional, and you'll learn enough to write a basic rootkit in only about 150 pages. Simple keyboard sniffers and covert channels are described in the code examples. Useful stuff.
I can't say I found many errors or nits in the book. There's some problems at times getting the code formatting just right, and what appear to be a few stray characters here and there, but nothing too obvious to me. Then again, I'm not a Windows kernel programmer, so I don't feel qualified to comment on the correctness of the code.
In the finest tradition of using a blog and dynamic website to assist your readers, the authors have set up rootkit.com, which nicely supplements their book. Most of the resources they mention in the book are available here, as well as a great array of contributors and evolving techniques. Without the book the site is still useful, but together they're a great combination. Too many books lose their value once you read them, and some books stay with you because you're having difficulty understanding the authors. Rootkits will stay near you while you develop your skills because it's a lot of material in a small space, and although it's very clearly written, there is a deep amount of material to digest. You'll be working with this one for a while.
My only major wish for this book is for it to have covered detection more significantly. One chapter covers how to detect rootkits, and although you may be able to look for some specific telltale signs of rootkits depending on how they were introduced, a more complete coverage of this approach would have made the book even more worthwhile.
Rootkits is an invaluable contribution in the wider understanding of advanced attack and hacker techniques. Previously, much of this material was known to only a handful of people, and assembling your own knowledge base was difficult. Hoglund and Butler write clearly, use great code examples, and deliver an excellent book on a high technical and specialized topic. If you're interested in learning how to write your own rootkit or detect someone else's rootkit on your system, you should definitely start with this book.
You can purchase Rootkits: Subverting the Windows Kernel from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page. -
Google to Include iTunes?
Baher Al Hakim writes "The Street suggests in a recent article that Apple is about to announce a deal with Google to offer Apple's iTunes Music service through Google's site. The rumored deal would pair the nation's leading online music store with its leading search engine." Update: 08/13 22:00 GMT by T : Truncation untruncated. -
Infosec Career Hacking
nazarijo writes "Plenty of people are curious as to how to become an information security professional. It's a profession that has a bit of an establishment atmosphere to it where entry to various levels is granted in secret. And it's often hard to understand where to start. Infosec Career Hacking attempts to demystify this process and show you not only generic strategies for employment, but ones specific to the information security field." Read on for the rest of Nazario's review. Infosec Career Hacking: Sell Your Skillz, Not Your Soul author Aaron W. Bayles, Chris Hurley, Johnny Long, Ed Brindley, James C. Foster, Christopher W. Klaus pages 448 publisher Syngress rating 7/10 reviewer Jose Nazario ISBN 1597490113 summary Career guide specifically tuned to the information security professional
The first part of the book is especially useful, and I think provides most of the value that's not available elsewhere. Things that are covered may seem like basics that people should have just picked up, but it's hard to know what you're supposed to know when you change environments, let alone see it all together in one place. I find this section to be especially useful and reasonably well written.
Chapter 1 opens up with a basic orientation of the infosec landscape, including the types of companies and organizations you may want to look at working with, the types of work and positions you see typically, and what kinds of skills you'll need to consider get the interview, let alone the job. Chapter 2 is much like a hacking book in that you're encouraged to perform some scout work on your potential places of employment. Good advice, and it's nice to see it demonstrated. Chapter 3 talks about getting experience and getting your feet wet in the infosec world. Things like conferences, local groups and meetings, and even security clearances are covered. A nice overview, but a it shallow in places, too. Chapter 4 focuses on the resume and the interview, the kinds of things that normally jump to mind when you think about career hacking. A decent overview, and good things to learn.
Part 2 focuses on technical parts. These chapters, I felt, were a bit thin on value and attempted to provide too much coverage but without the depth. What I felt this part of the book was trying to do was to be a quick overview of what you should know if you want a career in information security without any of the work it takes. Because this is such a broad amount of material, and the book only spends about 180 pages on it, the coverage isn't deep. Instead, the cursory coverage is a detriment to the book's value.
Chapter 5 is where I found the most material to complain about. This chapter is titled, 'The Laws of Security', and can be used for your benefit or your downfall. In the right hands, where the nuances that come from actually encountering these challenges in the wild and discovering the reasoning behind them, you can display wisdom. In the wrong hands, where you can't successfully defend a challenge to these axioms, at best you'll appear to be someone who parrots security luminaries, and at worst you'll look like an uninformed buffoon. If you decide to accept conclusions without understanding the reasoning behind them, you're asking for it.
Chapter 6 talks about building a home lab of machines for attack. I felt this chapter devoted too much time to drooling over gear and not enough time discussing more equipment and more valuable gear. Large classes of lab resources, including enterprise applications, networking gear, and even commercial security software was left out. The disclosure debate was reasonably well handled in chapter 7, discussing the various ways that people have established this process. What's missing here is how to actually find where to send the report to and how to ensure it's been acted upon. And finally, a nice, succinct and reasonably comprehensive (if a little too short at times) classification of vulnerabilities and attacks fills chapter 8.
Part 3, 'On the Job', is for when you finally have the position and now you want to keep your job, advance your career, and improve your skills. Unfortunately, this section feels a bit undeveloped in too many places. There's a lot to cover, but the chapters here lack any significant depth to them, and it doesn't feel like they really deliver as strongly as they could.
This section opens with an approach to your career much like an intruder would take to advancing their compromise. Chapter 9 covers how to perform scouting of your new environment, how to get through meetings without messing up, landing your own projects and succeeding with basic project management. Thinking about striking out on your own? That's natural, and the next few chapters will help with that. Chapter 10 is a short list of ideas on how you can use your new knowledge and skills to benefit others, which can help you build a name for yourself and maybe even clients. Chapter 11 looks like it's trying to encourage you to become a local leader of information security knowledge, using that information specifically for incident response. In a crisis, everyone loves a hero, so why can't that be you? And finally, the book closes with a chapter on how to start looking at being an independent consultant. It's been said that you'll never succeed working for someone else, so why not work for yourself? This chapter introduces you to some of the possibilities here, along with some of the considerations. Overall, these chapters have some clear value to them, but because they try and cover so much, they feel underdeveloped and fail to really deliver a strong benefit to the reader.
One of my big concerns when I began reading this book was that it would encourage you to simply become another script kiddy type consultant, capable of downloading a few tools and use old hat techniques to deliver sub-par results. That's a crowded marketplace already, so I didn't want to see anyone encourage that. Instead, it tries to impart valuable career skills. My big complaint is that it tries to do so much that it can't possibly succeed in all of them. It does a decent job, but in some places it definitely lacks the solid landing to make it stick. Overall, though, this uncommon book is a nice twist on the old career guides, tuned for the information security market.
You can purchase Infosec Career Hacking: Sell Your Skillz, Not Your Soul from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page. -
Ian Clarke and Freenet in the Crosshairs
EMIce writes "John Markoff of the New York Times writes of Ian, "Though he says his aim is political - helping dissidents in countries where computer traffic is monitored by the government, for example - Mr. Clarke is open about his disdain for copyright laws, asserting that his technology would produce a world in which all information is freely shared. ... Now, however, Mr. Clarke is taking a fresh approach, stating that his goal is to protect political opponents of repressive regimes." Wasn't freenet originally about dissent? Mr. Markoff appears to be re-writing a history that he probably only knows through a handful of lexis-nexis searches." Update: 08/01 18:32 GMT by T : Ian Clarke wrote to point out his comment posted to the story which lays out the actual subject of his Defcon talk. -
FreeBSD Ported to XBox
NetFiber writes "FreeBSD has been ported to the XBox. "Over the last 2 weeks, I have been working on a port of FreeBSD for the XBox. The port is fully functional. The framebuffer is fully supported, same goes for sound and USB devices (such as an USB keyboard). Only ethernet is missing, currently." The FreeBSD on XBox website is here, downloads included." Update: 07/27 14:50 GMT by T : Rink Springer writes with a request that you use the primary mirror instead. -
Google Hacking for Penetration Testers
nazarijo (Jose Nazario) writes "A couple of years ago, Johnny Long made a large splash in the press with his Google Hacking. He showed the world at large how easy it is to use Google to sift through mountains of information to discover facts about your adversary they didn't know were public (and would rather were private). Now he's written a book with a few other authors and shows you the kinds of techniques and queries you can do to mine Google for all sorts of information." Read on for the rest of Nazario's review. Google Hacking for Penetration Testers author Johnny Long pages 502 publisher Syngress rating 6/10 reviewer Jose Nazario ISBN 1931836361 summary Use the data stored in Google's database to study your adversaryGoogle Hacking for Penetration Testers (Google Hacking for short) is Johnny Long and company's tome on the subject of using what is widely considered to be the web's only worthwhile search engine and the myriad of ways that you can get very specific information out of it. Not just for web pages, you can find Excel spreadsheets, Word documents, and all sorts of information that the owners thought was hidden. This is what makes Google hacking, as an activity, so interesting.
The Google Hacking book starts with Google search basics, which is usually way more than most people do in a given week of using Google. With nary a pause, Chapter 2 covers advanced Google search operators, such as exclusions, file types, and restrictions like "inurl:" and "phonebook:". By this point, you should be sufficiently armed to do some serious Google hacking. Together with the skills and the imagination to phrase what it is you're looking for, you can mine the web.
Chapter 3 provides a simple, fast-paced introduction to using Google to do more than find porn and stalk potential mates. You can dig around in sites to find, for example, backup scripts (which may expose database parameters, useful for SQL injections later on) and eve use Google to hide your tracks as a proxy server (note this only partially works).
The next few chapters focus on the Penetration Testers portion of the title. Chapter 4 starts with the preassessment of the target (of your pen-test), including digging around for information left by employees (ie mails that reveal employee lists), information about the company leaked in job postings (which may include technologies used), and all the kind of stuff you want to know before you start knocking around. Chapter 5 shows you how to use Google and a few other sites to map the target. After all, Google's indexed their site, why not use the data they gathered. Chapter 6 has some real meat in it, including how to find vulnerable CGI programs via Google queries (ie looking for formmail.cgi scripts).
Chapter 7, which is described as "Ten Simple Security Searches That Work", is surprisingly succinct and effective. It basically helps you map the restrictions you learned earlier into queries and data to help you penetrate a target's security without ever leaving Google. Chapters 8 and 9 help you understand how to use Google to enumerate what you can about resources and authentication credentials, and Chapter 10 describes how to pull up documents for your perusal, some of which may be real gems.
Chapter 11 is another interesting chapter, where you learn how to use these same techniques on your own site to determine what kinds of exposures you have. This can include private communications, confidential memos, and even internal configuration information. What doesn't get stressed too clearly at all is that some sites don't respect "robots.txt", for example, and will archive pages indefinitely even if they weren't supposed to. As such, even if you are protected from Google you may not be entirely protected. Now is a good time to learn how to use other major search engines.
I liked where Chapter 12 is headed with automated Google searches via the API and page scraping, but I think more could have been done here to show better, more useful code. As it stands, you'll have to expend some more elbow grease to translate a lot of what you learned earlier into a useful tool for yourself (if you want to write your own). The two appendices on "Professional Security Testing" and "An Introduction to Web Application Security" seem out of place, though, and could have been bridged into the whole book much more cleanly.
Overall I'm not as thrilled with this book as I would have liked to have been for a few key reasons. First, I found the presentation of the book, specifically organization, language and screenshot displays, to be only average. The organization of the book itself seems to jump around sometimes, going from recon work to attacks and then back to basic outside recon work. This becomes a burden when you want to refer back to the book to find a useful portion or to understand the progression of an idea.
Secondly, I found the writing to be heavy with all kinds of 'Leet Hacker' types of references, which get old pretty quickly and only drown out useful information. At over 500 pages, you'd think this book was truly bursting at the seams with information, but a lot of it is redundant or hidden under excess fluff.
Finally, a number of the screenshots are full screens when they could have been only pieces of a screen or a window to achieve an improved effect. This matters because the halftone printing process leaves the images blurry, and a large window or screen is blurry at the book's printing resolution. This is something I've found in common between a bunch of Syngress books, and I hope they'll address it shortly by reviewing their screenshot design.
In conclusion, there's nothing too significantly special about Google hacking. With a bit of elbow grease, some example code for the Google API, reading Google's own docs, and some experimentation you can find yourself at the same level you'd be at with the book, and about $40 heavier, too. However, Long and co-authors have assembled a good number of Google methods together, and if you're the kind of person who prefers to get right to productive work with a book, it's probably the best book I've seen on using Google for more than simple searches.
You can purchase Google Hacking from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page. -
World's Smallest MP3 Player
Chris_Yates writes "mobiBlu is claiming to produce the world's smallest mp3 player, the DAH-1500 cube. The player is 24x24x24 millimeters (about the size of the tip of your finger or various small items), weighs 18 grams, uses OLED technology, and comes in a variety of colors. The 1GB version is currently selling for $130 at Wal-Mart. Buy one today, so you can lose it within a week!" Update: 07/25 22:56 GMT by T : An anonymous reader points out this review of the device. -
Dialup Redeemed: The WiFlyer Modem+Hotspot
Those who've moved to broadband connections and wireless links to each PC on their home or office network are unlikely to look back fondly on the days of 56K (or the not-so-snappy 300 baud of my first modem). Still, even if most Internet users really do have broadband, and (unless you've forsaken a landline telephone completely), dialup is a useful adjunct to even the spiffiest broadband access. And sometimes, it's the only access available. Most city dwellers don't face the distance limits of DSL (or even the geographic limitations of cable service), and cheapskate travelers know that free local calls are more common than hotels with free WiFi. However, wireless access and modems aren't the most common combination (especially when you're talking about laptops with a built-in modem port), and it's not fun being tied to whatever length of phone cord you have to hand. AlwaysOn Wireless's device called the WiFlyer (about $150) combines a wireless access point, a DHCP server, and a modem to make dealing with dialup a bit easier, and tosses in a few other features as well. The WiFlyer is a brilliant device, with some limitations; read on for my review.
Introducing the WiFlyerThere are some other small wireless base stations around, like the Asus WL-530g and the Apple Airport Express, both of which do a good job of turning an available broadband connection wireless, but to my knowledge no others which pack a modem into such a small base station. (The larger Apple Airports do have a modem, as have some devices from Lucent and others, but they're much bulkier.) Each of these tiny base station has its pros and cons -- the Airport Express adds in audio transport, for instance, and like the WL-530g it's a full-fledged 802.11g device -- so your use will determine which makes the most sense. For me, though, the WiFlyer basically hits the sweet spot: it's light, extensible, works as advertised (with one exception, below), and let me connect both my laptops via friends' DSL and cable modems, and over Plain Old Telephone Service.
Physically, the WiFlyer is a slightly rounded grey box that looks it should double as a radar detector. The case is small -- at roughly 1x3x5", about the size of my (old) Handspring Visor, and only 6.5 ounces including the AC power supply. That makes it a good candidate for tossing in a laptop case; at that weight, it's not exactly hefty, but seems solid enough to take travel without complaint. Helpfully, it comes with a wall-wart that's forgivable for not being a line-lump, because the transformer end is small enough -- tiny! -- to stick in one AC socket without obstructing the outlet's other plug. The rear of the device holds the various ins and outs: two ethernet ports (one in from a broadband connection, one out to a local machine), an RJ-11 jack for a telephone line, and the DC power jack.
My only complaint about the WiFlyer's physical design is that it lacks a built-in means (perhaps in the form of a plastic case like the expansion sleeve of the Compaq iPaq) for mounting it under or next to a desk, or high on a cubicle wall to provide better reception.
I recently used the device at several stops along an (ongoing) 6,000-plus mile road trip around the U.S., and found it an indispensable jack of all (networking) trades, with only a touch of "master of none." It neatly replaces everything in the Frankenmodem I assembled a few years ago and have relied on for temporary wireless-by-modem since. It just took a few more years for such a device to appear than I expected it to.
My testbed laptops: a Toshiba Satellite with a 1GHz Celeron chip (saddled with Windows XP), and a 500MHz iBook running Ubuntu Linux 5.0.4.The iBook wireless connection is an internal Airport card (Ubuntu supports the original Airport, though not yet the Airport Extreme), and the Toshiba is getting its wireless access from a USB dongle, a Netgear MA111. (And though the nature of the device means it shouldn't much matter, it's nice to see that Linux support is mentioned explicitly on the package.) In both cases, I used a recent build of Firefox to reach the device's admin page, and (except for better reception in the iBook) there is no difference in behavior, since the WiFlyer requires no client-side software.
Set-up is simple: plug the device in (there's no power switch) and connect it to either an active phone line or an ethernet cable leading to active Internet service. Upon starting a browser and entering the WiFlyer's default IP address (192.168.7.77), the user finds a configuration screen. By default, the WiFlyer is set up for dialup use, and here's one of the best features: stored in memory, the box has local access numbers for "most" major ISPs; a partial list includes Earthlink (the one I use), SBC/Yahoo, MSN, ATT Worldnet and NetZero. The handy thing about ISPs sharing modem pools is that chances are good any ISP with a national presence is reachable through the WiFlyer's list. Just select your location and ISP, supply your username and password, and the WiFlyer dials out. (A small dial on the side controls the modem's volume; it's reassuring to hear those banshees wail sometimes.) This feature worked flawlessly for me from several places around the country; I chose Earthlink's numbers from various locations, and got through without incident. Since Ubuntu Linux can't yet control the modem in my iBook, it's nice to have an external modem like this.
If you can scrounge an ethernet cable with active service upstream to the Internet, though, things are even easier (at least if you are happy with DHCP -- otherwise you'll have to punch in the right numbers in the configuration page). After clicking a button on the config page to switch to broadband, a firmware swap takes place (it requires around a minute; Always On says this was a necessary compromise in the cost of the device), and Shazam! -- miniature broadband wireless router. It seems to take the WiFlyer 60-90 seconds to establish the connection, though; this takes more patience than do my other wireless routers. If you're borrowing a friend's cable-modem line between the cable modem and his PC, connect the other ethernet port to the computer, so everyone's happy.
I didn't use the built-in security features (too far from interested eavesdroppers), but the WiFlyer includes the usual semi-secure means of securing a wireless network from the base-station end; 40/64 bit and 128-bit WEP and MAC address authentication.
LimitationsThe WiFlyer isn't perfect; it has a few drawbacks to take note of, and they could be deal-killers if you need what it doesn't offer.
Most importantly, the range of the WiFlyer is limited; that's what I expected, since it has no external antenna, but the working range is even shorter than I anticipated, and my reception was spotty outside anything more than 20 feet from the box, even with a perfect line of sight. (This is partly to blame on my wireless dongle, but not entirely -- with both the WiFlyer and a common Netgear 802.11b base station active in the same house, I received a much stronger signal from the Netgear even with the WiFlyer within three feet of my 802.11 USB key, while the Netgear was more than 30 feet away and blocked by two thick plaster walls.) That means that an out-of-the-box WiFlyer won't let me browse the web over waffles across the street from a motel. The only way I could get a connection which my Toshiba would call "excellent" was to lay the USB wireless dongle within a foot or two of the WiFlyer. Within a hotel room or small office, the reception is perfectly adequate, though, and if you choose to view the glass as half-full, no wireless moocher is likely to download naughty pictures (or upload naughty email) over your connection.
However, the designers have at least deflected my low opinion of the built-in antenna by including a jack for an MCX antenna, which -- thanks to the proliferation of wireless generally -- are widely available and cheap. The local computer superstore in El Paso (my location at the moment) has a vide variety of these available, starting around $40. So for a permanent installation, the range ought not be a huge concern, but don't expect to cover the footprint of a music festival or even much of a multi-room office without an antenna.
Another limitation is that the DHCP server supports only 5 users at a time. For situations where the WiFlyer is likely to be used, it doesn't seem worth carping too much about this low number -- sharing dialup with more than 5 users seems like a stretch anyhow. But as an emergency backup DHCP server (something it seems perfect for, though clearly not the intended application), this limits its utility. It can't take too much more expensive a chip to bump that number a bit higher. As a wireless Swiss Army Knife, it would also be handy if the WiFlyer featured a bridging mode, so it could be used to extend service from the edge of an existing hotspot. Since it's roughly the size of some USB wireless devices anyhow, this would make it a useful tool to receive as well as provide wireless access.
If you're used to 802.11g, another disappointment: the WiFlyer is 802.11b only. Since even 802.11b vastly outstrips the carrying capacity of American broadband connections generally, the distinction is probably less important than the makers of 802.11g equipment would have you believe; but be warned, the WiFlyer isn't built to facilitate ultra-high-speed intranetwork data transfers.
The UpshotThe only major disappointment I had with the WiFlyer is the short range; that factor aside, it's been a lifesaver. Now if the makers designed in a duck antenna for greater range, added a bridging mode, and removed the slight hassle of a firmware swap to move between broadband and dialup, it would be even snazzier. Hopefully the next generation WiFlyer will add some of those things, but don't get me wrong: if you travel where modem access is your link to the Internet, or you ever need to share a broadband connection temporarily, the WiFlyer is well worth buying and keeping in your hit-the-road bag.
-
Video iPod May Arrive in September
Fuzzball963 writes "MSNBC is reporting that Apple is in talks with major record labels to license and sell video content on the iTunes music store. The videos would sell for $1.99 and be playable on a video iPod, which Apple has reported may come out sometime in September." Update: 07/18 18:54 GMT by T : Carl Bialik from the WSJ writes "Just to add to the previous Apple post, here's a free link to today's Wall Street Journal article upon which the MSNBC article was based." -
Falling Window Cover Damages Discovery
Mz6 writes "At 5:30PM EDT, one of the space shuttle's protective window covers fell and struck the left Orbital Maneuvering System engine pod on Discovery today. The window cover hit the carrier panel around the OMS pod. NASA is taking a new panel to the launch pad to replace the one hit by the falling cover. NASA is expected to know by 7 PM EDT if the replacement panel will work and whether launch can proceed tomorrow as planned. The window cover in question is from one of the overhead windows. It fell on its own, not when workers were handling it. The cover was found after it had fallen and hit the orbiter. In addition to the carrier panel that workers plan to replace tonight, engineers are looking for any other damage." Update: 07/13 02:03 GMT by T : RmanB17499 points out a CNN story according to which "the launch of the space shuttle Discovery will go ahead as scheduled Wednesday after technicians replaced two protective tiles damaged near the spacecraft's tail Tuesday, a NASA spokeswoman said." -
City of Vienna Chooses Linux
Bill Kendrick writes "Back in January, ZDNet reported that the city of Vienna, Austria was looking to move at least a portion of its desktops to Linux. Well, it looks like it happened (in German; use the fish). Their official distro is based on Debian with KDE, and is called WEINUX." Update: 07/06 12:49 GMT by T : Several readers wrote to correct the spelling here: the correct name of the distro is "WIENUX." -
Britain's First Jedi Member of Parliament
earthlingpink writes "In his maiden speech to the House of Commons, the Hon. Member for Copeland, Jamie Reed MP, announced that he is a Jedi: "as the first Jedi Member of this place, I look forward to the protection under the law that will be provided to me by the Bill" (the quotation is a fair way down the page; search for 'Jedi,' not surprisingly). How long before we have a Congressional equivalent?" Update: 06/29 23:15 GMT by T : Reader JE_Hoover adds a correction: "Although the previous MP for Copeland was the Hon. Member for Copeland, the current MP for Copeland is not a member of the privy council. Debretts make it all clear." -
SUSE Linux 9.3 FTP Version Released
twener writes "Nine weeks after the official release of SUSE Linux 9.3 Professional Novell has made available the FTP edition for everyone as a 4.2GB DVD ISO image (both i586 and x86_64) or 5 CD images (stripped-down, i586 only) on their mirrors. Additionally, all until-now missing RPMs have been uploaded, so you can also do a network installation using the 57MB netinstall ISO." Update: 06/27 19:37 GMT by T : Yes, that's RPMs, not RMPs. Sorry! -
Supreme Court Rules against Grokster
furry_wookie writes "A few minutes ago, the U.S. Supreme Court has ruled unaniumously against Grokster today. This ruling means that developers of software violate federal copyright law when they provide computer users with the means to share music and movie files downloaded from the Internet. More info about the case here." That's not an entirely accurate statement -- what The Supremes said is that "One who distributes a device with the object of promoting its use to infringe copyright ... is liable for the resulting acts of infringement by third parties using the device, regardless of the device's lawful uses." The promotion is the key part of that statement. Update: 06/27 18:00 GMT by T : Reader SilentBob4 points out this interview with EFF attorney Wendy Seltzer on the decision. -
Google Launches Pay-Per-View Web Video
Elliot Shepherd writes "According to John Batelle, on Monday Google is launching in-browser video playback based on VLC. Google has been accepting video uploads in April, including allowing the video owner to specify that payment is required, through the Google Payment Program." Update: 06/27 22:21 GMT by T : An anonymous reader writes "Google Video is now up. The about page describes what kinds of content has been uploaded to their servers so far -- mostly a random assortment of stuff from Gamespot's archives, a few things from Greenpeace, a Google recruiting video, some breakdancing videos, and other randomness. The in-browser video plugin works seamlessly (although Windows only for now). Looks like it has potential." Check the top entry on Google Blog for a few more words on it, too. -
HOWTO: 0.5TB RAID on a Budget
Compu486 writes "Inventgeek.com has a new how-to article titled 'The Poor Mans Raid Array.' The article details how to make a modular .5 terabyte Raid 5 array for under $250 (USD), and it all runs on the Mandriva flavor of Linux." Drive prices being what they are, this seems cooler than it is practical. Update: 06/25 23:31 GMT by T : If that's not enough storage, Yeechang Lee writes "Let me show off the 2.8TB Linux-powered RAID 5 array I built for home use a few months ago. I provide lots of details on how I did it, what I used, and the results. The Usenet thread has good followup posts from others, too." -
Integrated Circuit Inventor Jack Kilby Dead at 81
geekotourist writes " Jack Kilby , inventor of the integrated circuit, one winner of the 2000 Nobel Prize in Physics (Robert Noyce died in 1990), died June 20th after a brief battle with cancer. In 1958 he invented the foundation for a trillion dollar industry as a substitute for going on vacation." Update: 06/22 02:03 GMT by T : Kilby was 81, not 91 as the headline originally indicated. -
The Art of Computer Virus Research and Defense
nazarijo writes "I think by now we're all familiar with viruses and worms. It may have been a term paper diskette chewed up by a virus back in college, a family member's computer infected with the latest worm, or your email inbox clogged with a mass mailer of the week. But how do AV researchers dissect such malware, especially when virus writers have devoted so much time to avoiding detection and perfecting their craft with self-decrypting viruses, polymorphic shellcode, and obfuscated loops. Haven't you wanted a peek into how that's done, and how you would analyze such a monster that landed in your computer? Well, Peter Szor's book The Art of Computer Virus Research and Defense (TAOCVRD) has been gaining lots of critical acclaim lately for filling that gap, and rightfully so. (Before we begin, however, I should make one thing perfectly clear: I was a technical reviewer of this book. I enjoyed it when I read it originally, and I'm even more pleased with the final result. And now on to your regularly scheduled review.)" Read on for the rest. The Art of Computer Virus Research and Defense author Peter Szor pages 713 publisher Addison Wesley Longman and Symantec Press rating 9 reviewer Jose Nazario ISBN 0321304543 summary Clear, sweeping coverage of virus history and technical detailsTAOCVRD opens with Part 1: Strategies of the attacker. Here we get to start to think about malicious code from the original ideas and viewpoints of its makers. Chapter 1 opens up with various games of the classic computer science world, including Conway's Game of Life and Core Wars, which is still fun after all of these years. From this we can start to think about computer viruses as a natural extension of other self-replicating computer structures. What's great about this chapter is that you can actually understand, and share in, the fascination of replicating code. It's as if you can understand the pure world that some virus writers live in.
Chapter 2 starts off the virus-analysis section, including some of the basics (like the types of malicious programs and their key features), as well as the naming scheme. Chapter 3, "Malicious Code Environments," serves as a lengthy and complete description of how various viruses work. The dependencies that you would expect to see, including OS, CPU, file formats, and filesystems, are all described. Then Szor goes on to describe how viruses work with various languages, from REXX and DCL to Python and even Office macros. Not all of the descriptions are lengthy, but you get to see how flexible the world of writing a virus can be. What I most enjoyed about the book overall is represented in this chapter, namely Szor's command of the history of the virus as well as his technical prowess, which he drops in as appropriate.
Chapter 4 gets a bit more technical and now focuses on infection strategies. Again, Szor isn't afraid to delve into history or technical meat, including a lengthy and valuable section "An In-Depth Look at Win32 Viruses." If you don't feel armed to start dissecting viruses by this point, you're in luck: there's so much more to read. Chapter 5 covers in-memory strategies used by viruses to locate files, processes, and sometimes evade detection. Szor has a list of interrupts and their utility to the virus writer, providing a comprehensive resource to the virus analyst.
Chapters 6 and 7 cover basic and advanced self protection schemes, respectively, used by viruses. TAOCVRD's completeness of information in a usable space, together with very functional examples and descriptions, is again evident. Szor walks you through a basic decryptor routine, for example, showing you how a self-contained virus can be both evasive and functional at the same time. Sadly little attention is given to various virus construction kits at the end of chapter 7, though.
Chapters 8 and 9 get a little less technical and somewhat more historical. These chapters cover virus payloads and their classification (ie benevolent viruses, destructive viruses, etc) and computer worms, respectively. The overview of payloads is almost entirely historical, giving a great overview of how virus writers have used their techniques to cause havoc or just have "fun" from time to time. Chapter 9 gives a concise and valuable overview of computer worms, almost boiling about half of my worms book down into just one chapter in a clear and easy to use fashion.
Part 1 concludes with chapter 10, which covers exploits and attack techniques used by worms and viruses. Again, Szor's clarity of explanation shines as he artfully gives a concise overview of how a buffer overflow attack works (including stack layout and address manipulation), heap-based attacks, format string attacks, and related methods. He then discusses these techniques in light of various historical examples, clearly explaining how they operated and were successful. If you've been yearning for a short overview of attack techniques and how malware has used them, this chapter is for you.
Part 2 covers the defender's strategies. Chapter 11 serves as a nice introduction to this section by describing many of the current and advanced defense techniques such as some of the first and second generation scanners, code and system emulation, and metamorphic virus detection. This is all covered in nice technical detail, always at a reasonable level to not leave everyone in the dust. Through it all small examples are constantly given, which reinforce the text nicely. Chapter 12 is very similar, this time focusing on in-memory scanning and analysis techniques.
Chapter 13 covers worm blocking techniques, focusing on host-based methods which can prevent the buffer overflow from being successful or the code from arbitrarily gaining network access again. Chapter 14 complements this with network specific defenses, including ACLs and firewalls, IDS systems, honeypots, and even counterattacks. These two chapters are a lot less technical than the previous two, but still quite valuable.
By this point I'm sure you're ready to try your hand at virus analysis, and Szor is eager to help you out. In chapter 15 he gives you a great setup for virus analysis, including various tools and examples of how they work and what kind of information they give you. Finally, in chapter 16 you have the obligatory (and valuable) resource roundup which complements the references given in every chapter, as well.
Overall I find Szor's book to be amazing, both in terms of its technical prowess over so many specifics in the field but also for its presentation. Without dumbing it down, Szor's able to communicate to most readers with clarity in a manner they'll understand, learn from, and be able to use. I think that many of us, especially those of us who get plundered in our email inboxes with malware, are curious to spend some time dissecting these beasts using techniques AV professionals use, and Szor's book does an exemplary job of introducing that world to us all. I consider this to be one of the most important computer security books I own due to it's clarity and completeness of coverage.
You can purchase The Art of Computer Virus Research and Defense from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page. -
UK anti-ID card campaign Gains Momentum
Jack writes "The British No2ID campaign, which opposes the creation of a National Identity Database to hold biometric data on all UK citizens, has created an online pledge as part of an effort to publicise their cause. The three-day old pledge has recently gained the attention of the blogging community, with bloggers bringing a thousand new signatories to the pledge today alone. Readers in the UK are invited to look at the No2ID FAQ on the plans for mandatory ID cards - some of it makes for scary reading." Update: 06/14 17:13 GMT by T : Side note: Tom Steinberg, director of MySociety.org (organizers of this petition) writes "The ID pledge is cool in that it is so big and successful, but it is a very small insight into what pledgebank.com can do." It's actually a much more general organizing tool. -
Fedora Core 4 Available
Limburgher writes "As of a few minutes ago, the torrents listed at duke went live. Nothing on the main site yet, however. The more people get on the torrents, the faster they will be. You all know the drill." Update: 06/13 19:07 GMT by T : Also in Red Hat-related news, halfbyte_hosting writes "CentOS 4.1 is now on the mirrors and ready for download." -
Results in for UCSB Capture the Flag Contest
Thorsten Holz writes "A few hours ago, the UCSB International Capture The Flag (CTF) contest ended. The CTF contest is a multi-site, multi-team hacking contest in which a number of teams compete independently against each other. It is the biggest contest worldwide and different from the DEFCON CTF because it involves nine educational institutions spread worlwide. Our team (called '0ld Eur0pe') managed to get second place, although our VMware image was 'rm -rf'ed during the contest! The final scoreboard shows the result and some impressions can be found at our homepage." Update: 06/12 00:17 GMT by T : Thanks to reader Bob MacSlack, who spotted my goof and corrects it thus: "The article incorrectly attributes the contest to UC Berkeley. UCSB is actually UC Santa Barbara." -
Writing Down Passwords?
Atryn wonders: "I was recently checking for the latest firmware for a Netgear router when I decided to click on their Guide to Internet Security where it states: 'Contrary to much 'expert' advice, there is very little risk writing down passwords. In fact, years from now you may discover you need them to access old files.' I'm wondering what Slashdot thinks of Netgear's recommendation." Update: 06/08 21:19 GMT by T : Reader 654043 reminds us of the Microsoft recommendation to write down passwords which ran a few weeks back, and which has some pretty sound reasoning behind it. -
First Google Maps Hack Takedown
An anonymous reader writes "Despite "users accelerating innovation" with Google Maps the 'hacks' are not immune from Google's legal team, who have taken down "Google Wallpapers for violating the terms of agreement. From a quick skim through the terms it would seem that most sites using the Google Maps data are in violation. Are Chicago Crime and Google Sightseeing next to go?" It may be a shame to shut down Google Maps offshoots, but that has to be the nicest take-down note I've ever seen; it's polite, friendly and reasonable. Update: 06/08 21:22 GMT by T : Below, a few more of the current uses for Google Maps. An anonymous reader submits "The AP is running a story about the multiple uses for Google Maps. Among the uses, Tracking sexual predators in Florida, Guiding travelers to the cheapest gas nationwide, Pinpointing $1,500 studio apartments for rent in Manhattan, and Finding crime in Chicago. It'll be interesting to see if Google allows these sites to remain online or not." -
HOW TO: Convert a Mac into an x86
inventgeek writes "With the recent announcements Apple has made regarding its operating environment, Inventgeek.com has a mod that seems rather fitting. They have converted a Mac G3 to an Intel P4 System capable of running Windows or Linux. Full how to is available on there site for those brave enough to bask in what many say could be Apples greatest folly, and a blow to Linux." Update: 06/08 17:53 GMT by T : A few further Mac-OS-X-on-Intel notes, about the new Intel development kit from Apple: Readers jimboman78 and shooflot sent in, respectively, links to (mostly positive) comments on the front page of Accelerate Your Mac and a more skeptical but equally preliminary description at Think Secret. -
PalmOne Releases 4GB PDA [updated]
davidconger writes "PalmOne has introduced the first device in their new line of Mobile Manager handheld devices. The LifeDrive includes an embedded 4GB Hitachi Microdrive and additional software for file/folder synchronization. The device includes both WiFi and Bluetooth. Price tag on the device $499. PocketFactory has done a complete review of the LifeDrive." Reader gandell adds a link to Brighthand's review. Update: 05/18 18:08 GMT by T : An anonymous reader corrects this story's original headline, writing "Despite rumors the LifeDrive would run Linux, it runs PalmOS 5 (Garnet). However, the device seems to have a Linux-friendly design, and is likely to run Linux soon, whether supplied by PalmOne's sister company PalmSource, or by Linux hobbyists. PalmSource is likely to offer a Linux OS upgrade for the LifeDrive, once it is ready to support the huge variety of legacy Palm apps under Linux." Update: 05/18 18:44 GMT by T : One more review, this one at MobileTechReview. -
PalmOne Releases 4GB PDA [updated]
davidconger writes "PalmOne has introduced the first device in their new line of Mobile Manager handheld devices. The LifeDrive includes an embedded 4GB Hitachi Microdrive and additional software for file/folder synchronization. The device includes both WiFi and Bluetooth. Price tag on the device $499. PocketFactory has done a complete review of the LifeDrive." Reader gandell adds a link to Brighthand's review. Update: 05/18 18:08 GMT by T : An anonymous reader corrects this story's original headline, writing "Despite rumors the LifeDrive would run Linux, it runs PalmOS 5 (Garnet). However, the device seems to have a Linux-friendly design, and is likely to run Linux soon, whether supplied by PalmOne's sister company PalmSource, or by Linux hobbyists. PalmSource is likely to offer a Linux OS upgrade for the LifeDrive, once it is ready to support the huge variety of legacy Palm apps under Linux." Update: 05/18 18:44 GMT by T : One more review, this one at MobileTechReview. -
Yahoo Introduces Competitor for iTunes
LadyDeath writes "After a year in development, Yahoo has launched its competitor to Apple's iTunes and Napster To Go, a subscription and download music service priced at only $4.99 per month. Tracks are offered in 192Kbps WMA, and can be transferred to portable devices. Perhaps most interesting to the Slashdot crowd is that the Yahoo! Music Engine is built on an open platform that facilitates plug-ins - both DLL and Web based. Podcasting and video playback plug-ins are already available." Update: 05/11 13:06 GMT by T : ian c rogers, formerly of Nullsoft, just led the build of the media player, and writes with information about "the the plugin architecture it supports as well as some of the 20 plugins that are already available for it. I've posted my thoughts on why someone should or shouldn't use the Yahoo! Music Engine on my blog." -
2 Firefox Security Flaws Lead to Exploit Potential
Marthisdil points out a News.com story which reports that "Two vulnerabilities in the popular Firefox browser have been rated "extremely critical" because exploit code is now available to take advantage of them." Security firm Secunia reported the vulnerabilities (and the "extremely critical" rating is theirs), but the News.com story points out that thus far, "no known cases have yet emerged where an attacker took advantage of the public exploit code." Update: 05/09 20:20 GMT by T : Rebron of the Mozilla Foundation sends a correction; this is really the same flaw reported yesterday. He suggests that you glance at the Mozilla security alert on this hole (as well other alerts at the Mozilla Security Center), and says "The Mozilla Foundation has made changes to our update servers that will protect users from this arbitrary code execution exploit." -
The Early History of Nupedia and Wikipedia: A Memoir
Larry Sanger was one of the moving forces behind the pioneering Nupedia project. That makes him one of the people to thank for Wikipedia, which has been enjoying more and more visibility of late. Sanger has prepared a lengthy, informative account of the early history of Nupedia and Wikipedia, including some cogent observations on project management, online legitimacy, dealing with trolls, and other hazards of running a large, collaborative project over the Internet. As Sanger writes, "A virtually identical version of this memoir is due to appear this summer in Open Sources 2.0, published by O'Reilly and edited by Chris DiBona, Danese Cooper, and Mark Stone. The volume is to be a successor to Open Sources: Voices from the Open Source Revolution (1999)." Read on below for the story (continued tomorrow). Update: 04/20 19:19 GMT by T : Here's a link to the continuation of Sanger's memoir.Contents:
Preface
Some recent press reports
Nupedia
The origins of Wikipedia
Wikipedia's first few monthsPreface
An impassioned debate has been raging, particularly since about the summer of 2004, about the merits of Wikipedia and the future of free online encyclopedias. This discussion has not benefitted by much detailed, accurate consideration of the origins of Wikipedia and of its parent project, Nupedia. But it seems to me that those origins are very important -- crucial, even -- to forming a proper judgment of the current state and best future direction of free encyclopedias.
Wikipedia as it stands is a fantastic project; it has produced enormous amounts of content, thousands of excellent articles, and now, after just four years, is getting high-profile, international recognition as a new way of obtaining at least a rough and ready idea about very many topics. Its surprising success may be attributed, briefly, to its free, open, and collaborative nature.
This has been my attitude toward Wikipedia practically since its founding. But a few months ago I wrote an article critical of certain aspects of the Wikipedia project, 'Why Wikipedia Must Jettison Its Anti-Elitism', which occasioned much debate. I have also been quoted, as co-founder of Wikipedia, in many recent news articles about the project, making various other critical remarks. I am afraid I am getting an undeserved reputation as someone who is opposed to everything Wikipedia stands for. This is completely incorrect. In fact, I am one of Wikipedia's strongest supporters. I am partly responsible for bringing it into the world (as I will explain), and I still love it and want only the best for it. But if a better job can be done, a better job should be done. Wikipedia has shown fantastic potential, and it is open content--and so if the project has problems (or features) which will keep it from being the maximally authoritative, broad, and deep reference that I believe could exist, I firmly believe that the world has the right to, and should, improve upon it.
Wikipedia's predecessor, which I was also employed to organize, was Nupedia. Nupedia was to be a highly reliable, peer-reviewed resource that fully appreciated and employed the efforts of subject area experts, as well as the general public. When the more free-wheeling Wikipedia took off, Nupedia was left to wither. It might appear to have died of its own weight and complexity. But, as I will explain, it could have been redesigned and adapted--it could have, as it were, "learned from its mistakes" and from Wikipedia's successes. Thousands of people who had signed up and who wanted to contribute to the Nupedia system were left disappointed. I believe this was unfortunate and unnecessary; I always wanted Nupedia and Wikipedia working together to be not only the world's largest but also the world's most reliable encyclopedia. I hope that this memoir will help to justify this stance. Hopefully, too, I will manage to persuade some people that collaboration between an expert project and a public project is the correct approach to the overall project of creating open content encyclopedias.
I am not writing to request that Nupedia be resuscitated now, as nice as that would be. But I would like to tell the story of Nupedia and the first couple years of Wikipedia, as I remember it. A more complete history of the projects, as opposed to a memoir, must await a careful study of the Nupedia and Wikipedia archives--if early archives of them still exist (I have no idea if they do)--or else these entries from the "Wayback Machine." Interviews with many of those heavily involved in the projects would also help a great deal, so long as interviews were done of people on different side of the disputes that helped to shape the project.
By the way, the "overall project of creating open content encyclopedias" is something of which I have been writing since at least 2001. For example, in July of 2001, while still working on both Wikipedia and Nupedia, I wrote, "if some other open source project proves to be more competitive, then it should and will take the lead in creating a body of free encyclopedic knowledge." Since Wikipedia is open content and hence may be reproduced and improved upon by anyone, I have always been cognizant that it might not end up being the only or best version. My personal devotion has always been to the ideal project as I have envisioned it, not necessarily to particular incarnations of Nupedia or Wikipedia; and I think this attitude is fully consistent with the (very positive) spirit of open source collaboration generally.
This being said, let me also emphasize strongly that, throughout this discussion, I am not suggesting that Wikipedia needs to be replaced with something better. I do, however, think that it needs to be supplemented by a broader, more ambitious, and more inclusive vision of the overall project.
Some recent press reports
The following memoir seems all the more important to publish now because the early history of Nupedia and Wikipedia has been mischaracterized in the press recently. If there were only a few inaccuracies, which made no difference, I would be happy to leave well enough alone. But some of the mischaracterizations I've seen do make a difference, because they give the public the impression that Nupedia failed because it was run by snobbish experts whose standards were too high. As the following should make clear, that is not quite correct. One might also gather from some reports that the idea for Wikipedia sprang fully grown from Jimmy Wales' head. Jimmy, of course, deserves enormous credit for investing in and guiding Wikipedia. But a more refined idea of how Wikipedia originated and evolved is crucial to have, if one wants to appreciate fully why it works now, and why it has the policies that it does have.
For example, in the Nov. 1, 2004 issue of Newsweek, in "It's Like a Blog, But It's a Wiki," reporter Brad Stone writes:
[Jimmy] Wales first tried to rewrite the rules of the reference-book business five years ago with a free online encyclopedia called Nupedia. Anyone could submit articles, but they were vetted in a seven-step review process. After investing thousands of his own dollars and publishing only 24 articles, Wales reconsidered. He scrapped the review process and began using a popular kind of online Web site called a "wiki," which allows its readers to change the content.
This capsule history is, of course, very brief and so should be expected not to have every relevant detail. But some of the claims made here are not just vague, they are actually misleading, and so several clarifications are in order (all of this is elaborated below):- The article makes it sound as if Jimmy were the only person making the relevant decisions. That is incorrect; the Nupedia system (indeed, seven steps) was established via negotiation with Nupedia's volunteer Advisory Board, mostly Ph.D. volunteers, who served as editors and peer reviewers. I articulated our decisions in Nupedia's "Editorial Policy Guidelines." Jimmy started and broadly authorized it all, but as to the details, he really had little to do with them.
- Nupedia's Advisory Board might be surprised to learn that Jimmy (alone!) "scrapped the review process." Jimmy was certainly disappointed with the process (as were many people), and he did not actively support it after 2001 or so. But in fairness to the people actually working on Nupedia, the fact is that work on Nupedia gradually petered out in 2001-2. I in particular was stretched thin--in 2001, I was both chief organizer of Wikipedia and editor-in-chief of Nupedia--and my own slowing work on Nupedia was obvious to all active Nupedia contributors. It might be better to say that Nupedia withered due to neglect--which was largely due to a lack of sufficient funds for paid organizers--which was as much due to the bursting of the Internet bubble as anything else.
- Also, to the best of my knowledge, the "thousands of his own dollars" invested in these projects were, if I am not very mistaken, the dollars of Bomis.com, which is jointly owned by three partners, Jimmy, Tim Shell, and Michael Davis. (The money for Wikipedia now comes from donations.) But again, Jimmy was the prime motivating force within Bomis.
- Moreover, Nupedia had fewer than 24 articles when Wikipedia launched, being not quite a year old at that time. The idea of adapting wiki technology to the task of building an encyclopedia was mine, and my main job in 2001 was managing and developing the community and the rules according to which Wikipedia was run. Jimmy's role, at first, was one of broad vision and oversight; this was the management style he preferred, at least as long as I was involved. But, again, credit goes to Jimmy alone for getting Bomis to invest in the project, and for providing broad oversight of the fantastic and world-changing project of an open content, collaboratively-built encyclopedia. Credit also of course goes to him for overseeing its development after I left, and guiding it to the success that it is today.
A March 2005 Wired Magazine article by Daniel Pink also got a number of things wrong, despite being, in other respects, an excellent article:
With Sanger as editor in chief, Nupedia essentially replicated the One Best way model. He assembled a roster of academics to write articles. (Participants even had to fax in their degrees as proof of their expertise.) And he established a seven-stage process of editing, fact-checking, and peer review. "After 18 months and more than $250,000," Wales said, "we had 12 articles."
This too needs clarifications:Then an employee told Wales about Wiki software. On January 15, 2001, they launched a Wiki-fied version and within a month, they had 200 articles. In a year, they had 18,000. ... Sanger left the project in 2002. "In the Nupedia mode, there was room for an editor in chief," Wales says. "The Wiki model is too distributed for that."
- The "roster of academics" (the aforementioned Nupedia Advisory Board) was not limited to academics; they were experts in their fields, in any case. Moreover, they were editors and peer reviewers; the general public was able to propose and write articles on subjects about which they had some knowledge. (Consult the old assignment policy if you are interested.)
- It is incorrect to say that participants had to fax their degrees as proof of their expertise; we did verify bona fides by matching the names and e-mail addresses of editors and reviewers with a web page--often, but not always, an academic web page. Indeed there was one (but only one) case that I recall in which I asked someone, who had no web page or any other easy way to prove who he was, to fax a degree. Verifying bona fides seemed like a good idea especially when initially building what was to be an academically-respectable project.
- Again, I did not establish the editorial process alone; I had considerable assistance (for which I am still grateful) from Nupedia's excellent Advisory Board.
- And as I wrote on July 25, 2001 for Kuro5hin, "Britannica or Nupedia? The Future of Free Encyclopedias," Nupedia had "just over 20" articles--not 12--after 18 months. We always suspected that we would wind up scrapping our first attempts to design an editorial system, and that we would learn a great deal from those first attempts; and that's essentially what happened. But Nupedia could have evolved, and would have, had we continued working on it.
- The second paragraph begins, "Then an employee told Wales about Wiki software." I don't know how Jimmy first learned about wikis, but as I will explain below, I proposed to him and to the Nupedia community at large that we start a wiki-based encyclopedia.
- The context of the line "Sanger left the project in 2002"--particularly with Jimmy quoted as saying, "In the Nupedia mode there was room for an editor in chief"--makes it sound as if I were let go specifically because I was working only on Nupedia and that I was no longer needed for that. In fact, I was working on Wikipedia far more at the time than Nupedia, and the reason for my departure from both projects was that Bomis was, like virtually all dot-coms, losing money. They could not afford to pay me; I was told that I was the last of several newer Bomis employees to be laid off on account of the tech recession. But Wikipedia indeed was able to continue on without me, and I agreed even at the time that Wikipedia could survive without me, and that it had become essentially "unmanageable" (as I put it--the following memoir should make it clear what I meant by that).
Nupedia
I'm going to begin this memoir with several paragraphs about Nupedia, because the origin of Wikipedia cannot be explained except in that context. Moreover, the Nupedia project itself was very worthwhile, and I think it might have been able to survive, as I will explain. Finally, some errors regarding Nupedia have been passed around (a few examples are above), which are little better than unfounded rumors. It is unfortunate that the thousands of hours of excellent volunteer work done on Nupedia should be thus disrespected or grossly misunderstood. I personally will always be grateful to those initial contributors who believed in the project and our management, worked hard for a completely unproven idea, and laid the groundwork for the growing institution of open content projects.
In 1999, Jimmy Wales wanted to start a free, collaborative encyclopedia. I knew him from several mailing lists back in the mid-90s, and in fact we had already met in person a couple of times. In January 2000, I e-mailed Jimmy and several other Internet acquaintances to get feedback on an idea for what was to be, essentially, a blog. (It was to be a successor to "Sanger and Shannon's Review of Y2K News Reports," a Y2K news summary that I first wrote and then edited.) To my great surprise, Jimmy replied to my e-mail describing his idea of a free encyclopedia, and asking if I might be interested in leading the project. He was specifically interested in finding a philosopher to lead the project, he said. He made it a condition of my employment that I would finish my Ph.D. quickly (whereupon I would get a raise)--which I did, in June 2000. I am still grateful for the extra incentive. I thought he would be a great boss, and indeed he was.
To be clear, the idea of an open source, collaborative encyclopedia, open to contribution by ordinary people, was entirely JimmyÃââs, not mine, and the funding was entirely by Bomis. I was merely a grateful employee; I thought I was very lucky to have a job like that land in my lap. Of course, other people had had the idea; but it was Jimmy's fantastic foresight actually to invest in it. For this the world owes him a considerable debt. The actual development of this encyclopedia was the task he gave me to work on.
So I arrived in San Diego in early February, 2000, to get to work. One of the first things I asked Jimmy is how free a rein I had in designing the project. What were my constraints, and in what areas was I free to exercise my own creativity? He replied, as I clearly recall, that most of the decisions should be mine; and in most respects, as a manager, Jimmy was indeed very hands-off. Nevertheless, I always did consult with him about important decisions, and moreover, I wanted his advice. Now, Jimmy was quite clear that he wanted the project to be in principle open to everyone to develop, just as open source software is (to an extent). Beyond this, however, I believe I was given a pretty free rein. So I spent the first month or so thinking very broadly about different possibilities. I wrote quite a bit (that writing is now all lost--that will teach me not to back up my hard drives) and discussed quite a bit with both Jimmy and one of the other Bomis partners, Tim Shell.
I maintained from the start that something really could not be a credible encyclopedia without oversight by experts. I reasoned that, if the project is open to all, it would require both management by experts and an unusually rigorous process. I now think I was right about the former requirement, but wrong about the latter, which was redundant; I think that the subsequent development of Wikipedia has borne out this assessment. But I fully realize that all of this is a matter of debate. Some will claim that the experience of Wikipedia refuted my original judgment that expert oversight is necessary for a very credible encyclopedia; but I disagree with them. More on this below.
Also, I am fairly sure that one of the first policies that Jimmy and I agreed upon was a "nonbias" or neutrality policy. I know I was extremely insistent upon it from the beginning, because neutrality has been a hobby-horse of mine for a very long time, and one of my guiding principles in writing "Sanger's Review." Neutrality, we agreed, required that articles should not represent any one point of view on controversial subjects, but instead fairly represent all sides. We also agreed in rejecting an alternative that (for a time) Tim and some early Nupedians plugged for: the development, for each encyclopedia topic, of a series of different articles, each written from a different point of view.
I believed, moreover, that a strongly collaborative and open project could not survive if its contributors were not "personally invested" in the project, and that this required some input and management by its users. So I think it was very early on that I decided that Nupedia should have an Advisory Board--editors, and peer reviewers, who would together agree to project policy--and that the public should have a say in the formulation of policy.
An early incarnation of NupediaÃââs Advisory Board was in place by summer of 2000 or so. It was made up of the project's highly-qualified editors and reviewers, mostly Ph.D. professors but also a good many other highly-experienced professionals. Eventually the Advisory Board agreed to an extremely rigorous seven-step system. A lot of the details of the Nupedia policy and processes were, I think, proposed by me, but then tweaked and elaborated by others, and the policy was not published as project policy until we had a quorum of editors and peer reviewers who could fully discuss and approve of a policy statement. But I do not think that we discussed the proposal well enough, and further initial discussion could have made a difference, because, as it turned out, a clear mistake of mine and others was to assume that such a complicated system would be navigated patiently by many volunteers, even if they had clear-enough instructions. That is a mistake I doubt anyone designing volunteer content creation systems will make again; I certainly would not make it again.
I spent a huge amount of time recruiting people for Nupedia, e-mailing new arrivals, posting to mailing lists, giving interviews, etc. I had had some experience publicizing Internet projects when I worked on several philosophy discussion groups as a grad student in the 1990s (I had perpetrated an "Association for Systematic Philosophy" as well as a "Tutorial Manifesto"), and I knew that getting many willing and active participants was difficult but important. I even had an administrative assistant for six months in 2000 and 2001, Liz Campeau, whose sole job was to recruit people to work on Nupedia and then Wikipedia. I think a large part of the reason Wikipedia got off the ground so quickly and so well is that it was started by Nupedians, who were then a very large base of people who wanted to work on an encyclopedia, and who had many definite ideas about how it should be done. Maybe 2,000 Nupedia members were subscribed to the general announcement list in January of 2001, when Wikipedia launched--I forget how many but an old project news page indicates that 2,000 is about right.
We operated the system initially using e-mail and mailing lists, while planning and finalizing process details. That lasted from spring through fall 2000. I think our first article ("atonality" by Christoph Hust), that made it entirely through the system, was published in June or July of 2000. To move the system to a completely web-based one, there was, of course, a great deal of design and programming to do. So in fall of 2000 I worked a lot with a specifically-hired programmer (Toan Vo) and the Bomis sysadmin (Jason Richey) to transfer the system from a clunky mailing list system to the web. But by the time the web-based system was ready--I think December of 2000, just a month before Wikipedia got started--it had become obvious to Jimmy and me that the seven-step editorial process would move too slowly, even when managed on the web. But Magnus Manske later, in 2001, made some very nice additions to the Nupedia system.
Some institutional traditions begin easily but die hard. So, in 2001, it was only after many months and uncomfortable comparison of Nupedia with the thriving, younger Wikipedia, that Nupedia's Advisory Board was willing to consider a simpler system seriously. That was because Nupedia editors and peer reviewers had a very strong commitment to rigor and reliability, as did I. Moreover, as Wikipedia became increasingly successful in 2001, Jimmy asked me to spend more and more time on it, which I did; Nupedia suffered from neglect. But by the summer of 2001, I was able to propose, get accepted (with very lukewarm support), and install something we called the Nupedia Chalkboard, a wiki which was to be closely managed by Nupedia's staff. It was to be both a simpler way to develop encyclopedia articles for Nupedia, and a way to import articles from Wikipedia. No doubt due to lingering disdain for the wiki idea--which at the time was still very much unproven--the Chalkboard went largely unused. The general public simply used Wikipedia if they wanted to write articles in a wiki format, while perhaps most Nupedia editors and peer reviewers were not persuaded that the Chalkboard was necessary or useful.
By early winter, 2001, Nupedia had published approved versions of only about 25 articles, although there were many more (I vaguely recall over 150 drafts) at various stages in process. I was finally able to persuade the Advisory Board to move the system to a much simpler two-step process, virtually identical to that used to run many academic journals: articles would be submitted to an editor; the editor would, if the article seemed good enough, forward it to a reviewer for acceptance or rejection; if accepted, the article would be posted. We also were thinking of various ways of allowing public comment on or moderated editing of posted articles. I believe this new, simpler system would have produced thousands of articles for Nupedia very quickly. The general public on Nupedia was certainly interested and motivated, and I think it was finally becoming generally accepted by the Advisory Board that the complexity of the system was the main reason that they were not starting articles and getting them through the system.
But, unfortunately, Nupedia's new system was never adopted when it should have been--the winter of 2001-2--because at the same time, Wikipedia was demanding as much attention as I could give it, and I had little time to implement the new Nupedia system. I am quite sure we could have started the new Nupedia system in early 2002, if we had made the time. But Bomis lost the ability to pay me and, newly unemployed, I did not have the time to lead Nupedia as a volunteer. I did not entirely lose hope on Nupedia, however, as I will explain below.
The origins of Wikipedia
In the fall of 2000, Jimmy and I were very well agreed that Nupedia's slow productivity was probably going to be an ongoing problem and that there needed to be a way, moreover, in which ordinary, uncredentialed people could participate more easily. Uncredentialed people could (and did) participate in Nupedia, particularly as writers and copyeditors, but it was pretty painful for most of them to get articles through the elaborate system. So there seemed to be a huge fund of talent, motivated to work on an encyclopedia but not motivated enough to work on Nupedia, going to waste.
It was my job to solve these problems. I wrote multiple detailed proposals for a simpler, more open editing system--two or three, at least--and I ran them by Jimmy, and I think his reply to all of them was that it would require too much programming and he couldn't afford to pay more high-priced programmers (they were very high-priced at the time, you will recall, and we already had Toan and Jason working quite a bit on Nupedia's new web-based system). Now, of course, I fully realize that we could have found a way to enlist volunteers to develop the system. Jimmy and I both probably knew that at the time; I'm not sure why we didn't pursue it.
So it was while I was thinking hard about how to create a more open system, that would require minimal programming to set up, that I had dinner with an old Internet friend of mine, Ben Kovitz. Ben had moved to town for a new job and we were out at a Pacific Beach Mexican restaurant on January 2, 2001, talking about jobs, techie stuff, and philosophy, no doubt. (Ben, Jimmy, and I were all active on those philosophy mailing lists in the mid-90s and we all knew each other.) So Ben explained the idea of Ward Cunningham's WikiWikiWeb to me. Instantly I was considering whether wiki would work as a more open and simple editorial system for a free, collaborative encyclopedia, and it seemed exactly right. And the more I thought about it, without even having seen a wiki, the more it seemed obviously right. So I'm sure it was that very evening or the following morning that I wrote a proposal--unfortunately, lost now--in which I said that this might solve the problem and that we ought to try it. After he had nixed my several earlier proposals, and given that setting up a wiki would be very simple and require hiring no programmer, Jimmy could scarcely refuse. I vaguely recall that he liked the idea but was initially skeptical--properly so, as I was, despite my excitement.
Wiki advocates often used to point out (and I'm sure some still do) that Wikipedia is nonstandard as a wiki. This is partly because we began just with the very basic wiki concept and not so much of the culture. Wiki culture is very distinctive. I cannot hope to explain even the highlights briefly, so I will not try; I will simply give a few notions. Wiki pages can be started and edited by anyone, but, in "Thread Mode" (as in "the thread of this discussion") the dialogue can become complex. In that case, or when consensus is reached, or when positions have hardened, it is considered a good idea to "refactor" pages (a term borrowed from programming), i.e., to rewrite them, but honestly, taking into account the highlights of the dialogue. Then the dialogue might be represented as in "Document Mode." Opinions are very welcome on a typical wiki. There are many other collective habits that make up typical wiki culture; these are only a few.
But I denied the necessity of organizing Wikipedia according to these precise principles. To be sure, a few other participants wanted Wikipedia to adopt wiki culture wholesale, so that it would be "just another wiki," and they had some small influence over the direction of the project; but speaking for myself, I viewed wiki software as simply a tool, a way to organize people who want to collaborate. I saw no necessity whatsoever of partaking in all aspects of the idiosyncratic culture that happened to be associated with the advent of this very generally-applicable tool, since we were engaged in a very specific sort of project, with very specific requirements. This caused some consternation among some wiki advocates, who appeared to think that Wikipedia should, or inevitably would, become just another wiki, somehow necessarily partaking of typical wiki culture. Ward Cunningham's prediction, when Jimmy asked him whether wiki software "could successfully generate a useful encyclopedia," was: "Yes, but in the end it wouldn't be an encyclopedia. It would be a wiki." As I said in reply: "Wikipedia has a totally different culture from this wiki, because it's pretty singlemindedly aimed at creating an encyclopedia. It's already rather useful as an encyclopedia, and we expect it will only get better."
Typical wiki culture aside, wiki software does encourage, but does not strictly require, extreme openness and de-centralization: openness, since (as the software is typically designed) page changes are logged and publicly viewable, and (again, only typically) pages may be further changed by anyone; de-centralization, because in order for work to be done, there is no need for a person or body to assign work, but rather, work can proceed as and when people want to do it. Wiki software also discourages (or at least does not facilitate) the exercise of authority, since work proceeds at will on any page, and on any large, active wiki it would be too much work for any single overseer or limited group of overseers to keep up. These all became features of Wikipedia.
My initial idea was that the wiki would be set up as part of Nupedia; it was to be a way for the public to develop a stream of content that could be fed into the Nupedia process. I think I got some of the basic pages written--how wikis work, what our general plan was, and so forth--over the next few days. I wrote a general proposal for the Nupedia community, and the Nupedia wiki went live January 10. The first encyclopedia articles for what was to become Wikipedia were written then. It turned out, however, that a clear majority of the Nupedia Advisory Board wanted to have nothing to do with a wiki. Again, their commitment was to rigor and reliability, a concern I shared with them and continue to have. Still, perhaps some of those people are kicking themselves now. They (some of them) evidently thought that a wiki could not resemble an encyclopedia at all, that it would be too informal and unstructured, as the original WikiWikiWeb was (and is), to be associated with Nupedia. They of course were perfectly reasonable to doubt that it would turn into the fantastic source of content that it did. Who could reasonably guess that it would work? But it did work, and now the world knows better.
Wikipedia's first few months
So we decided to relaunch the wiki under its own domain name. I came up with the name "Wikipedia," a silly name for what was at first a very silly project, and the newly independent project was launched at Wikipedia.com on January 15, 2001. It was a ".com" at first because, at the time, we were contemplating selling ads to pay for me, programmers, and servers. It was easy to deprecate ".com" in favor of ".org" in 2002, after Jimmy was able to assure users that Wikipedia would never (at least I think he said, or clearly implied, "never") run ads to support the project.
I took it to be one of my main jobs to promote Wikipedia, and this resulted in a steady influx of new participants. I wrote on the Wikipedia announcement page January 24, "Wikipedia has definitely taken [on] a life of its own; new people are arriving every day and the project seems to be getting only more popular. Long live Wikipedia!" By the end of January we reportedly and approximately had 600 articles; there were 1300 in March, 2300 in April, and 3900 in May. Not only was the project growing steadily, the rate of growth was increasing.
Wikipedia started with a handful of people, many from Nupedia. The influence of Nupedians was, I think, pretty important early on; I think, especially, of the tireless Magnus Manske (who worked on the software for both projects), our resident stickler Ruth Ifcher, and the very smart poker-playing programmer Lee Daniel Crocker--to name a few. All of these people, and several other Nupedia borrowings, had a good understanding of the requirements of good encyclopedia articles, and they were good writers and very smart. The direction that Wikipedia ought to go in was pretty obvious to myself and them, in terms of what sort of content we wanted. But what we did not have worked out in advance was how the community should be organized, and (not surprisingly) that turned out to be the thorniest problem. But the facts that the project started with these good people, and that we were able to adopt, explain, and promote good habits and policies to newer people, partly accounts for why the project was able to develop a robust, functional community and eventually to succeed. As to project leadership or management, we began with me, Jimmy, and Tim Shell; but Tim stopped participating so much after the first few months.
But the many rank-and-file users did the heavy lifting, and if there had not been a reasonable consensus among them about what the project should look like, it just wouldn't have happened. In any collaborative project, it is the contributors who are responsible for the outcome. Those early adopters should feel proud of themselves, because they were absolutely instrumental in shaping a thing of beauty and usefulness.
I recall saying casually, but repeatedly, in the project's first nine months or so, that experts and specialists should be given some particular respect when writing in their areas of expertise. They should be deferred to, I thought, unless there were some clear evidence of bias. (I recall an interesting discussion with a Polish scientist, Piotr Wozniak, about this issue when we came to a small disagreement about the "sleep" article.) So, in those first months, deference to expertise was a policy that at least I usually insisted upon, but not strongly or clearly enough. It was nearly a year after the project began that I finally articulated this view reasonably clearly as a policy to consider. Perhaps this was because, indeed, most users did make a practice of deferring to experts up to that time. "This is just common sense," as I wrote, "but sometimes common sense needs to be spelled out!" What I now think is that that point of common sense needed to be spelled out quite a bit sooner and more forcefully, because in the long run, it was not adopted as official project policy, as it could have been.
Some questions have been raised about the origin of Wikipedia policies. The tale is interesting and instructive, and one of the main themes of this memoir. We began with no (or few) policies in particular and said that the community would determine--through a sort of vague consensus, based on its experience working together--what the policies would be. The very first entry on a "rules to consider" page was the "Ignore All Rules" rule (to wit: "If rules make you nervous and depressed, and not desirous of participating in the wiki, then ignore them entirely and go about your business"). This is a "rule" that, current Wikipedians might be surprised to learn, I personally proposed. The reason was that I thought we needed experience with how wikis should work, and even more importantly at that point we needed participants more than we needed rules. As the project grew and the requirements of its success became increasingly obvious, I became ambivalent about this particular "rule" and then rejected it altogether. As one participant later commented, "this rule is the essence of Wikipedia." That was certainly never my view; I always thought of the rule as being a temporary and humorous injunction to participants to add content rather than be distracted by (then) relatively inconsequential issues about how exactly articles should be formatted, etc. In a similar spirit, I proposed that contributors be bold in updating pages (the current version is much expanded, as it should be).
I also, for similar reasons, specifically disavowed any title; I was organizing the project but I did not want to present myself as editor-in-chief. I wanted people to feel comfortable adding information without having to consult anything like an editor. Participation was more important, I felt. (Others referred to me, later, as Wikipedia's editor.)
As we set it up, Wikipedia did have some minimal wiki cultural features: it was wide open, extremely decentralized, and (provisionally anyway) featured very little attempt to exercise authority. Insofar as I was able to organize it at all, I guided the project through force of personality and what "moral authority" I had as co-founder of the project. Jimmy and I agreed early on that, at least in the beginning, we should not eject anyone from the project except perhaps in the most extreme cases. Our first forcible expulsion (which Jimmy performed) did not occur for many months, despite the presence of difficult characters from nearly the beginning of the project. Again, we were learning: we wished to tolerate all sorts of contributors in order to be well-situated to adopt the wisest policies. But--and in hindsight this should have seemed perfectly predictable--this provisional "hands off" management policy had the effect of creating a difficult-to-change tradition, the tradition of making the project extremely tolerant of disruptive (uncooperative, "trolling") behavior. And as it turned out, particularly with the large waves of new contributors from the summer and fall of 2001, the project became very resistant to any changes in this policy. I suspect that the cultures of online communities generally are established pretty quickly and then very resistant to change, because they are self-selecting; that was certainly the case with Wikipedia, anyway.
So I could only attempt to shame any troublemakers into compliance; without recourse to any genuine punitive action, that was the most I could do. In about the first eight months of the project, this was usually sufficient for me to do my job. After that, however, my job got increasingly difficult, as I will explain.
So Wikipedia began as a good-natured anarchy, a sort of Rousseauian state of digital nature. I always took Wikipedia's anarchy to be provisional and purely for purposes of determining what the best rules, and the nature of its authority, should be. What I, and other Wikipedians, failed to realize is that our initial anarchy would be taken by the next wave of contributors as the very essence of the project--how Wikipedia was "meant" to be--even though Wikipedia could have become anything we the contributors chose to make it.
This point bears some emphasis: Wikipedia became what it is today because, having been seeded with great people with a fairly clear idea of what they wanted to achieve, we proceeded to make a series of free decisions that determined the policy of the project and culture of its supporting community. Wikipedia's system is neither the only way to run a wiki, nor the only way to run an open content encyclopedia. Its particular conjunction of policies is in no way natural, "organic," or necessary. It is instead artificial, a result of a series of free choices, and we could have chosen differently in many cases; and choosing differently on some issues might have led to a project better than the one that exists today.
Though it began as an anarchy, there were quite a few policies that were settled upon, more or less, within the first six months or so. This required some struggle, especially on my part, particularly because, since the project was a wiki, some participants thought that there should be no rules at all. (Enforceable rules were regarded as "anti-wiki," which was supposed to be a bad thing.) But it was made clear from the beginning that we intended Wikipedia to be an encyclopedia, and so we were able to plug for at least those rules that would help define and sustain the project as an encyclopedia.
For instance, throughout the early months, people added various content that seemed less than encyclopedic in various ways. Many people seemed to confuse encyclopedia articles with dictionary entries, and eventually I wrote a page called "Wikipedia is not a dictionary." (I am surprised to discover that this page still exists as of this writing, with a good deal of its original content.) As people found new ways not to write encyclopedia articles, I started "What Wikipedia is not": I and others would note on an article's discussion page that some certain content did not belong in an encyclopedia, and then underscored the point by adding an entry to the "What Wikipedia is not" page. To take another example, Wikipedia was not to be a place for publishing original research. In fact, this is a policy that had been settled upon and even enforced in Nupedia days; enforcing it actually led to the departure of Nupedia's erstwhile Classics editor sometime in 2001.
Many of our first controversies were over these restrictions. At the time, I had enough influence within the community to get these policies generally accepted. And if we had not decided on these restrictions, Wikipedia might well have ended up, like many wikis, as nothing in particular. But since we insisted that it was an encyclopedia, even though it was just a blank wiki and a group of people to begin with, it became an encyclopedia. There is something very profound about that. I also like to think that we helped to show the world the potential that wikis have.
Another policy that was instituted early on was the nonbias or neutrality policy. This was borrowed from the Nupedia project and made a Rule to Consider--in a very early version, the policy was put this way:
Avoid bias: Since this is an encyclopedia, after a fashion, it would be best if you represented your controversial views either (1) not at all, (2) on *Debate, *Talk, or *Discussion pages linked from the bottom of the page that you're tempted to grace, or (3) represented in a fact-stating fashion, i.e., which attributes a particular opinion to a particular person or group, rather than asserting the opinion as fact. (3) is strongly preferred.
Jimmy then started a specialized policy page he called "Neutral Point of View" (here is the current version). I confess I don't much like this name as a name for the policy, because it implies that to write neutrally, or without bias, is actually to express a point of view, and, as the definite article is used, a single point of view at that. "Neutrality", "neutral", and "neutrally" are better to use for the noun, adjective, and adverb. But the acronym "NPOV" came to be used for all three, by Wikipedians wanting to seem hip, and then the unfortunate "POV" came to be used when the perfectly good English word "biased" would do.
In addition to these, I recall suggesting a number of other rules--no doubt most matters of historical fact, along these lines, can be verified in archives. I believe I am responsible for the original formulations of a lot of the article naming conventions, as well as the conventions of bolding the title of the article, starting articles with full sentences, making article titles uncapitalized, and much else. I think these policies were just a matter of common sense for anyone who understood what a good encyclopedia should be like. And of course I was not the only person proposing conventions. Moreover, actual project policy, or community habits, succeeded in being established only by being followed and supported by a majority of participants. It was then, we said, that there was a "rough consensus" in favor of the policy. And consensus, we said, is required for a policy actually to be considered project policy. For our purposes, a "consensus" appeared to consist of (1) widespread common practice, (2) many vocal defenders, and (3) virtually no detractors.
But that way of settling upon policy proposals--viz., by alleged consensus--did not scale, in my opinion. After about nine months or so, there were so many contributors, and especially brand new contributors, that nothing like a consensus could be reached, for the simple reason that condition (3) above was never achievable: there would after that always be somebody who insisted on expressing disagreement. There was, then, a non-scaling policy adoption procedure, and a crying need to continue to adopt sensible policies. This led to some pretty serious problems in the community, which I will relate below. But first, something more positive.
It's a cliff-hanger; you'll have to wait until tomorrow to read about what made Wikipedia start to work. -
Aggressive Network Self-Defense
nazarijo (Jose Nazario) writes "Continuing in the new theme of fiction and technical how-to, Aggressive Network Self-Defense brings together several authors to provide a wide range of material. Syngress' niche in this space seems to be breaking new ground -- and for the most part, it works. While you don't get as in-depth a treatment as a typical technical book gives you, there is an added dimension: namely, a more realistic scenario of how these tools fit together in a real, live series of actions." Read on for the rest of Nazario's review. Agressive Network Self-Defense author Neil Archibald, Seth Fogie, Chris Hurley, Dan Kaminsky, Johnny Long, Haroon Meer, Bruce Potter, Roelof Temmingh, Neil R. Wyler, Timothy Mullen pages 416 publisher Syngress rating 8 reviewer Jose Nazario ISBN 193183625 summary take your security into your own hands to identify, target, and nullify your adversariesNot being a big fan of most fiction (I tend to prefer history), it's hard to say definitively good or bad things about the quality of the writing. What I can say is that it's infinitely less irritating, and far more realistic, than Neal Stephenson's Cryptonomicon or Gibson's Neuromancer. No over-the-top smearing of adjectives to describe the mundane, and no unrealistic sequences of events. Then again, there's no character development and no real story progression, so it's not great fiction.
As a series of hacker vignettes, the book works just fine, and very well for the purposes at hand. Basically, what the authors want you to get from the book is two-fold: First, they want you to debate the issues around "strike back" attack methodologies. Several of the authors are open advocates of what are legal grey areas and open moral questions in the field of network security. Secondly, they want you to see how it's done, what you do when you actually use a tool to achieve a goal. Most books that do this, like Hacking Exposed, cover far more tools, but they usually do so without showing you each tool's use in a real-world scenario.
I won't bore you with a lengthy, detailed overview of the first part of the book. Like I said, it's a series of part fiction, part tutorial series of short stories. In them, you'll see tools like Metasploit, virus creation, some nmap, sniffers, and keystroke loggers, all in action, being used as an operator would use them, and achieving real goals. This is more valuable than a basic manual, and the stories themselves act as a nice setting. While not great fiction writers, the authors are decent enough at the job, and they write the technical material clearly.
The second part of the book is interesting. It makes up about a fifth of the book in volume, but a lot more in technical weight. The book bills this section as "The technologies and concepts behind network strike-back," and that's an accurate summary. It's a series of four unique perspectives and technical chapters that complement the rest of the book quite well.
The first introduces ADAM, the "Active Defense Algorithm and Model," which develops a methodology for network administrators to actively defend their networks against attacks. It's quite interesting, and brings together a number of risk models in an uncommon take. The authors are academic researchers from the University of Idaho, so it's a lot more academic than the previous material in Aggressive Network Self-Defense, but it formalizes a lot of the thinking that was present in the writing of the stories and techniques.
The second is Tim Mullen's classic "Defending your right to defend." This is the original position paper shared by Mullen with the information security community in 2002 or so. Here, Mullen makes a compelling case for actually striking back at worm infected hosts. After all, the position holds, someone should do something about them to help clean up the Internet. While it's a position I disagreed with at the time and still do, Mullen's writing is articulate and an important read. It really helps you understand a lot of the thinking that went into the book itself.
Dan Kaminsky wrote the next chapter, "MD5 to be considered harmful someday." Largely considered to be a follow-on to Joux and Wang's one-way hash function research, what it shows is how practical such an attack can be. Kaminsky never fails to come up with interesting ideas he puts into practice, and he adds another level of depth to this book.
Finally, Aggressive Network Self-Defense ends with an interesting paper, "When the tables turn: Passive strike-back." Like any good paper, it has a clear and thoughtful motivation, and really demonstrates the principles at play, namely building network resources that don't simply lure the attacker in, they trip her up. There are so many ways to do this, the authors show us, and ultimately it's almost fun. A good way to end the book.
An over-arching concern with the book that I have is the question of ethics. Mullen, in the foreword, states that he hopes the book stirs a debate about the ethics of the actions in the book. However, the book itself falls short in this area. Instead, sometimes the characters get busted, and sometimes they don't, but just because they didn't get caught doesn't mean some ethical lines weren't crossed. All too often the authors leave the ethical debate up in the air. While I prefer this to overt preaching or questions, the style leaves me wondering if this goal was achieved.
So, where do I stand on Aggressive Network Self-Defense? In the end, I like it, more so than a book like Hacking Exposed or other "hacking how-to" types. The style of presentation doesn't lend itself all that well to exploring a very wide number of tools, but it does give you a deeper context to see how they assemble into something larger. For many people I expect it will be a page turner, and I think the format has some utility, as shown here.
You can purchase Aggressive Network Self-Defense from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page. -
Linus Defends Proprietary File Formats [Updated]
Simon (S2) writes "Torvalds launched a blast against OpenOffice.org, and defended Microsoft's right to keep its binary Office formats proprietary. 'I'm happy with somebody writing a free replacement for Microsoft Office. But I'm not fine with them writing a free replacement just by reverse engineering the proprietary formats,' said the Linux founder. 'Microsoft has its own reasons for keeping them proprietary, and I can't argue with that.' At the heart of Torvalds' decision to refrain from using Bitmover's BitKeeper source code management tool last week, a day after BitKeeper decided to drop its limited functionality free client, is a dispute between BitKeeper developer Larry McVoy and Samba developer Andrew 'Tridge' Tridgell. It has subsequently emerged that Tridgell was working on a clean room reverse engineered implementation of McVoy's proprietary software, and Torvalds has come down on the side of his friend McVoy." Update: 04/13 17:24 GMT by T : As reader Daniel Callahan points out, this is a goof. "The Register article made up the Torvalds quote. The article offers the quote and then continues: 'Actually he didn't - we just made that quote up. But what Torvalds really did say this weekend is only slightly less bizarre.'" -
Bioinformatics in the Post-Genomic Era
nazarijo (Jose Nazario) writes "As a biochemist by training, Jeff Augen's Bioinformatics in the Post-Genomic Era was very interesting to me. Though I left the field some years ago, I was using the bioinformatics tools that are covered in the book daily and still look in from time to time. Naturally I was curious to see a larger perspective, as well as any progressions, that have occurred in the past few years. Augen's book gave me part of the larger picture, but it could have done more." Read on for the rest of Nazario's review. Bioinformatics in the Post-Genomic Era author Jeff Augen pages 388 publisher Addison-Wesley Longman rating 7 reviewer Jose Nazario ISBN 0321173864 summary Genome, Transcriptome, Proteome, and Information-Based MedicineBioinformatics is the science of biological information, namely sequences and metadata about organisms and sequences. What's interesting about this field to many people, both in the sciences and outside of it, is the large volume of data that gets analyzed and the results that emerge on a daily basis. Obviously interesting for the medical advances and the rapidly growing business in the life sciences, there's a complex field that has developed in the past ten years or so. And following the sequencing of the human genome, new challenges have arisen for everyone involved. Augen's Bioinformatics provides a good introduction to this new field of research for students in the sciences, and anyone with a decent undergraduate education in modern biology. I think that this accessibility of the material is one of the book's biggest winning points.
After an introduction to the book and the subject area of bioinformatics (chapters 1 and 2), Augen begins at the level of the structure of a gene (chapter 3). Here, anyone with an undergraduate level understanding of genetics or molecular biology can begin using the book and bridging the gap to the new areas of modern bioinformatics. Augen then describes how basic sequence analysis is performed at the DNA sequence level (in chapter 4). The material in Bioinformatics covers some of the higher-level methods for sequence analysis, including hidden Markov models, neural networks, and pattern discovery, and introduces some of the common algorithms found to do this analysis.
Chapter 5 then covers transcription, the process of going from DNA to mRNA. Beginning with the biology behind this activity (the ribosome and the larger "transcriptome"), Bioinformatics then describes how you would perform transcriptional analysis. Here, Augen shows how you go from a wet lab to a computational lab and describes what classes of experiments you perform to gather data and then what kinds of analysis you perform on it. This chapter introduces some of the more common clustering techniques for data aggregation and understanding.
The next step in the DNA -> RNA -> protein chain is found in chapter 6, which covers the translation process. Coupled to chapter 7, which describes protein structure prediction and searching, these two chapters bridge the next gap between laboratory data and computational analysis. Protein folding and structure analysis was one of my pet areas of study as a graduate student, and Augen's text does a decent summarization of the field to date. The resources listed and techniques described are definitely on par with the common practices in the field.
Finally, Bioinformatics gets into the next major area of bioinformatics, medical databases. Augen's bridge from genetics to medical science is complete, and he discusses how medical professionals utilize databases and can begin to predict disease, for example, based on data mining. The final chapter, "New Themes in Bioinformatics," covers exactly that, but also what Augen refers to as "workflow computing," or basically going about being a bioinformatics scientist. One of my favorite emerging areas in bioinformatics, metabolic pathway elucidation, is also covered briefly.
I've shared this book with a few friends who are all studying computer science or practicing computer scientists. I did so because Augen's material does a good job of explaining my background and introducing them to some of the analysis forms I introduce into my own work. It does a good job of that, and gets them quite excited. Bioinformatics really bridges a number of fascinating areas of computer sciences, including data mining and high performance algorithms. Augen's Bioinformatics is a good introduction to the field for them, and really anyone who has studied a couple of biology courses in college.
Where the book falls short, however, can be grouped into two main areas. The first is the failure of Augen's presentation of the algorithms. While the methods used to describe computational algorithms in Bioinformatics is common for non-computer scientists, it's completely unusable for computer scientists who are used to a specific algorithm presentation style that looks more like pseudocode than rambling text. The ambiguities this presents for a technical reader are unfortunate, especially if anyone studying bioinformatics is supposed to be computer science literate. The book itself assumes a life science literacy, so this isn't an unreasonable expectation of the reader.
The second area that consistently falls short in the book is in the utility of the information given. While I am significantly happier with the quality and depth of material presented in Augen's book than in the O'Reilly bioinformatics series, where the book fails to deliver is in showing the reader how to actually use the data they gather. After all, the book shows various sequence analysis algorithms and discusses tools available to do this work, but it only devotes a few pages (out of over 370 in total) to a workflow that can be used. Also, the book fails to point the reader at very worthwhile web resources sometimes, including meta sites like the SDSC Biology Workbench site, and just says "some Perl scripts" for local data analysis. As such, you'll have to go a few extra miles on your own to make use of the data sources.
I guess a third complaint of the book for me is that Augen has ignored or omitted significant bodies of research that fit squarely into the scope of the book. For example, Ken Dill's research into protein folding models, as well as Martin Karplus' work on the subject, receives no mention, nor does the topic of Bayesian network analysis when Augen discusses time series data analysis. These aren't new, they've been around for many years and influenced most of the field, and their absence is noted. The book's spotty coverage in some places, like these, is noticeable.
Bioinformatics does a few things well, but overall reads too much like a biology textbook to be useful to the average computer scientist. More emphasis on the practice of bioinformatics and data analysis would have made this book stronger and complemented the substantive background material well. Finally, using an approach more similar to the computer science approach would have been a tremendous benefit, since the material really is computer science in part. That said, I think this is probably the best introduction to this exciting area of science that I have yet seen.
You can purchase Bioinformatics in the Post-Genomic Era from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page. -
Humanoid Robot KHR-1 SDK Released
ls129 writes "KHR-1, the Japanese Robo-One humanoid robot kit from Kondo (previously mentioned here), is finally open for creative software development. The first unofficial implementation of the PC-controller API has just been posted. Using the SDK, the 17 servos that give the robot its mobility can be remote controlled by the PC via WiFi or bluetooth, and their current position can be retrieved several times per second. This unique feature will allow robot fans to go beyond simple performance of motion sequences or low-level gyro-based motion correction and develop algorithms that involve feedback control and AI." Update: 04/05 16:59 GMT by T : As originally posted, I erroneously changed the robot's nationality from Japanese to Korean; that was a boo-boo; the linked site with an English translation is Korean, but the robot itself is Japanese. Apologies to the submitter, who had it right. -
What Ever Happened to 'Toothing'?
Jim Hanas excertps from his very funny article on the quiet disappearance of last-year's promised digital bacchanal. "Remember 'toothing'? It was a craze that was sweeping England last year as bored commuters arranged sexual encounters using Bluetooth-enabled cellphones. You probably read about it over at Wired or Reuters or the BBC. There's a decent chance you even blogged about it. Well. What happened?" Update: 04/05 00:10 GMT by T : Hanas writes with a followup: "The original source on the whole toothing thing has just admitted it was a hoax -- in response to my email and your picking up of my post." -
Preview of Intel's Dual-Core Extreme Edition
ThinSkin writes "Intel let ExtremeTech.com sneak behind the curtain of its anticipated Dual-Core Pentium Extreme Edition processor for a full performance preview with benchmarks. Bundled with essentially two Prescott cores on one die, the Extreme Edition 840 processor clocks at 3.2GHz and contains a beefed-up power management system to keep the CPUs running cool during use. Expect Intel's dual-core line to hit the streets sometime this quarter. No word on pricing yet." Update: 04/04 17:26 GMT by T : Timmus points out FiringSquad's preview, too, writing "The benchmark results are mixed, with a few applications taking advantage of the new CPU, and some that don't." And Kez writes in reference to this article to say: "Our article on HEXUS.net, covering the P4 EE in detail, states the price as £650 (that's what we're looking at in the UK anyway, not sure about the U.S.)." -
Car Powered by Compressed Air
gripperzipper writes "CNN reports that a Korean company created a small car powered by compressed air. ENERGINE created its PHEV, or Pneumatic-Hybrid Electric Vehicle, which uses a two-stroke compressed air engine for start, acceleration, and uphill climbs. The car switches to an electric motor when its speed reaches 20-25 km/h (32-40 mi/h). Although major auto manufacturers have invested heavily in gasoline hybrids, it will be interesting to see if a market will open for this type of vehicle." Update: 04/04 17:18 GMT by T : Reader Tapsu spotted the incongruity here, writing "Interesting post, but the speed conversion has gone wrong way: "20-25 km/h (32-40 mi/h)". ... Thus the correct speed range in miles would be something like 12-15 mi/h." -
Senator Clinton Slams GTA
Joining the ranks of such luminaries as Jack Thompson and Governor Blagojevich, GamesIndustry.biz has the word that Senator Hillary Clinton has joined right wing advocates in decrying the gaming industry as a paragon of loose morals and corrupting influences. From the article: "Children are playing a game that encourages them to have sex with prostitutes and then murder them...This is a silent epidemic of media desensitisation that teaches kids it's OK to diss people because they are a woman, they're a different colour or they're from a different place." Commentary available at The Australian. Update: 03/30 02:22 GMT by T : Thanks to reader mantle_etching, here is a link to the entire speech as delivered, so you can judge its content for yourself.