Slashdot Mirror

Craig Reynolds writes "In his recent book Artificial Intelligence for Computer Games: An Introduction , author John Funge takes us on a whirlwind tour of techniques from the literature of academic AI research and discusses their application to the nuts and bolts of game AI programming. While some of these topics are quite advanced, the text remains easily readable and grounded in what the techniques mean to real game programmers developing real game AI." Read on for Reynolds' review. Artificial Intelligence for Computer Games: An Introduction author John David Funge pages 160 publisher A K Peters rating 8 reviewer Craig Reynolds ISBN 1568812086 summary Written for game AI programmers, this book provides a practical introduction to advanced AI techniques and practices for constructing sophisticated non-player characters.

Funge's background includes both academic AI research and commercial development of game AI technology. This has allowed him to write a refreshingly practical book for the game AI programmer which will also expand the reader's knowledge of AI. He presents advanced AI research in a way that is meaningful to the working game AI programmer. Non-player characters (NPCs) are the focus of this book, although it touches upon techniques applicable to other kinds of AI. Funge begins with a simple NPC architecture, then goes on to consider how they act in their world, perceive and react to their surroundings, remember their past experiences, plan their actions, and learn from the past to improve their future behavior. In addition, Funge hopes his book will contribute to a "common framework and terminology" to promote better communication between practitioners interested in game AI, leading to better interoperability for their software. (Please note that John Funge is a friend and former coworker of mine. I was pleased to accept John's invitation to review his book.)

The field of Artificial Intelligence has been actively studied since the 1950s. In that half century many useful techniques have been developed and applied to a broad range of scholarly and commercial applications -- most quite serious and sometimes a bit dry. In contrast, today the most economically significant application of AI is in computer games. This commercial application motivates today's students to study AI and drives a good deal of academic AI research. Modern games have incredible graphics and their animation technology is becoming very sophisticated. As graphic animation increasingly becomes a solved problem, more and more attention is being paid to game AI. It seems likely that the next few years will see a tremendous investment in game AI technology leading to significant improvements in the state of the art.

As I read Funge's book I was struck by how oriented it was to the interests of AI programmers working on commercial games. Certainly the discussion focused on the practical rather than the theoretical. (There are many asides, footnotes and citations of the academic literature for those with an interest in pursuing the theory.) More concretely, the text is peppered with fragments of C++ code. A working programmer who visits the academic literature is often faced with the daunting task of converting prose, equations or breezy pseudo-code into something suitable for compilation. If a reader of this book does not follow a bit of the discussion, a glance at the nearby C++ code listing will usually set things straight. I have it on good authority that functioning source code for the examples in the book will appear on the www.ai4games.org website "soon."

The book is divided into seven chapters (Introduction, Acting, Perceiving, Reacting, Remembering, Searching, and Learning) plus a Preface, two appendices, an extensive Bibliography and an Index. The chapter on "Acting" introduces the simple game of tag used as an example throughout the book. It further sets the stage by describing the principal components of the game engine and the AI system. The third chapter, "Perceiving," introduces percepts -- the formal framework used to encapsulate and manipulate an NPC's awareness of its world. In many games a key concept is filtering out information which is available in the game state but should not be "known" by the NPC. Chapter 4 describes reactive controllers. Funge uses a very strict definition of reactive -- informally, it means a non-deliberative controller, but in this book the term is used to mean strictly stateless. This distinction has a practical consequence since a stateless controller can be shared among multiple NPCs. (Yet I wondered how important this was in practice. That point was not explored in any depth, and a "slightly stateful" reactive controller can be very useful.) The chapter on "Remembering" introduces memory percepts, mental state, beliefs and communication between NPCs. The sixth chapter covers "Searching" -- through trees of possible future actions, often referred to as planning. The extensive treatment of search includes both examining the host of options that are available to an NPC at each juncture, as well as reasoning about the interaction of one NPC's behavior with another, known as adversarial search. The final chapter covers "Learning." It looks at both offline learning (which happens before the game is shipped) and online learning (happening during gameplay). The first is merely an aid to game development, the latter promises NPC that can adjust to the player's skill and style of play. Online learning present many more technical challenges. In fact, my first impression on reading this section that it was less practical than the rest of the book because of the difficulties of online learning. However, from the description of this GDC 2005 lecture, it appears that Funge and his colleagues have made significant progress in this area.

I recommend Artificial Intelligence for Computer Games: An Introduction to commercial game AI programmers, as well as other game programmers and designers who wish to learn more about this area. Because of its sound academic underpinning, the book will also be of interest to students of artificial intelligence and to professionals in related areas such as agent-based simulation and training.

Reynolds is a Senior Research Scientist in the R&D group of Sony Computer Entertainment America. His interests center on modeling behavior of autonomous characters, particularly steering behaviors for agile life-like motion through their worlds. See his page on Game Research and Technology. You can purchase Artificial Intelligence for Computer Games: An Introduction from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Craig Reynolds writes "In his recent book Artificial Intelligence for Computer Games: An Introduction , author John Funge takes us on a whirlwind tour of techniques from the literature of academic AI research and discusses their application to the nuts and bolts of game AI programming. While some of these topics are quite advanced, the text remains easily readable and grounded in what the techniques mean to real game programmers developing real game AI." Read on for Reynolds' review. Artificial Intelligence for Computer Games: An Introduction author John David Funge pages 160 publisher A K Peters rating 8 reviewer Craig Reynolds ISBN 1568812086 summary Written for game AI programmers, this book provides a practical introduction to advanced AI techniques and practices for constructing sophisticated non-player characters.

Funge's background includes both academic AI research and commercial development of game AI technology. This has allowed him to write a refreshingly practical book for the game AI programmer which will also expand the reader's knowledge of AI. He presents advanced AI research in a way that is meaningful to the working game AI programmer. Non-player characters (NPCs) are the focus of this book, although it touches upon techniques applicable to other kinds of AI. Funge begins with a simple NPC architecture, then goes on to consider how they act in their world, perceive and react to their surroundings, remember their past experiences, plan their actions, and learn from the past to improve their future behavior. In addition, Funge hopes his book will contribute to a "common framework and terminology" to promote better communication between practitioners interested in game AI, leading to better interoperability for their software. (Please note that John Funge is a friend and former coworker of mine. I was pleased to accept John's invitation to review his book.)

The field of Artificial Intelligence has been actively studied since the 1950s. In that half century many useful techniques have been developed and applied to a broad range of scholarly and commercial applications -- most quite serious and sometimes a bit dry. In contrast, today the most economically significant application of AI is in computer games. This commercial application motivates today's students to study AI and drives a good deal of academic AI research. Modern games have incredible graphics and their animation technology is becoming very sophisticated. As graphic animation increasingly becomes a solved problem, more and more attention is being paid to game AI. It seems likely that the next few years will see a tremendous investment in game AI technology leading to significant improvements in the state of the art.

As I read Funge's book I was struck by how oriented it was to the interests of AI programmers working on commercial games. Certainly the discussion focused on the practical rather than the theoretical. (There are many asides, footnotes and citations of the academic literature for those with an interest in pursuing the theory.) More concretely, the text is peppered with fragments of C++ code. A working programmer who visits the academic literature is often faced with the daunting task of converting prose, equations or breezy pseudo-code into something suitable for compilation. If a reader of this book does not follow a bit of the discussion, a glance at the nearby C++ code listing will usually set things straight. I have it on good authority that functioning source code for the examples in the book will appear on the www.ai4games.org website "soon."

The book is divided into seven chapters (Introduction, Acting, Perceiving, Reacting, Remembering, Searching, and Learning) plus a Preface, two appendices, an extensive Bibliography and an Index. The chapter on "Acting" introduces the simple game of tag used as an example throughout the book. It further sets the stage by describing the principal components of the game engine and the AI system. The third chapter, "Perceiving," introduces percepts -- the formal framework used to encapsulate and manipulate an NPC's awareness of its world. In many games a key concept is filtering out information which is available in the game state but should not be "known" by the NPC. Chapter 4 describes reactive controllers. Funge uses a very strict definition of reactive -- informally, it means a non-deliberative controller, but in this book the term is used to mean strictly stateless. This distinction has a practical consequence since a stateless controller can be shared among multiple NPCs. (Yet I wondered how important this was in practice. That point was not explored in any depth, and a "slightly stateful" reactive controller can be very useful.) The chapter on "Remembering" introduces memory percepts, mental state, beliefs and communication between NPCs. The sixth chapter covers "Searching" -- through trees of possible future actions, often referred to as planning. The extensive treatment of search includes both examining the host of options that are available to an NPC at each juncture, as well as reasoning about the interaction of one NPC's behavior with another, known as adversarial search. The final chapter covers "Learning." It looks at both offline learning (which happens before the game is shipped) and online learning (happening during gameplay). The first is merely an aid to game development, the latter promises NPC that can adjust to the player's skill and style of play. Online learning present many more technical challenges. In fact, my first impression on reading this section that it was less practical than the rest of the book because of the difficulties of online learning. However, from the description of this GDC 2005 lecture, it appears that Funge and his colleagues have made significant progress in this area.

I recommend Artificial Intelligence for Computer Games: An Introduction to commercial game AI programmers, as well as other game programmers and designers who wish to learn more about this area. Because of its sound academic underpinning, the book will also be of interest to students of artificial intelligence and to professionals in related areas such as agent-based simulation and training.

Reynolds is a Senior Research Scientist in the R&D group of Sony Computer Entertainment America. His interests center on modeling behavior of autonomous characters, particularly steering behaviors for agile life-like motion through their worlds. See his page on Game Research and Technology. You can purchase Artificial Intelligence for Computer Games: An Introduction from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Craig Reynolds writes "In his recent book Artificial Intelligence for Computer Games: An Introduction , author John Funge takes us on a whirlwind tour of techniques from the literature of academic AI research and discusses their application to the nuts and bolts of game AI programming. While some of these topics are quite advanced, the text remains easily readable and grounded in what the techniques mean to real game programmers developing real game AI." Read on for Reynolds' review. Artificial Intelligence for Computer Games: An Introduction author John David Funge pages 160 publisher A K Peters rating 8 reviewer Craig Reynolds ISBN 1568812086 summary Written for game AI programmers, this book provides a practical introduction to advanced AI techniques and practices for constructing sophisticated non-player characters.

Funge's background includes both academic AI research and commercial development of game AI technology. This has allowed him to write a refreshingly practical book for the game AI programmer which will also expand the reader's knowledge of AI. He presents advanced AI research in a way that is meaningful to the working game AI programmer. Non-player characters (NPCs) are the focus of this book, although it touches upon techniques applicable to other kinds of AI. Funge begins with a simple NPC architecture, then goes on to consider how they act in their world, perceive and react to their surroundings, remember their past experiences, plan their actions, and learn from the past to improve their future behavior. In addition, Funge hopes his book will contribute to a "common framework and terminology" to promote better communication between practitioners interested in game AI, leading to better interoperability for their software. (Please note that John Funge is a friend and former coworker of mine. I was pleased to accept John's invitation to review his book.)

The field of Artificial Intelligence has been actively studied since the 1950s. In that half century many useful techniques have been developed and applied to a broad range of scholarly and commercial applications -- most quite serious and sometimes a bit dry. In contrast, today the most economically significant application of AI is in computer games. This commercial application motivates today's students to study AI and drives a good deal of academic AI research. Modern games have incredible graphics and their animation technology is becoming very sophisticated. As graphic animation increasingly becomes a solved problem, more and more attention is being paid to game AI. It seems likely that the next few years will see a tremendous investment in game AI technology leading to significant improvements in the state of the art.

As I read Funge's book I was struck by how oriented it was to the interests of AI programmers working on commercial games. Certainly the discussion focused on the practical rather than the theoretical. (There are many asides, footnotes and citations of the academic literature for those with an interest in pursuing the theory.) More concretely, the text is peppered with fragments of C++ code. A working programmer who visits the academic literature is often faced with the daunting task of converting prose, equations or breezy pseudo-code into something suitable for compilation. If a reader of this book does not follow a bit of the discussion, a glance at the nearby C++ code listing will usually set things straight. I have it on good authority that functioning source code for the examples in the book will appear on the www.ai4games.org website "soon."

The book is divided into seven chapters (Introduction, Acting, Perceiving, Reacting, Remembering, Searching, and Learning) plus a Preface, two appendices, an extensive Bibliography and an Index. The chapter on "Acting" introduces the simple game of tag used as an example throughout the book. It further sets the stage by describing the principal components of the game engine and the AI system. The third chapter, "Perceiving," introduces percepts -- the formal framework used to encapsulate and manipulate an NPC's awareness of its world. In many games a key concept is filtering out information which is available in the game state but should not be "known" by the NPC. Chapter 4 describes reactive controllers. Funge uses a very strict definition of reactive -- informally, it means a non-deliberative controller, but in this book the term is used to mean strictly stateless. This distinction has a practical consequence since a stateless controller can be shared among multiple NPCs. (Yet I wondered how important this was in practice. That point was not explored in any depth, and a "slightly stateful" reactive controller can be very useful.) The chapter on "Remembering" introduces memory percepts, mental state, beliefs and communication between NPCs. The sixth chapter covers "Searching" -- through trees of possible future actions, often referred to as planning. The extensive treatment of search includes both examining the host of options that are available to an NPC at each juncture, as well as reasoning about the interaction of one NPC's behavior with another, known as adversarial search. The final chapter covers "Learning." It looks at both offline learning (which happens before the game is shipped) and online learning (happening during gameplay). The first is merely an aid to game development, the latter promises NPC that can adjust to the player's skill and style of play. Online learning present many more technical challenges. In fact, my first impression on reading this section that it was less practical than the rest of the book because of the difficulties of online learning. However, from the description of this GDC 2005 lecture, it appears that Funge and his colleagues have made significant progress in this area.

I recommend Artificial Intelligence for Computer Games: An Introduction to commercial game AI programmers, as well as other game programmers and designers who wish to learn more about this area. Because of its sound academic underpinning, the book will also be of interest to students of artificial intelligence and to professionals in related areas such as agent-based simulation and training.

Reynolds is a Senior Research Scientist in the R&D group of Sony Computer Entertainment America. His interests center on modeling behavior of autonomous characters, particularly steering behaviors for agile life-like motion through their worlds. See his page on Game Research and Technology. You can purchase Artificial Intelligence for Computer Games: An Introduction from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

NTK was kind enough to point out the Inkulator 9000, software to render pen-and-ink style drawings from 3D meshes. NTK also points to a number of other handy tools and papers.

Nat Friedman often seems to live in the shadow of his famous coworker, Miguel de Icaza, but today it's his turn to shine. You asked Nat questions last week. This week he answers, in detail, with lots of links, touching on subjects ranging from Gnome's future directions to how Microsoft is dealing with Linux as a competitor to Windows. 1) Exchange Like Product
by Kaypro

Currently the Exchange Connector seems to integrate quite well, are there any plans to create a standalone server with similar capabilities to Exchange Server?

Nat:

There are no plans today, but it's a really appealing idea.

Ximian's goal is to enable corporations to deploy and use open source-based desktops. One of the major barriers to this happening today is interoperability with the rest of the corporate computing environment. In the world we all inhabit, that means interoperability with Microsoft products.

When we were doing some product planning and market research early last year we found all these cases in big companies where people had to have two computers on their desk: a Unix machine for their real work -- development, CAD/CAM, 3d rendering, etc -- and a Windows machine so that they could speak all the protocols and file formats that the rest of the office speaks. And we were like: this just ain't right.

In many of these cases, when we asked people, they said that what was keeping that Windows machine on their desk was not, as we expected them to say in all cases, Word or Excel or Powerpoint, but it was actually in many cases Outlook. What happens is that the IT department will proclaim from on high that Exchange is the corporate scheduling standard, and if you ever want to coordinate a meeting or to schedule time in a room or with a projector or any other resource, you have to use Exchange, or you're simply out of the loop.

So this was a situation where providing this functionality under Linux eliminated the need for that Windows machine. This is a clear financial win for the customer and a clear win for the open source desktop. Basically, the Connector was a really obvious product to build.

Will we ever build a collaboration server of our own? It is something we've had some requests for before, and of course we're always listening to our customers and users, but we have no plans to build one today. Tell you what, if you would be interested in paying for such a thing, send email to sales@ximian.com and let us know. :-)

2) Microsoft and Mono?
by zoward

Have you gotten a sense of how Microsoft views the existence of an open source alternative to .NET? Do you think that, over the long term, Microsoft will grow to love, ignore or loathe (and perhaps seek to undermine) Mono?

Nat:

Open source software is a threat to Microsoft's business model, and it is a competitor which they cannot attack with their traditional maneuvers. At the same time, the events of the past seven years, especially the emergence of the web, Linux, Java and XML, have shown Microsoft the marketplace power of open standards. For these reasons, Microsoft's posture toward Mono and similar projects can be hard to gauge.

But the fact is, Linux and other open source efforts are a source of competition for Microsoft, and that is why they are investing 25 million dollars with Unisys to discredit Unix: they are once again facing competition, but this time there is a united front of users and companies around the globe that opposes them. Open source has given the world a common ground.

At the O'Reilly Developer Conference last year on a panel with Michael Tiemann, Tim O'Reilly, and others, Craig Mundie, Microsoft's CTO of Advanced Strategies and Policy, said (I am paraphrasing): "The thing Microsoft does not like about the GPL is that it creates a closed community." Yes, he actually said this, and while the entire audience sat stunned and struggling for oxygen, I remember Tim O'Reilly did not miss a beat, responding with "But so does Microsoft!"

Mono is an open source implementation of the C#, CLR and CLI cross-platform development framework that have been submitted to ECMA for standardization. We are implementing this framework because we believe it is important technology, and that the world should have a free, standards-compliant version of it.

Microsoft wants the ".NET platform" to be adopted, which is why they submitted it to ECMA. Whether or not Microsoft will change their minds, retract their submission, and decide that they do not like Mono is not something I can predict, but if they do, we are ready to adapt to the change and ensure that this technology is available to the world.

3) Core Gnome technologies
by wrinkledshirt

Despite its relatively short lifetime, Gnome's been really great about embracing all sorts of different technologies -- gtk, ORBit, bonobo and now Mono. However, it's sometimes difficult trying to figure out how this all ties together (if it's supposed to at all). Generally speaking, if someone's going to want to develop for Gnome in the future, how should they prepare themselves? What should they want to learn?

Nat:

Actually, the goal of the infrastructural work in GNOME is to abstract all of the underlying technologies away from you so that you can focus on writing your application. We want you to feel the joy of being able to sit down and easily build something, not to hand you a whole bunch of new stuff to learn.

Nowadays GNOME application development can be done rapidly and easily using Python or Perl and the Glade GUI construction tool.

For a lot of people, these languages and tools are the best way to build an application. The GtkPerl site has an example of a GNOME panel applet written in just 60 lines of Perl (and I'm sure it could be done in less). Not everyone knows that Anaconda, the Red Hat Linux installer is actually written using PyGtk.

Using Glade to create your user interfaces not only frees you from the arduous task of manually doing all of the widget creation and packing, it also makes your application more flexible because the GUI layout is loaded at run-time from an XML file. For the GNOME project this has been really helpful, since it means that a lot of UI design and prototyping work can be done without the need to even touch the code.

If you want to learn more, developer.gnome.org has a pretty good overview of the GNOME architecture.

All of the GNOME technologies that you've heard about work under the hood to provide consistency, configurability, and scripting features that you, as a programmer, only come into contact with if you need them. The goal, to steal directly from Larry Wall, is to make the easy things easy and the hard things possible.

For example, you might (or might not) have heard of Atk, Gail and at-spi. These are accessibility ("a11y") technologies that are in GNOME 2 to make it possible for applications to be used by people with various kinds of impairments. But you do not need to be exposed to any of the details of CORBA in order to use them, and in fact, some of the a11y features come for free just from building your application using GNOME 2.

By the way, I happen to think that accessibility is a killer feature in GNOME 2. At GUAD3C, Marc Mulcahy gave a great demo of how a sightless person can navigate the desktop using a screen reader. And we have been working on a set of accessible icons for GNOME 2 as well. There are cool side effects too: Because GNOME's accessibility infrastructure is done programmatically and at the widget level, you can actually attach to a remote running application and introspect and act on its widget tree. This may make it possible for us to eventually have a very high-quality automated UI testing tool.

Check out the GNOME Accessibility Project web page for more information.

As for Mono, it is still a technology under development, and the GNOME project has not made a decision to adopt it in any way yet. Work on C# bindings for Gtk is progressing, however, so you will be able to write Gtk and, eventually, GNOME applications in C#.

4) Usability research
by nakhla

One of the big problems facing GNOME and other open-source software is that of ease-of-use. Microsoft and Apple spend millions of dollars when developing new operating systems or UIs in order to ensure that their product is easy to use for the non-geek end user. What kind of useability studies has Ximian conducted? What is Ximian doing to correct any problems that the research has brought to light?

Nat:

Ximian and the GNOME project have learned from standard, existing industry practices for building usable software. In short this means designing for usability, performing formal usability testing on real users, and treating usability problems as first-class bugs.

The GNOME Usability Project is a nice central resource for a lot of the usability work that has gone into GNOME. Recently the project has been making a lot of progress on the GNOME Human Interface Guidelines, a set of UI rules that will help GNOME achieve much better consistency in its user interfaces. The results of the comprehensive GNOME desktop usability study that Sun performed last year are worth a read, too, even if we've already overcome a lot of that stuff in GNOME 2.

In the course of the design of Evolution 1.0 and 1.2 (due out this summer), Anna Dirks, our UI designer, performed many dozens of usability tests on various parts of Evolution, using a wide variety of people with varying degrees and types of experience using computers. Anna delivered a nice talk on the usability testing process at the GUADEC Conference

An application's usability is directly related to the ease with which a user can predict its behavior when he gives it input. This is why usability testing is a productive activity. In its basic form, it goes like this:

1. Create a prototype of the interface you are designing. In some cases prototypes are created using "scripting" languages or "RAD" tools, and sometimes they are just printed onto "paper." This last type is called a "paper prototype," the name deriving from the "paper" on which it is printed, and the fact that it is a prototype.

2. Coerce an appropriately representative set of individuals into participating in the usability test. The use of lethal force may be necessary.

3. Ask the user to perform a certain task, using the prototype.

4. Observe and record the steps the user takes, with particular attention to his mistakes.

5. Rinse, lather, repeat.

The fundamental premise of the usability test is that the user has certain expectations of how a given interface will behave, and the thing that a designer must do is to identify the places where his interface does not conform to those expectations, and to fix them.

At Ximian we've gotten subjects for our usability tests from a variety of places; there's a movie theater downstairs from our office and sometimes we'll hang out there and offer people free movie passes to participate in usability tests. So we get a pretty broad audience.

All usability issues that arise during a usability test are filed as bugs in bugzilla alongside other issues, and of course the subject's comments inform the revised design of the interface in question.

For GNOME 2, we decided to revamp all of the GNOME stock icons to improve their consistency, usability and to brighten up the style a bit. Ximian has contributed all of these new icons back to GNOME; you can check them out on developer.ximian.com.

Havoc recently wrote a nice piece which covers UI design in free software, and in GNOME in particular.

5) Conflict of Philosophies
by polyphemus-blinder

I would like to know:

What is your take on the apparent paradox resulting from:

1. the goal of uniformity on the Linux desktop, and

2. the many, many, groups who have this as their own special goal?

Mandrake and RedHat work toward this on the OS level, and Gnome and KDE battle it out on the desktop integration level, and many others espouse some sort of a "grand unification theory" of Linux.

Do you subscribe to the theory that less is more, or that multiple groups with a common goal will result in the goal's earlier acheivement?

Nat:

In any large-scale human endeavor, consistency is a very difficult goal. I once heard a senior Microsoft project manager express the goal of consistency in software thusly: "A program should look as if it were written by one person." This is a thing that everyone struggles with.

To give you a short summary of my answer:

(1) Consistency is hard.

(2) Decentralization and parallel development are inherent to open source software.

(3) Open standards and making an effort to work together are key. Let's try to do more of that.

Consistency in software applications means fewer surprises, a gentler learning curve, and being able to get your work done without tripping over an application's special quirks along the way. This is especially true of the interfaces that the application exposes.

For human interfaces, consistency means that the elements of the application do what the user expects them to do, and that the interface, consequently, does not get in the user's way. This means that a dialog's Close button is always in the same place, the menubar always appears at the top of the window, and Ctrl-Q always quits. Usability flows predictability which flows from consistency.

For programming interfaces, or APIs, consistency means that the methods you invoke have predictable characteristics: similar naming, the same memory management semantics, the same return values in an error condition. This means cleaner code, less time spent hunting through documentation, and fewer bugs.

So we can agree that consistency is a good thing. Two things are needed to achieve it: a standard, and a way to enforce that standard.

In more centralized environments, such as companies, these things are easier to do. It is naive to think that any company, even Microsoft is fully centrally controlled, but it is certainly much easier to enforce a single standard on people when you are paying them, and when you have editorial control over the final product.

But even with a single, documented standard and even if you are paying people's salaries, consistency does not come easily, even in the most centralized environments. At one point Microsoft had at least nine separate internal implementations of SOAP, and only recently have these all been consolidated...into four.

So how on earth do we achieve consistency in a decentralized environment? Given that starting your very own open source "project" is a matter of a few clicks on sourceforge, how do we "prevent" people from creating applications that do not adhere to some common set of ideas as to how they should behave? Given that there is no central control of what happens in the open source desktop world, how can we even create a standard that we all agree on?

I remember when Mac OS X first came out, people asked a lot of similar questions: How can we ever create an interface that is as consistent as this in our weirdo free code, free love, gift economy, bazaar-inspired noospheric environment?

This question can be considered at different scales: how can consistency be achieved within a single project, and how can it be achieved in the open source world in general.

And this issue of decentralized development comes up in other guises as well. In addition to bemoaning a lack of consistency, people talk about duplication of effort and fragmentation. They say things like: "If only we could focus all of the energy that has gone into producing all of the IRC clients in the world on building just one IRC client, think how awesome it could be!" People really say this sort of thing. I have heard them.

And, of course, there are those in the press and on the mailing lists who see this very same pattern in what they call the "GNOME vs KDE wars" or "the desktop wars." This is the "How many Linux distributions can you count?" conundrum.

Many people who are much smarter and better looking than I have responded to this question at various times.

Linus has said that he believes that in the Linux development community today, there is a "psychological barrier to fragmentation," and that this barrier is the learned result of the Unix wars of the 1980s.

Alan Cox has said that implementation fragmentation is not important, as long as care is taken not to break interface compatibility. The important thing, quoth Alan, is the existence and adherence to open standards. And Eric Raymond has pontificated at length about how it is the nature of the open source community, when confronted with a problem to solve, to try "all solutions at the same time." That is, I think Eric would tell you, the nature of the open source world, and, in many ways, its greatest strength. And of course, Eric is right. Seriously, I love that guy.

If on an iron-gray fall day you have looked up and seen a dark spot moving against the sky and changing shape and size but still moving smoothly in one direction and then it came closer and when you looked you could see the individual birds flapping their wings and shifting forward and back in the formation and alternately turning against and away from each other but still somehow moving all together as one mass, I think you have seen something that resembles the greater open source development community, if there can be said to be such a thing.

The thing that the birds are doing is called "flocking," and today the problem of flocking is still an interesting issue in algorithmic circles. The basic scenario is that, with each element in the flock making its own individual movement decisions based on its own individual and unique sensory input of what is happening immediately around it, the flock must somehow move along a single path, as a whole. The analogy of the Boids flocking algorithm actually runs deeper than you might expect; check it out sometime.

What is important in open source software is doing the actual human work of getting people together and creating the open standards that will allow us to function as a group, and to move in the same direction. And the way to do that is through open, shared standards.

I'm not talking about a kind of abstract standards process where an aesthete group of monks argues for centuries in the thin mountain air about file system standards before descending with etched tablets, but a process where implementors agree on good-enough standards of existing practices in the places that matter, today. Standardization is a way for us to align our directions, maintain implementation distance, and follow a common flight path, not an end in and of itself.

The thing to recognize is that the problem of creating a consistent desktop experience and the fact that our approach is a multi-pronged, decentralized, evolutionary one do not have to be at odds with each other. The key to consistency is to work toward it.

6) As a business
by Fizzlewhiff

Is it frustrating to see potential revenue lost due to offering the same products for free? Do you ever run the numbers to see what your income potential might be if you stopped giving away the same software you sell or do you believe that the Linux community, as a whole, cannot and will not support companies who only sell Linux software?

Nat:

If in the last two years we hadn't put out approaching 2 million lines of GPL'd and LGPL'd code, we would not have nearly the success that we have today.

If you're going to run those kinds of numbers, you should also calculate:

1. How much extra would you have to spend on development in order to compensate for the fact that you will no longer have the help of a large community of testers, translators and hackers?

2. How much do you have to spend on marketing to even reach the same level of name recognition you can achieve by being a responsible, active open source software development company? Would you have the same amount of credibility?

During the several months that preceded Evolution 1.0, we averaged around 10,000 daily downloads of the Evolution snapshots, and many of the downloaders were actively reporting and fixing the bugs that they found. How much would it have cost us to manually test Evolution against the wide variety of IMAP, LDAP and Palm devices that the Evolution codebase was exposed to by this army of users?

This kind of thinking may sound cold and not particularly ideological, but if you're going to perform one kind of calculation, you gotta do them all. I have actually heard of open source companies sitting down and working out the second, marketing calculation, and including it in their business plans as a rationale for writing free code.

7) Co-existance of Red-Carpet and up2date/RHN
by yusufg

Hi, Red-Carpet seems to offer functionality similar to up2date/redhat network. However, there seems to be a very substantial lag between packages made available via Ximian's redhat channel and up2date.

An example being (till now, RPM 4.0.4) is not available via the Redhat 7.2 channel. Is Ximian going to ever make a policy statement as to what is the maximum duration their userbase will be diverged from receiving the latest updates of their respective distributions.

If there are specific packages which are likely not to be made available via red-carpet, can their be an official statement on this so that users are aware of the pros/cons of using multiple update mechanisms?

Nat:

Our policy is that all distribution and third-party updates are made available through Red Carpet as soon as they can reasonably be pushed without breaking other software for the user.

For example, with security updates, these are always made available as soon possible, often within just a few hours, always within a day.

With something like the RPM 4.0.4 update, however, sometimes we have to lag behind the upstream provider, in order to ensure compatibility. This does not mean that we hate Red Hat or that we do not care about users, or that we are lazy.

In the particular case of RPM, new releases of RPM often break binary or database compatibility with old versions (this was true with 4.0.4), and so we are cautious about making these available to users until we have first ensured that Red Carpet will continue to work on your system. I am not trying to pass the buck to Red Hat here. They are great people. Our userbase, in running Red Carpet, just happens to have a different set of needs than Red Hat's, and this is what, in the case of RPM 4.0.4, created the delay you noticed.

To answer your second question, as long as the packages that are shipped by the upstream providers are open source, and as long as we can legally redistribute them, we will make them available via Red Carpet.

8) Lack of documentation for GNOME internals
by Tet

Are there any plans to increase the amount of documentation on GNOME internals? While GNOME seems to have plenty of trivial documentation (such as the GNOME User's Guide [redhat.com], there's virtually nothing that explains what's going on underneath. Are there any plans for a "GNOME Administrator's Guide"? I'm thinking of something that documents usage of files in $HOME/.gnome, what session management is and how it works, what controls the contents of the GNOME menu, and so on. For example, when GNOME fails to correctly save session information, I'd like to be able to check the documentation to see what should be being written to .gnome/session. At the moment, I just have to guess. Some of it is reasonably obvious from context, but it's the sort of thing that really needs formally documenting.

Nat:

So, for a lot of the stuff you're talking about, the documentation is out there. If you want to learn about the session manager and how to configure it, check out the man pages for "gnome-session" "default.session" and "save-session". There's also a white paper covering a lot of the configuration files, though it is out of date. Collecting and updating all of these things into a single "GNOME System Administrator's Guide" sounds like a great idea for a project for someone :-).

The GNOME Documentation Project and the individual efforts of developers and users have produced a large amount of documentation to date. In addition to the GNOME User's Guide that you mention, there is the user's manual work that Sun has been doing. There is also a lot of developer documentation on developer.gnome.org, including some useful tutorials and white papers.

With all of the large vendors that are shipping GNOME on their workstations, I think it's a safe bet that the components of an administrator's guide will come together in the near future. I know that, inside Ximian, we have recently written for a customer some documentation specifically focused on issues that would be interesting to system administrators, and naturally we will be working to release this to the community at some point soon.

Of course, if you or anyone else out there wants to join up with the GNOME Docs team and start assembling such a guide, you would be welcomed with open arms :-). If you don't have time to do that, you can contribute by filing bugs in bugzilla.gnome.org whenever you find problems or missing pieces and by contributing fixes to the individual projects. Check out the gnome-doc-list mailing list for more information on how you can help.

9) Why subscribe?
by JThaddeus

I was considering subscribing in order to improve the performance of downloads (which have gone to a snail's pace since the subscription program began) but two out of three of my last update attempts have ended in file not found errors. This type of error doesn't give me confidence in how well RedCarpet setups are tested. So why shouldn't I just forget about subscriptions and go with KDE?

Nat:

Without more information, I can't say exactly what the problem is that you were experiencing. That type of issue can sometimes happen if you're updating from one of our mirrors that is in the process of syncing from our master site.

I can tell you that we do directed testing on all updates that are pushed to Red Carpet, on every single supported platform, before an update is released. Additionally, we pay close attention to the bug reports that our users file in our bug tracking system, and make an effort to address all of those as quickly as possible.

Just last week we released a new channel in Red Carpet called "Untested," which contains the pre-QA versions of all of our Ximian GNOME updates before they hit the main channel. Similar to the Mandrake Cooker or Debian unstable, this is a way for the update junkies of the world to get an early glimpse at new packages and versions before they hit the official channel. And of course, this is a way for us to get broader user testing and resolve problems earlier.

Also, by the way, the bandwidth we've allocated to our free public Red Carpet servers has been steadily increasing since the launch of the subscription program. If the servers have gotten slower, it's because the user demand keeps increasing.

But whatever your experiences with Red Carpet, they should not be brought to bear on your choice of desktop. Red Carpet is a software management service that is independent of your choice of desktop or web browser or editor or whatever. And because the Red Carpet client is statically linked, you don't even have to have GNOME installed to use it. In fact, about 20% of Red Carpet usage is by people who want to get updates to the packages provided by their distribution, not Ximian GNOME.

10) External Compatibility
by dspeyer

What plans do you have to improve compatibility with the non-GNOME world?

For example, do you think it's practical to implement Xaw as a front-end to GTK? That would get OpenOffice integration real fast, among others. What about a unified theme format with KDE? Or a common protocol for copy/paste?

It seems like this sort of stuff would be really helpful -- what's actually in the works?

Nat:

Compatibility actually has less to do with an application's choice of drawing toolkit than you might think. Of course, there's nothing to prevent you from running a non-GTK application in GNOME, and it's not necessarily the case that the user experience is hugely degraded if you do. I know of a lot of KDE users who started using Evolution in the last couple of months, because the functionality is so rich, which is great.

GNOME and KDE have had drag-n-drop and cut-n-paste interoperability for quite a while, and we also use the same .desktop file format to store launchers and menu items. You can track a lot of this stuff at freedesktop.org.

Open Office does not use Xaw. That being said, it would be great if OpenOffice used the Gtk drawing primitives so that OpenOffice would be theme-compatible with GNOME. It would not be a total integration, and the behaviour might still be different, but it would help the desktop to look more like a single unit. In fact, it would be possible to get Qt to use Gdk as well, which could make shared themes possible there too.

Another step would be to adopt a common set of icons; baby steps like this can improve visual harmony a lot, even if the "compatibility" is only at a very superficial level. These first steps could be followed by deeper integration, like a working bridge between Bonobo and Uno, the OpenOffice component system.

A unified theme format with KDE would probably be difficult, having a theme or set of core themes for GNOME and KDE which looked and felt the same on both would be a nice step toward making the desktops more compatible to the user. There have been noises made recently that this kind of thing is a possibility, and Ximian would be fully supportive of that.

Though these surface integration steps would be nice, the area where inter-project compatibility is most badly needed is configuration. If someone is running a mixture of GNOME and KDE applications, Mozilla, OpenOffice, and older Xtk-based programs, they need to be able to make configuration changes that are reflected in every application. Having to go to N different places to set your default URL handler, theme, or MIME type bindings is a real usability problem. Jim Gettys talked about this a lot at the most recent GUADEC. Keith Packard's recent fontconfig work is an excellent example of this.