Slashdot Mirror

Amazingly this actually seems to be the approach lots of "Web 2.0" companies are taking. See Tim O'Reilly's database war stories series for details.

Some quotes from that series:

• Gabe Rivera of Memeorandum.com: "I didn't bother with databases because I didn't need the added complexity... I maintain the full text and metadata for thousands of articles and blog posts in core. Tech.memeorandum occupies about 600M of core. Not huge." [full story]
• Mark Fletcher of Bloglines: "[T]raditional database systems were not appropriate (or at least the best fit) for large parts of our system." [full story]
• Greg Linden of Findory: "We make thousands of random accesses to this read-only data on each page serve; Berkeley DB offers the performance necessary to be able to still serve our personalized pages rapidly under this load." [full story]

The "databases cause more problems than they solve" sentiment was so pronounced in O'Reilly's interviews that he took the question to Brian Aker of MySQL for rebuttal, but ended up concluding that

I didn't hear that flat files don't scale. What I heard is that some very big sites are saying that traditional databases don't scale, and that the evolution isn't from flat files to SQL databases, but from flat files to sophisticated custom file systems. Brian acknowledges that SQL vendors haven't solved the problem, but doesn't seem to think that anyone else has either.

Amazingly this actually seems to be the approach lots of "Web 2.0" companies are taking. See Tim O'Reilly's database war stories series for details.

Some quotes from that series:

• Gabe Rivera of Memeorandum.com: "I didn't bother with databases because I didn't need the added complexity... I maintain the full text and metadata for thousands of articles and blog posts in core. Tech.memeorandum occupies about 600M of core. Not huge." [full story]
• Mark Fletcher of Bloglines: "[T]raditional database systems were not appropriate (or at least the best fit) for large parts of our system." [full story]
• Greg Linden of Findory: "We make thousands of random accesses to this read-only data on each page serve; Berkeley DB offers the performance necessary to be able to still serve our personalized pages rapidly under this load." [full story]

The "databases cause more problems than they solve" sentiment was so pronounced in O'Reilly's interviews that he took the question to Brian Aker of MySQL for rebuttal, but ended up concluding that

I didn't hear that flat files don't scale. What I heard is that some very big sites are saying that traditional databases don't scale, and that the evolution isn't from flat files to SQL databases, but from flat files to sophisticated custom file systems. Brian acknowledges that SQL vendors haven't solved the problem, but doesn't seem to think that anyone else has either.

Amazingly this actually seems to be the approach lots of "Web 2.0" companies are taking. See Tim O'Reilly's database war stories series for details.

Some quotes from that series:

• Gabe Rivera of Memeorandum.com: "I didn't bother with databases because I didn't need the added complexity... I maintain the full text and metadata for thousands of articles and blog posts in core. Tech.memeorandum occupies about 600M of core. Not huge." [full story]
• Mark Fletcher of Bloglines: "[T]raditional database systems were not appropriate (or at least the best fit) for large parts of our system." [full story]
• Greg Linden of Findory: "We make thousands of random accesses to this read-only data on each page serve; Berkeley DB offers the performance necessary to be able to still serve our personalized pages rapidly under this load." [full story]

The "databases cause more problems than they solve" sentiment was so pronounced in O'Reilly's interviews that he took the question to Brian Aker of MySQL for rebuttal, but ended up concluding that

I didn't hear that flat files don't scale. What I heard is that some very big sites are saying that traditional databases don't scale, and that the evolution isn't from flat files to SQL databases, but from flat files to sophisticated custom file systems. Brian acknowledges that SQL vendors haven't solved the problem, but doesn't seem to think that anyone else has either.

Amazingly this actually seems to be the approach lots of "Web 2.0" companies are taking. See Tim O'Reilly's database war stories series for details.

Some quotes from that series:

• Gabe Rivera of Memeorandum.com: "I didn't bother with databases because I didn't need the added complexity... I maintain the full text and metadata for thousands of articles and blog posts in core. Tech.memeorandum occupies about 600M of core. Not huge." [full story]
• Mark Fletcher of Bloglines: "[T]raditional database systems were not appropriate (or at least the best fit) for large parts of our system." [full story]
• Greg Linden of Findory: "We make thousands of random accesses to this read-only data on each page serve; Berkeley DB offers the performance necessary to be able to still serve our personalized pages rapidly under this load." [full story]

The "databases cause more problems than they solve" sentiment was so pronounced in O'Reilly's interviews that he took the question to Brian Aker of MySQL for rebuttal, but ended up concluding that

I didn't hear that flat files don't scale. What I heard is that some very big sites are saying that traditional databases don't scale, and that the evolution isn't from flat files to SQL databases, but from flat files to sophisticated custom file systems. Brian acknowledges that SQL vendors haven't solved the problem, but doesn't seem to think that anyone else has either.

Amazingly this actually seems to be the approach lots of "Web 2.0" companies are taking. See Tim O'Reilly's database war stories series for details.

Some quotes from that series:

• Gabe Rivera of Memeorandum.com: "I didn't bother with databases because I didn't need the added complexity... I maintain the full text and metadata for thousands of articles and blog posts in core. Tech.memeorandum occupies about 600M of core. Not huge." [full story]
• Mark Fletcher of Bloglines: "[T]raditional database systems were not appropriate (or at least the best fit) for large parts of our system." [full story]
• Greg Linden of Findory: "We make thousands of random accesses to this read-only data on each page serve; Berkeley DB offers the performance necessary to be able to still serve our personalized pages rapidly under this load." [full story]

The "databases cause more problems than they solve" sentiment was so pronounced in O'Reilly's interviews that he took the question to Brian Aker of MySQL for rebuttal, but ended up concluding that

I didn't hear that flat files don't scale. What I heard is that some very big sites are saying that traditional databases don't scale, and that the evolution isn't from flat files to SQL databases, but from flat files to sophisticated custom file systems. Brian acknowledges that SQL vendors haven't solved the problem, but doesn't seem to think that anyone else has either.

No, the post is refering to the maintainer of electoral-vote.com, Dr. Andrew Tanenbaum, who in 1992 instigated a heated debate with Linus Torvlads on the comp.os.minix newsgroup about the relative merits of microkernels vs. monolithic kernels. Tanenbaum maintained that "LINUX is obsolete" and suggested that "people who want a **MODERN** "free" OS look around for a microkernel-based, portable OS, like maybe GNU or something like that." Of course, history tells us that millions of businesses and individuals disagree with his position.

O'Reilly has a transcript:
http://www.oreilly.com/catalog/opensources/book/ap pa.html

I highly recommend O'Reilly's LDAP System Administration. That's where I started when I was interested in setting up LDAP. When I started, I knew nothing. By the time I was through the book, I had a working LDAP directory with TLS encryption and master-slave duplication, and I actually understood what how I got there.

Yet if they don't get it into print, it can't be used in a classroom setting.

The new formats are zipped by default. The zip files do contain the data as XML

I was surprised by the general shittiness of Ruby In a Nutshell. I found it difficult to use to actually learn ruby. On a co-worker's recommendation, I picked up a copy of Programming Ruby: The Pragmatic Programmer's Guide, which I've been much more happy with.

That's the first O'Reilly book I've encountered that's been so thoroughly unsatisfactory. A shame, really. I'd like to believe this is an exception to the rule rather than the harbinger of a general downard trend in the quality of O'Reilly books.

The hardware isn't free but I really think that is something _software_ license isn't supposed to deal with.

Does music on an iPod fall under the definition of "digital musical recording?" There are other programs stored on an iPod (i.e. the calendar, games, etc.) which are not "incidental" (def. "related to and relatively minor by comparison") to playing music. Does that affect the Rio decision you mentioned?

The definition does not force the "digital musical recording" to exist only on an AHRA device/medium, so a computer dedicated to playing music could be covered, since the OS and other programs would be incidental to that role.

The 9th makes bad law. It defines the "active judiciary." It is consistently the most reviewed and reversed (relative to size) circuit (cf. the 5th).

Copying off an AHRA CD to a computer so you can at some indeterminate time make a "digital musical recording" is a "use" in much the same way that someone growing dope in their backyard or a butterfly flapping it's wings is regulatable interstate commerce, or that invoking eminent domain to transfer private property to a private corporation is "public use." :-)



Thanks for the discussion, it can be nice arguing with lawyers, since they usually don't go all "ad hominem" when presented with something with which they disagree.

Camraderie seemed to be running pretty high, over in the non-profit corner (wrapped around the slashdot lounge, which was a bunch of laptop zombies on slashdot beanbag chairs)... it seemed like there was a lot of action at the Debian booth, the postgresql booth, and so on (though there was no perl foundation presence this year, not sure why). Lots of people were passing out free CDs for this-and-that.

Out in the corporate world, there was a pretty elaborate demo of Suse 10 handled by Novell: a class room layout with enough laptops setup for a few dozen people to play along with the demos. These demos were extremely slick, very impressive... it's too bad RedHat ("linux isn't ready for the desktop") wasn't present at the Expo. Running Suse might seem like an excessive compromise with proprietary software (it does to me -- it looks like I'm going with Knoppix and Kubuntu these days) but there's no question it would be better than being locked into the offerings of Windows or Apple.

In the light of these Suse 10 demos, the OpenSuse project -- which had a small booth off in the aforementioned corner -- seems very interesting. They were passing out disks that apparently included a few non-"open" components though (flash, etc).

The O'Reilly booth had it's fair share of people browsing, though there didn't seem to be all that much excitement about their present offerings, at least not to my eye. They had a nice series of talks going that I appreciated (e.g. two seperate talks by Bill Childers and Kyle Rankin, the authors of the new Ubuntu Hacks).

Out in hardware land, there was a nice array of server hardware (e.g. impressive booths by Tyan and Supermicro) ... I always appreciate this kind of thing, because not being a sysadmin type I don't often get that close to high-end hardware like this.

Emperor penguin was in the house, with demo models of all of their laptop models. Still no AMD64 versions, I'm afraid: apparently they're waiting for Dell to get on it...

It seemed like the general theme out on the floor was "virtualization"... I was hanging around with a friend of mine, listening to a sales pitch on the subject (by EMC, I think), trying to figure out what was so cool about it, but without much success. Hardware is cheap enough that it wouldn't seem all that onerous to stick with one box per OS installation... and after all, you can run NFS if you want to use large disk arrays more efficiently. But everywhere I turned someone was talking about it... Bill Childers mentioned in passing that his company had gotten a 12 to 1 reduction in servers by using vmware (which has a freeware version, but is not free/open), and one of the Debian folks was talking about how it's really good for some random legacy app that needs a particular platform that otherwise you wouldn't want to run.

Do you have any data that shows that Mono deployment in the enterprise is increasing, relative to java deployment?

Well, it's not particularly scientific and it speaks more to .NET in general than to Mono specifically, but if you believe Tim O'Reilly's book sales data C#/.NET passed Java in popularity/interest over the last year, and is still growing strong (C# sales up 68%, general .NET book sales up 125%; Java book sales are down 6% over the same period).

Of course one could always argue that more .NET books sell because .NET is harder to learn... but having dipped my toe in the Java waters a few times, I find that hard to believe :-)

• Open source project leaders can't order anybody to do anything
• Open source project workers can fork at any time if they disagree with the direction of the project

There are a ton of commercial utilities and add-ons for MS Access (check out Access Advisor at your local bookstore), but most of those are just VB apps or ActiveX controls that just do what you could do yourself with a little Visual Basic. Once you have got the basics down from some online tutorials, Access Cookbook by Kurt Getz is a great investment.

MS Access has a large community online, especially comp.databases.ms-access. Google is your friend - just about everything you'll ever want to do has already been done and has VB code examples online.

Here is a thread that has code demonstrating how to dump the contents of an Access database as DDL into text files:

comp.databases.ms-access: Exporting jet table metadata as text?

PS - If you are impatient with the limitations of VBA (aka "VB Classic"), there are Microsoft Office interop libraries that will let you automate Access Databases in .NET.

There are a ton of commercial utilities and add-ons for MS Access (check out Access Advisor at your local bookstore), but most of those are just VB apps or ActiveX controls that just do what you could do yourself with a little Visual Basic. Once you have got the basics down from some online tutorials, Access Cookbook by Kurt Getz is a great investment.

MS Access has a large community online, especially comp.databases.ms-access. Google is your friend - just about everything you'll ever want to do has already been done and has VB code examples online.

Here is a thread that has code demonstrating how to dump the contents of an Access database as DDL into text files:

comp.databases.ms-access: Exporting jet table metadata as text?

PS - If you are impatient with the limitations of VBA (aka "VB Classic"), there are Microsoft Office interop libraries that will let you automate Access Databases in .NET.

There are a ton of commercial utilities and add-ons for MS Access (check out Access Advisor at your local bookstore), but most of those are just VB apps or ActiveX controls that just do what you could do yourself with a little Visual Basic. Once you have got the basics down from some online tutorials, Access Cookbook by Kurt Getz is a great investment.

MS Access has a large community online, especially comp.databases.ms-access. Google is your friend - just about everything you'll ever want to do has already been done and has VB code examples online.

Here is a thread that has code demonstrating how to dump the contents of an Access database as DDL into text files:

comp.databases.ms-access: Exporting jet table metadata as text?

PS - If you are impatient with the limitations of VBA (aka "VB Classic"), there are Microsoft Office interop libraries that will let you automate Access Databases in .NET.

There are a ton of commercial utilities and add-ons for MS Access (check out Access Advisor at your local bookstore), but most of those are just VB apps or ActiveX controls that just do what you could do yourself with a little Visual Basic. Once you have got the basics down from some online tutorials, Access Cookbook by Kurt Getz is a great investment.

MS Access has a large community online, especially comp.databases.ms-access. Google is your friend - just about everything you'll ever want to do has already been done and has VB code examples online.

Here is a thread that has code demonstrating how to dump the contents of an Access database as DDL into text files:

comp.databases.ms-access: Exporting jet table metadata as text?

PS - If you are impatient with the limitations of VBA (aka "VB Classic"), there are Microsoft Office interop libraries that will let you automate Access Databases in .NET.

I posted this before, but thought it was good enough to post again...

Oreillys radar's site take on the new features of the OS (by nat):

A good read actually:

http://radar.oreilly.com/archives/2006/08/apple_ea ts_whiners.html

Its on the radar sites: (by nat):

A good read actually:

http://radar.oreilly.com/archives/2006/08/apple_ea ts_whiners.html

No doubt there's still a lot of room for Python, but since you brought up the bookstore metric: http://radar.oreilly.com/archives/2005/12/ruby_boo k_sales_surpass_python.html

> Having said that I really don't see why you
> have to devote a complete book on regex.
> A small tutorial does just fine

I think it depends on how deep you want to go into regular expressions. Mastering Regular Expressions by Jeffrey Friedl is almost 500 pages but is an excellent treatment of the subject - by the time you're done reading it you'll feel comfy even with such madness as negative lookbehind.

If you think that then Herbert caught you as he did me and so many others. You need to read the first three of the series as a single piece of work whose overarching theme is: superheroes - whether good or evil - are dangerous to humanity. It's easy to stop at the end of Dune and rejoice in the final triumph of Paul and the defeat of the bad guys, and then miss his decline and eventual humiliation in the next two books, because the events he triggered have gone beyond his control.

Tim O'Reilly's biography of Herbert explains the author's purpose very well. It's available online here.

Found one: http://www.oreilly.com/catalog/phpnut/

I got the O'Reilly Ubuntu Hacks book http://www.oreilly.com/catalog/ubuntuhks/ and was able to easily setup and roam with my old Linksys WiFi card on an ancient P3 Winbook. Anything I wanted to do with Ubuntu I now can, thanks to this book. The laptop is actually much faster and more useful than it ever was with Windows installed.

With the right info on hand Ubuntu is a true gem of an OS - possibly the best available (on older machines be sure to install in "safe Mode" so it doesn't "hang" on you).

It is a full-featured and low-overhead ORM. Go to http://www.hibernate.org and check out the NHibernate link.

Plus, there are several books avilable from, e.g., O'Reilly, Manning.

Who cares? Well, some very smart people do. (Of those, Tim Bray himself switching as well.)

Whether you personally know or respect Mark, Tim and Cory, they're being looked to by a huge amount of others for guidance. This isn't a lightly made switch - "oh you know, I have a spare box lying around and I'm going to see how this shiny new OS works out, and then next week I'll go and play with Gentoo, and I've always been meaning to give Solaris a try as well". This is people with a tremendous amount of experience and knowledge, having spent their whole life on Macs, deciding that enough is enough, that the bough has broken, and that they care more about their data than about anything else. They all have a huge following, and their thoughts will reverberate.

Most people who will actually read their thoughts (rather than going for the knee-jerk "no, it's Monday so apple is good!" slashdot reaction that I've seen far too many posters here resort to) will probably be set thinking because of it. And everyone will make up their own minds, and most people will probably decide not to switch, for reasons that for them will be very valid. But you can sure as hell bet that the importance of open data formats and lack of DRM will become more of a talking point in the months to come, and that if Apple doesn't heed this warning, more and more people will come to the same conclusions as Mark, Time and Cory have.

(If you want to get the whole story, I'd read the following articles in this order:

1. Mark Pilgrim: Bye Apple
2. Mark Pilgrim: When the Bough Breaks
3. John Gruber: And Oranges
4. Mark Pilgrim: Juggling Oranges
5. Tim Bray: Time to Switch?
6. Cory Doctorow: Mark Pilgrim's list of Ubuntu essentials for ex-Mac users

You might prefer Perl Hacks then.

I guarantee that it contains a few things you have never seen before.

Well, there is this Oreilly Windows XP Hacks book...

Try Perl Hacks then. I don't use much as intended.

As far as network drivers in particular go, the layers that use them, such as IP, live in the kernel, so it's rather annoying for them to talk to drivers that are up in user space. Specific network cards, especially wireless, might have bits that live up in user space, such as user interfaces for loading in crypto keys, but the bulk data transfer applications normally belong down in or near the kernel.

Why are there a whole pile of network card drivers in the kernel when you'd normally use only one or two? Same reason there are a whole pile of drivers for other devices in the kernel, when you've normally only got one graphics card and one sound card. If you're shipping a pre-compiled kernel, you want it to support as many different users as possible, and all it costs you is some RAM to store the code you're not using, or if you can handle them as loadable modules, it only costs you the work to keep track of those. But if you want to compile your own kernel for specific machines, and leave out the drivers you don't want, and while you're at it compile all your applications programs with the level of optimization your hardware supports, get a copy of Gentoo Linux and have fun learning lots more detail about Linux internals.

That's especially funny considering that Mr. Stallman is one of the few Americans who actually bathes less than the French. There's a fairly typical photo of Richard on the cover of "FREE AS IN FREEDOM", and that cover is online at http://www.oreilly.com/openbook/freedom/

... and I'll add my own bit. Read a lot. I like http://safari.oreilly.com/ because I save a ton of money on books.

Install your own Linux network and way over engineer it. You'll learn lots by setting up DNS, NIS, Apache and other services.

When you don't know something Google is your friend. Lot's of people forget this. Got an error? Google it. Want to see how NIS works? Google it. It's pretty rare to have a question that hasn't been asked a hundred times before.

Pick a subsystem and study it. Do a "ps ax" and pick a process and learn what it does, how to configure it and whatnot. That's the beauty of Linux you can dig as deep as you want.

I also hear from a number of friends that installing Gentoo is a great way to learn. The docs are excellent and you learn a lot by the time you've got a base system up and running.

Never lose your curiosity for how things work. It's the key to learning.

There's been a lot of speculation that RIM will be coming out with a BlackBerry with a camera in the near future, too. However, the current 8700 has lots of multimedia features already (support for video, etc) and there's talk of an impending software release unlocking even more capabilities (like more frames per second).

With that always on data connection and most users subscribing to an unlimited data plan, it's easy to imagine ways where as iTunes enabled BlackBerry would solve a lot of Apple's iPhone problems (OTA podcasting anyone?)

I've always thought of both of these companies as user interface kings in their respective categories. I'd bet that they've at least had some discussions about the possibilities of such an agreement.

Dave Mabe
Shameless plug: Author of BlackBerry Hacks

In the news today: Tanenbaum charged of subliminally brain-washing people on his microkernel design. People with tin-foil hats live to tell the day!

* lon3st4r *

And i quote:

2.

MediaLive filed for the trademark on the Web 2.0 Conference back in November 2003, when they first entered into the partnership agreement with O'Reilly on this conference. This was before Web 2.0 became such a popular term -- the filing actually preceded the first conference. However, I wasn't personally aware of this trademark filing till this past February, as a result of discussions with CMP after the MediaLive purchase.

Next, is the issue of proportional response. O'Reilly as an INSTUTION apologized for the gaff that resulted in sending this man a C&D. The shit storm that resulted from his blog, and then the rest of the half-cocked idiots such as parent post was not warranted, accurately sourced, or anything more than mis-reported hearsay. Please, for the love of mike, READ before posting. The apology issued to O'Reilly was justified.

Finally, if you'd read the other comments before posting, no finger pointing has taken place. O'Reilly CITES Torvolds and others as examples of trademark holders who also want to protect their trademarks. Again, if parent post had RTFA (s)he'd know that. But parent post clearly did not.

Go to a bookstore and look at the O'Reilly book covers or check out the Alphabet Riddle. No fact is too obscure for Google if you got the time to look. :)

It seems they regret the way this was handled

As noted in the letter to IT@Cork (sent from CMP's attorney, but with our knowledge and agreement), ...

....was granted without adequate review of prior art, and further, that even were it ultimately found valid, such broad patents serve only to hold back further innovation.

In retrospect, we wish we'd contacted the IT@Cork folks personally and talked over the issue before sending legal correspondence. In fact, it turns out that they asked Tim to speak at the conference, though our Web 2.0 Conference team didn't know that. We've sent a followup letter to Donagh Kiernan, agreeing that IT@Cork can use the Web 2.0 name this year. While we stand by the principle that we need to protect our "Web 2.0" mark from unauthorized use in the context of conferences, we apologize for the way we initially handled the issue with IT@Cork.

That's just an excerpt, follow the link for the whole [brief] comment. They also point out, rightfully so, that they would not be able to have a "LinuxWorld conference," and this is no different. It's a service mark, they have to defend it, end of story.

Three nice books, but I'm still in favor of starting everyone out with Learning the UNIX Operating System. Anyone can do it in a few hours and it will save days of frustration down the road. It's probably the only one that gives you just about all the information you can absorb in one go. And with no fat or carbs added.

Sure you can. As a general principle, I agree with you. But not everything they've published is high quality.

I agree with these book selections though I think that it's wrong to say "these are must-haves for the Linux/Unix user" if they cost money. That's because Linux should be free, you shouldn't 'need' to drop $200 to be proficient in it. You need to invest time but not money.

Perhaps there are free resources out there.

Oops - forgot the link:

O'Reilly Head First Series

You need to see how you want to access your storage, and what is going to be running on it, as to how you go:-

SAN - block level data access to storage. Good for databases; low client counts (because SAN ports are expensive relative to ethernet) - but with high IO demands. EMC are good, but pricey - a low to mid end Clariion would probably be the right range to aim at.

NAS - file level data access to storage. Good for situations where there are many clients connecting, and their IO demands are not excessive. Netapps filers are very good at this (if youy can find information on their new OS (10GX) then it's VERY interesting. ILM use them in their render farms.

iSCSI - a blend of the best of both, but it's still looked upon as an emerging technology. You get (or did) free iSCSI licenses with netapps filers.

O'Reilly have a good book on this. "Using SAN's and NAS" which is vendor agnostic http://www.oreilly.com/catalog/sansnas/index.html

If only. Old modem users will remember the regular, periodic appearance of junk characters when a T1 in the path was free-running, and on every bit slip, a phase error was inserted. It's still a problem. See the T1 Survival Guide.

So many frameworks, so little time...

I would love a comparison between Rails and perls Maypole and Catalyst or Phythons TurboGears and Django or php's cakePHP and smart3

Hard to find Ruby programmers, and their are more commonly known languages with comparable frameworks, some of which have been around longer (like maypole). Then again, Ruby fanatics are generally cream of the crop programmers and one wouldn't have to worry about PHP n00bs who can't really code or perl programmers who write line noise or the overhead of Java and it's seemingly required department of architects

Seems other languages are just as good, just harder to manage, but if you have top notch PHP programmers and Perl programmers who work well together to write maintainable code , I wonder how they would compare then?

That's who and why and I can understand that.

Her reaction is generally "just plan better". I argue that the industry has been struggling with this issue for decades. I don't think we're all morons to have built so much infrastructure and come so far, but we still can't solve the simple parts like accurately identifying how long it will take us to accomplish our goal.

Hmmm, I'm still not sure what you want to explain but I'll take a swing anyway. I can think of social, technical and legal complexities to software development. I've talked to my wife about all three. You might be thinking of something completely different.

Talking to my wife is not all that hard, even though she has no interest in programming. Her first and only practice was some kind of basic in grade school. She was an interior designer for a Steelcase for eight years and understands all three classes of difficulties.

Others have done a great job explaining complexities in terms of free software. Voices from the Open Source Revolution has a lot of clear thinking from software masters. Vixie's article about software engineering is particularly germain. You can also get a lot of good thought from the Free Software Foundation's philosophy pages. The Cathedral and the Bazaar deals with the issue explicitly. Indeed, there's an embarrassment of riches matched only by the wealth of text editors in the free software world.

So, how do you get from there to dinner table conversation with the wife who's never written a line of code? It's the same way you try to simplify everything and the largeness of the subject actually helps.

You start with what a program is and everything flows from there. My wife, like most people, understands modularity. "You eat an elephant one bite at a time," is one of her favorite sayings. She also has a basic idea that a program is something that takes information and does something with it. It does not take too much to explain that programs expect specific organization of their inputs to be able to deal with it and that smaller, simpler programs are easier to work with that big complex ones, and the wife then understands modular programming. It's a division of labor kind of thing that runs right into group development and organizational and social complexity. How do you know what the customer really needs? How do you make decisions about meeting those needs and turn those into a blueprint that you can follow? The free software world has solved those problems by letting the customer make the software themselves, and those customers have been organizing themselves very well. At that point, you zoom back into the perspective of a developer getting their hands on some huge project. If you can imagine that the free software developer knows what they want to accomplish, you are then faced with another embarrassment of riches: so many great tools, each of which can take years to explore. Did I say "free software developer"? Yes I did, because I did not want to wade into the swamp of NDA's, cross licensing, binary blobs and other horror stories of legal complexity. That can come later. By now, your wife's head will have popped but you will have explained software development complexity.

Like most things, none of the parts is particularly difficult, there's just a lot of parts.