The Limits of Software

Thanks to Jason Bennett, who wrote this review of The Limits of Software. Robert N. Britcher explores in this book what software is and where software is going -- and what it really means.

The Limits of Software author Robert N. Britcher pages 214 publisher Addison Wesley rating 7 reviewer Jason Bennett ISBN 0-201-43323-0 summary Where we've been, where we're going, and the implications therein

Background

Before I launch into my latest review, I'd just like to say thanks to Hemos and Slashdot on the occasion of my twentieth review posted here. It's been 25 months since the first one (August, '98), and I've really appreciated the opportunity they've given me. Nice excuse to do something I should do anyway! :-)

The Scenario

"But it is not the practitioners alone who are so moved. A thousand years in the making, the religion of technology has become the common enchantment, not only of the designers of technology but also those caught up in, and undone by, their godly designs. The expectation of ultimate salvation through technology, whatever the immediate human and social costs, has become the unspoken orthodoxy, reinforced by a market-induced enthusiasm for novelty and sanctioned by a millenarian yearning for new beginnings. This popular faith, subliminally indulged and intensified by corporate, government, and media pitchmen, inspires an awed deference to the practitioners and their promises of deliverance while diverting attention from more urgent concerns. Thus, unrestrained technological development is allowed to proceed apace, without serious scrutiny or oversight -- without reason. Pleas for some rationality, for reflection about pace and purpose, for sober assessment of costs and benefits -- for evidence even of economic value, much less larger social gains -- are dismissed as irrational. From within the faith, any and all criticism appears irrelevant, and irreverent." (TLOS, xxiii)

-- David F. Noble, The Religion of Technology, as quoted in The Limits of Software

I had the privilege of spending a few weeks with a good friend of mine in Eastern Europe back in July. Of course, to go anywhere on a budget in Europe requires a lot of train travel. Alas, there are no bullet trains in Slovenia, which gave me plenty of time to take in some reading when I wasn't chatting with my fellow passengers ...

The Limits of Software is a unique book in many ways, not the least of which is that it reads more like a collection of life stories than a lecturing textbook. Most computer books simply give you data, or even information, in a straightforward manner, hopefully punctuated by some interesting anecdotes. Britcher, instead, has packaged with words slices of time which illustrate various points about where computer programming has been, and where software development is going (note the terminology change). I certainly won't try to describe them all, but theme which runs through the book is illustrated in the opening quotation: software is not our savior. There is no "one great system" that will be able to handle things. The FAA's botched air traffic control system is used as one illustration in the book, but the point is made about all software: we cannot and must not worship it.

There's one point that I find simultaneously funny and sad: It's in the chapter on testing, and the inherent futility of such an activity on complex programs. Britcher discusses the Y2K bug, and mentions the survivalist movement.

"Just as regular folks built bomb shelters in the 1950s and 1960s to add life time to a planet white with nuclear snow, regular folks are now storing large caches of food, water, toilet paper, clothing, and, of course, the American twinship: sacred literature and ammo. One man who agreed to be interviewed for the piece was quoted: 'When you first hear about it, most people are in total denial. They can't believe that Bill Gates won't come up with a magic bullet.' (That the general population believes that Bill Gates has the answers to our programming problems is more frightening than the rollover of the millennium.)" (TLOS, 59)

I quote this not as a shot at Bill (although, this being Slashdot, I'm sure some will take it that way), but to point out the inherent risks in the statement, which illustrate Britcher's point. Software is dangerous, because it does so much yet is so fragile. We (even we programmers at times) view it as a holy grail. We cannot understand how our mechanical saviors could possibly fail us. Yet, software failures are rampant, in every facet of our society (see the Risks Digest if you need examples). Software cannot solve our problems. Our problems are inherent within ourselves. As we continue to rely more and more on machines to live for us, we must remember that they, like their creators, are fallible. What's Bad? / What's Good?

When I finished TLOS, my first reaction was to think of the old saw about the life of a fighter pilot: "hours and hours of sheer boredom, punctuated by moments of sheer terror." Britchner's stories seemed to drone on at points. The FAA story was left to the end. Why did he have to go on and on about all this random stuff?

In retrospect, though, I think I have a better grasp of what Britcher was trying to convey. This is not a disaster movie told in the guise of software engineering; this is a story about one man's journey through software, and the conclusions he's come to. Read this as an technological autobiography, and I think you'll appreciate the points being made. As I said earlier, it's different, but rewarding in the end.

So What's In It For Me?

A reminder that the Tower of Babel still lives in the hearts and minds of men.

You can purchase this book at Fatbrain.

Table of Contents

• Foreware by Robert L. Glass
• Prologue
• Part I
  1. Early Systems
  2. Theories of Programming
  3. The Human Element
  4. Designing
  5. Code: The Stuff of Programs
  6. Testing Computer Systems
  7. The Impossible Profession
  8. Life on the Project
• Part II
  1. Supervision Through Language
  2. How Technology Changes Methods
  3. Size and Intellectual Gravity
  4. The Marketing of Science
  5. Errors
  6. The One Great System
  7. The Government of Programming
  8. The System to End All
  9. The End-All of Programming
• Afterward
• Reading List

Comments

Nice quote.

[Lonesmurf]

"hours and hours of sheer boredom, punctuated by moments of sheer terror."

funny, that's pretty much my life. :)

how about: "hours and hours of coding, punctuated by moments of delirium."

or: "hours and hours of sheer boredom, puctuated by lunch."

Rami
--

People never change

[FascDot Killed My Pr]

I bet there was some ancient Sumerian who gave long lectures on "The Limits of Literacy" warning that we shouldn't worship reading and writing--those skills can't make our lives better. Imagine his face if we were to show him a modern, industrialized nation.

It's a poor workman who blames his tools. If there is a way for an air-traffic to be controlled by a system, and your air-traffic control system doesn't do it, the reason is not inherent in the "limits of software"--there's some problem with your design/implementation. Software is a universal machine simulator--pure algorithms implemented as 1's and 0's. If that idea isn't worthy of awe I don't know what is.
--
Linux MAPI Server!
http://www.openone.com/software/MailOne/

Re:People never change

[David A. Madore]

I entirely agree. That's a comparison I like to make. When people say to me things like "we'll always need graphical user interfaces and user-friendly software like Windows: people will never learn to master the complexities of the Command Line", I reply that in the middle ages, the few literate people might have thought "we'll always need to have little icons in front of the shops so people can tell the inn from the blacksmith; these ignorant peasants will never learn to master the complexities of writing".

Guess what? In the year 2000, over half of the population of the world knows how to read and write. Let's make it our objective for 2100 that over half of the population of the world have a computer and know how to program.

Re:People never change

[Reality Master 101]

Let's make it our objective for 2100 that over half of the population of the world have a computer and know how to program.

That's like that middle ages guy saying, "Let's make it our goal that by 2000 half the world knows how to read and knows how to operate printing presses, binding machines and book distribution.

I agree that it's important that people become computer "literate", but it's NOT important that everyone learn to program. I make this point because it comes up far too much on Slashdot, particularly in the context of education ("Let's teach every student how to program!"). It's an important point that is lost on many Slashdotters... the vast majority of the population are not interested in engineering, and never will be. And that's OK. Everyone doesn't have to know or care how the hammer is forged to drive a nail.

--

Re:People never change

[wnissen]

It's a poor workman who blames his tools. If there is a way for an air-traffic to be controlled by a system, and your air-traffic control system doesn't do it, the reason is not inherent in the "limits of software"--there's some problem with your design/implementation.

I believe this misses the point of the book, at least judging from the quote by Prof. Noble. The point is, it's a poor workman who assumes his tools are all-powerful and not subject to normal criteria like cost/benefit analyses. The engineers who built cathedrals (no ESR metaphors intended here) used secret design principles painstakingly determined by trial and error through the centuries. If the building stayed up for hundreds of years, it was a Good Thing. Software is a young discipline, largely without solid rules, and yet is widely acknowledged to be changing the world. Software will drastically change the next generation of children, whose parents will prbably all have to use a computer at some point. There are what, a hundred million computers in the US? And yet the primary operating system on them is actively bad. Even Linux is often klunky, and despite better stability isn't even as easy to use as that mass of inconsistency, Windows. Why is this? Because the culture of software believes "good enough" really is good enough. Most people who become programmers are lucky if they get a few semesters of training learning different languages. They get no training at all in controlling the number of defects in their code. This results in lots of software of average quality, that is to say, shit.

A lot of this is driven by the fact that many people believe it is okay to admire software as math. It's not. Software is meant to accomplish something, and to do it better or faster or cheaper than however it was done before. It has absolutely nothing to do with pretty math, or purity, or any of those things you say are worthy of awe. If I designed a bridge with a novel way of laying out the supports that was really neat, but the bridge fell over because the supports interfered with the arches, no one would complement me on my support placement algorithm. They'd think I was an idiot who couldn't design a bridge to save his life. And yet I see all the time, "Wow, this code is really cool, it doesn't all work yet but isn't it neat?" because as long as it's software no one cares.

Walt

Re:People never change

[Bazzargh]

I agree with the sentiment, but not the words. I hope computing will become ubiquitous to the extent that you will not be able to point at something and say 'that is a computer', because everything is.

Anyway, in the year 2000, there are still far too many people killed in petty conflicts and too many people with inadequate food+water (usually the result of the conflicts, not nature).

Miss Worlds should still ask for world peace, not world computer literacy.

Re:People never change

[YKnot]

You don't have to be an expert at programming, but having an idea about how a computer does its job certainly helps using it. When you're really good at using a computer, you've necessarily understood the basic principles of programming. Future computer users will probably use them almost everywhere and any time. There will be no "just need to know Word" users, and even those better know about scripting to get simple repetitive jobs done quick. The difference between a user and a programmer is that a user memorizes all the stuff he uses, while a programmer understands it and generalizes. With ever growing computer use, tell me which way is more efficient...

Re:People never change

[YKnot]

The reason people don't see programming as part of required general knowledge is that they think it is an awfully complex skill. What I am talking about is iteration, variables and structure. The "user" has to wait for someone to design a special user interface for the task. Someone with basic programming skills can just "script" his way through it. You have almost written the program when you have understood the problem. From that point on, formalization is all that is left to do. Maybe people shouldn't learn C++ or Java or whatever classic programming language you prefer, but graphical programming is still programming.

Or better yet

[bluGill]

my local small town book store has a web site where I can enter the ISBN for any (in print or at least avaiable) book and they will put it in their next order. I pick it up there, but since it is in town it isn't a big deal (and I don't worry about it arriving in the rain). I pay sales tax, but avoid shipping. I support a small town book store that gives me a place to look over books for those that are worth reading.

Now if I just had time to read...

Customer bug finding and third-party software

[flatpack]

Unfortunately it's a valid point - no piece of software ships with all of its bugs fixed, or even discovered (note that when working for software companies these two things are not synonymous). A decent testing department given a reasonable amount of time can spot many of the more obvious bugs, but for every obvious bug there will be a dozen sneaky ones which ship.

You can guarantee that it'll be the end users that'll find these bugs, and that they'll complain to whoever sold/provided them with the software. Remember, end users can be relied upon to use the software in the most convoluted, even stupid, ways, and when they're doing something in a manner the programmer didn't anticipate chances are they'll find bugs.

This is why I think that for any large software project an open database allowing customers to report bugs is essential. As long as the programmers work on clearing these lists then problems get ironed out over time, and making sure patches are freely available reduces the number of flaws in a program and makes everyone happy.

Another problem that introduces a lot of errors today is the increasing use of third-party components and applications within software projects. These introduce another point of failure, and one in which the programmers have less control over what goes on, and less knowledge about possible solutions. Even if you have the source for the component, it's still much more likely that using it will introduce problems, either in the interface between your app and the component, or in the component itself.

The solution? I don't really think there is one that can guarantee perfect reliability. The best you can do is ensure that the number of problems is kept to a minimum, and that problems that are found are quickly dealt with.

Technology is simply a tool

[Private Essayist]

"The expectation of ultimate salvation through technology, whatever the immediate human and social costs, has become the unspoken orthodoxy, reinforced by a market-induced enthusiasm for novelty and sanctioned by a millenarian yearning for new beginnings. This popular faith, subliminally indulged and intensified by corporate, government, and media pitchmen, inspires an awed deference to the practitioners and their promises of deliverance while diverting attention from more urgent concerns. Thus, unrestrained technological development is allowed to proceed apace, without serious scrutiny or oversight -- without reason."

I know people who go to the opposite extreme -- they see the problems with tecnological developments (auto pollution, atomic weapons, and the like) and take the attitude that technology is evil, or that it often is used that way.

Both the worship of techology, and the hatred of technology seems to me to miss the point. Technology is simply a tool. It can be used to create good things, and bad. It can result in good side-effects, and bad.

Speaking specifically of software development, there are times when people outside the field look with awe on the process of creating code. That ignorance of the process may contribute to those extreme ways of thinking about technology. Once you know how something works, it gets de-mystified in your mind and becomes just another tool or skill. Programming, I tell people, is like playing the piano -- it's a skill that can be learned, not magic.

Similarly, technology is a tool, not magic, not a panacea, not the solution to human problems by itself.
________________

Re:Technology is simply a tool

[(void*)]

Exactly. But just like playing a piano, there are master pianists and there are casual players. And unlike the piano, we desperate need the talented programmers to work on genuinely useful software.

Software is limited

[ch-chuck]

only by the ability of the human mind to manage complexity. I'm tempted to fit Gödel's incompletness of formal systems in there somewhere but that seems to imply an UNlimited condition rather than limited. If 'software' is something inbetween mathematics and hardware, we see that the ability to count toward infinity is limited only by the physical realization of the computer (register size, etc).

As for fragile software, just try to 'diskcopy' a Linux boot disk in NT4-BugFix5, hehehe.

How Ironic

[guinsu]

I find it ironic that the opinion that technology can't solve our problems comes right after a story about Dean Kamen, who thinks technology can solve our problems.

My personal feeling is that it can solve a lot of our problems permanently (hunger, disease, etc) however it won't spontaneously happen, it does require people and their priorities to change too. If a large enough portion of this planet decided tommorow that clean burning engines, safe drinking water and better agriculture were the areas they wanted to focus research on instead of the latest electronics gadget, then eventually we would solve those problems. In a certain way, technology is the answer, but only if society puts effort/money into moving technology ina certain direction.

Re:How Ironic

[(void*)]

Read carefully. He is talking about software, not technology in general.

Re:How Ironic

[Hard_Code]

I agree. Sitting in our $2000 polycarbonate ergonomic chairs with our lava lamps, and geek chic wardrobe, isolated from "RL", it is all too easy to get wrapped up in the novelty of invention. Now I'm not a luddite, I think this stuff is great. But, like Bill Joy, I have this creeping shadow on my conscience...that perhaps this mad rush forward needs to be analyzed a bit more soberly. What we need to realize is that technology is a tool. It should serve a purpose. A lot of people talk about "progress". What are we "progressing" to, I wish people would ask themselves. We should stop once in a while and ask ourselves what *is* this "manifest destiny" for which we are rushing into the golden west of technology. Civilizations existed quite happily for millenia without the "progess" we think new technologies will bring us. That is being evinced in things like the Long Now Foundation.

Re:Software is a fad.

[Nicolas MONNET]

I beg to differ; it will more likely be much hotter with all those lightbulbs.

Fragile software

[Kaa]

Software is fragile -- but that is a problem only if you don't recognize this fact and are unprepared to deal with it.

Yes, the world depends more and more on software. Yes, there is no such thing as a bug-free piece of code. But failures need not be catastrophic. It is perfectly possible to build robust systems, ones which assume that something will crash and are able to deal with it.

It's all really risk management. Software can fail -- but so can mechanical parts. So far, at least, the biggest problems (like Chernobyl) were caused by human error, not a software glitch.

Robust systems can be build out of fragile components.

Kaa

Re:Fragile software

[Frater 219]

Yes, the world depends more and more on software. Yes, there is no such thing as a bug-free piece of code.

There is, however, software that approaches bug-free as a limit, as opposed to merely replacing one set of bugs with another. I offer TeX as an example.

Re:Fragile software

[Chris Johnson]

No such thing as a bug-free piece of code? Nonsense. Here:

10 GOTO 10

or if you prefer,

X = 200

This is ridiculous, a complete cop-out. What you are really trying to say is, "Above a certain (nowadays common) complexity, software is such a complicated mechanism that the human brain can't keep track of it well enough to avoid mistakes".

This is a DIFFERENT! claim than 'There's no such thing as bug-free software'. It points to obvious possible solutions (don't expect the human brain to keep track of it but assemble big software out of smaller known pieces made to interact in known ways, or alternately stick to writing software that _will_ fit within the human brain) rather than claiming no solution is possible. I would say 'There's no such thing as bug-free software' is very akin to saying 'There is a God'. You can look around and see many things that might suggest it- does that constitute proof? More relevantly, what attitude does this belief encourage? One hopes it encourages robust error checking and loads of sanity checking (when programming, it's helpful to assume the program's environment is insane- early Apple code was tested on an automated 'user from hell' program that just fed in endless random actions and was great at accidentally jockeying the victim program into 'insane' conditions that caused a crash). However, every time I see 'There is no such thing as bug-free software' my first reaction is to think it's being said by some marketroid type making excuses for why they shipped early, buggy, and don't actually plan to fix the broken bits. And thanks to the wonderful suggestibility of the human brain and the phenomenon of justification, those marketroids have heard this little mantra too and to _them_ it means 'software by definition is all screwed up and non-deterministic, so we won't even try, we'll just plan PR campaigns'. _They_ don't have the technical background to put such a remark in context. They will be taking it literally, as a license to produce untrustworthy crap without a bit of self-examination, with no attempt to do better- and it becomes their argument to ship earlier and earlier, buggier and buggier. "You're not done fixing bugs? ...button it up, we ship it just as it is now!"

*sigh*

If you write software, write bug-free software.

If you can't write bug-free software, try.

If you won't try- please, quit writing the stuff. :P

Re:The solution - use Formal Methods

[Philbert Desenex]

"Formal methods" sounds really great at first, as does "better quality". But each phrase contains so much hidden meaning that using them identifies you as someone who hasn't really thought about the issues at all. "Formal methods" falls down because using them assumes that we can specify accurately and precisely what the system should do. And you have to do that specification up front. Such exacting up front specification poses more of a problem than iterative development by service packs and patches. "Better quality" has no specific meaning. The American Society for Quality says: "Quality is a jubjective term for which each person has his or her own definition." This allows each of us, programmers, managers, salesmen or customers to "know" what quality is, and use the phrase informally. Then, when the rubber has to meet the road, you can't actually write code that represents "quality" because it doesn't exist except as a figment of some executive director's imagination.

Re:We ARE too reliant on software

[Paul Johnson]

Misquoting the previous poster: I think wires and electricity should be developed and used to assist us and make our lives easier, however at no point should we become so reliant on electricity that we would not be able to function without it...

The fact is that we are already highly reliant on technology in general. Over here in the UK the last few days have just taught us an object lesson in that, when a few people decided to blockade fuel supplies.

Anyone remember "Connections" by James Burke? The first programme opened with a description of a nationwide power cut in the US in the 60s, and the way in which people just assumed that power and civilisation would be restored RSN. Then he went on to discuss survival options if it had not been restored. 1: get out of the city. 2: find a farm, take and hold it by force. 3: learn to plough. Face it: if the electricity grid goes down and stays down then 90+% of the population of an industrial nation is toast.

The same thing is now happening to software: its becomging ubiquitous to the extent that things don't work without it. Fortunately we have the opportunity to ensure that systemic failures that bring down civilisation are not possible. The question is, will we have the wisdom to take advantage of this?

Paul.

Software developers going too far.

[AFCArchvile]

At my college, my Engineering Graphics class is right after a Java class. The workstations (each one a Pentium 90, 64MB RAM, NT4SP6, ughhh...) are always bogged down with the Java libraries; I always have to run taskmgr.exe and clear out the stupid compiler to free up 14MB of RAM so I can use AutoCAD to its fullest.

Right now I'm in the computer lab, on a Dell OptiPlex GX110 with 256MB of RAM (funny, Windows 2000 regcode stickers on each computer's right side, and they have NT4 installed. One step forward, two steps back). And, for some lame reason, the Real Player "Start Center" is in the taskbar. I closed that damn icon on about 3 stations; it's just 3MB of memory being taken up. I kinda hate this; I'm in a college where people are learning the very programming language I've come to hate: Java. I sincerely think that Java is a lazy excuse for the programmers to clip off minutes from their coding schedule just so they don't have to wait through compiling code. I say, program in C++, the compile process is my most favorite! It is the defining moment which determines whether you have made progress or not.

As for the rest of my opinion on software engineers, see my quote, and think of this quote from TRON: "Oh that'll be great. Programs will start to think and the people will stop!"

A keen understanding of the obvious

[PHAEDRU5]

First, thanks for the review.

That said, humans are fallible. What can we expect but that the artifacts they create will also be fallible? There are assuredly limits on software - limits on computability - but the limits I see here are more limits on human ability to describe solutions to problems that become larger and larger as the simpler ones are mastered.

As for technology as religion, it's not religion to me, it's a source of comfortable living.

Now, if only I could stop my VCR from blinking "12:00....12:00...."

Re:A keen understanding of the obvious

[Black Parrot]

> There are assuredly limits on software - limits on computability - but the limits I see here are more limits on human ability to describe solutions to problems that become larger and larger as the simpler ones are mastered.

I don't really think complexity is the ceiling we're up against. Software benefits much more than most disciplines in its ability to break a complex problem down into simpler pieces.

I think the main problem we're having is that software is, by and large, a marketing discipline rather than an engineering discipline. Obviously so for shrinkwrapware, but to a big extent also for DenverAirportware.

I hold out hope that as our society becomes more and more aware of its dependence on software, it will eventually decide that software that is stable and behaves as expected is every bit as important as bridges that are stable and petrochemical plants that behave as expected. At that point (my dream goes) we'll see marketing, hype, an unrealistic expectations concerning cost and delivery fade into the background, and good design principles move to the foreground.

Once we learned to build good bridges, we quickly learned to build bridges that are aesthetically pleasing as well as good. Maybe when the world learns to build good software, it will soon learn to build featureful software that is still good, too.

Not to say that there aren't limits on computability, and probably also on managable complexity; it's just that with few exceptions, such as the on-board shuttle group, we're running up against social barriers before we get to the computational ones.

--

Re:Software is a fad.

[Nicolas MONNET]

Vacuum tubes .. light bulbs ... who cares?

Anecdote to share

[Mindwarp]

Readng this review brought to mind an anecdote shared by an old professor of mine in a class discussing the need for completeness testing and mathematical proof of algorithms.

The incident related was to do with the testing of a new auto-pilot/terrain following system being installed on the Tornado fighter-bomber (the British air-to-ground attack aircraft). All was going well with the test flight until the aircraft dipped into a valley. Shortly after entering the valley the aircraft inverted and performed the classic 'lawn dart' manouver.

There was of course an insuing investigation and analysis of the auto-pilot software during which it was found that this exact behaviour would occur if the altimeter returned a negative value.

How could this happen? The valley was below sea level.

--

Re:We ARE too reliant on software

[Frater 219]

I think wires and electricity should be developed and used to assist us and make our lives easier, however at no point should we become so reliant on electricity that we would not be able to function without it...

Sorry, too late. Our present level of population cannot be maintained without our present level of technology. More generally, no level of population could be maintained without the corresponding level of technology: you can't support Renaissance Europe on Babylonian-era agriculture.

The corollary of this fact is that any person who advocates "rolling technology back" or "going back to the land" is advocating genocide -- and should be regarded on that level of (im)morality.

Not an entirely fair assessment

[symbolic]

Why single out software? What is being discussed is a problem with virtually *any* complex system, and I personally think it has more to do with the (human) processes of discovery, invention, innovation, and implementation. These are fragile processes, because they require, by nature, the identification and eradication of the various aspects that cause failure. Add to this any additional problems associated with organizatonal dynamics (management, motivation, morale, compensation, interpersonal dynamics, etc.), and it easy to see how something complex can fail.

I don't think it's possible to work out all of the bugs in a piece of software, simply because there are too many unseen variables. But I also don't think it's fair to cast software development in a different light merely because it is subject to the same "rules" that govern any complex system. As a means of comparison, consider how long it takes to implement a new commercial aircraft design. Consider how many defense-related projects are, or have been over budget, poorly designed, or scrapped altogether. It was, after all, a hardware "bug" that caused the explosion of the Challenger shuttle. Hell, I'd argue that it's even a hardware "bug" responsible for the Firestone debacle.

Is it fair to hold software development to a different standard? No matter which complex system you're dealing with, it always involves getting from point A to point B, and it's not always going to be an easy ride.

Re:Not an entirely fair assessment

[snoitpo]

All through the 1800's steam boiler explosions were quite common--steam technology was pushed until it was understood and matured. The raw material used to build boilers improved with the making of steel. Early Comet jets exploded until DeHavilland understood how a previously unknown phenomenon (metal fatigue from repeated pressurizations) caused a failure. But today steam boilers and jet airplanes very very rarely explode.

One difference is that with complex physical systems it is easy enough to build in redundancy. The Brooklyn Bridge has had dozens of diagonal stays (and maybe a few suspender cables) fail over the past century yet there is enough redundancy to prevent its collapse. But if your computer system doesn't have some sort of watchdog timer feature added by the manufacturer you have no protection form a wayward
linelabel BranchUnconditional linelabel
from hanging your system.

Today, we license boiler engineers, civil engineers, plumbers, CPAs, and even hairdressers. Not only does licensing provide revenue for the state, but it also insures that the person holding the license has passed a series of tests demonstrating minimal competence (think Driver's license). Once a licensed xyzzy has [designed a boiler] [designed a bridge] [installed a toilet] [cooked your books] [cut your hair] you can go back and hold that person responsible (i.e. sue them and their company) for a bad boiler/bridge/pipes/audit/shave.

Perhaps this is the next phase of the maturity of the software development profession. (A few years back New Jersey was thinking of this, but mostly as a scheme to raise revenue.) We'll have licensed Software Engineers. They won't be every member of a firm (not all Civil Engineers are necessarily licensed, but all designs are reviewed by them). We'd have to start with an operating environment that someone is willing to sign their name to "it does this-subset-of-functions." From that base, we can have a set of utilities that does another subset of functions, then applications that does the final set of functions.

We already have a software that people have signed their names to--the example that immediately comes to mind are the ascent, descent, and life support software of the Shuttle. The downside is that such software is expensive, but if sold in quantity the unit cost will go down. The APIs of such a system are nice and orthoganol (unlike Windows). Processor technology is at a level where even cheap processors can put processes in their own virtual space and hardware will detect something amiss.

Certifications are a start, but Certificates are not the same as Licenses. A certificate can go stale, but true licensed Professionals (doctors, lawyers, CPAs) have to keep up to keep their license. If they don't, they can be sued for malpractice.

So if I were to predict what's going to happen in the next few years (and I'll be the first to admit you can't predict anything about this business) I'll say that we may see Software Development emerging as a true Profession. A body of software developed by Licensed Professionals will start to be developed, and this licensed software will be used to perform critical tasks (run a Space Station or a nuclear plant, run a microwave, route IP packets). There will still be a lot of "unlicensed" software that no-one cares if it crashes (games, DVDs, toaters, office applications). Most of the programmers working at companies will be licensed, but almost everything going out the door will be reviewed, picked over, prodded, and stretched by someone who will sign their name on it. And if the program crashes badly, that someone will have to fix it, and/or be sued, and/or lose their license.

I'm kind of looking formward to this. I'm not willing to put my name on anything I've written recently, but there have been programs that did a few things and did them well--one program I wrote ran umnodified for about 6 years without stopping. (A work distribution system which during its tenure grew to handle about four times the maximum load it was originally designed for--the UNIX system it ran on was rebooted about twice a year and the program would pick right up from checkpoints. The good old days.)

Risk verus Reward

[FallLine]

It's easy to say that we should avoid any risk of failure. But what does that really mean? The real question we should ask ourselves comes down to risk versus reward. Does the potential benefit outweigh the potential risk? Even where the costs of a total failure are high, that doesn't mean we must avoid it.

Take our telephone infrastructure for instance. It's provided society with tremendous and undeniable benefits. Yet the possibility of a total failure is not an impossibility, and certainly wasn't earlier. Would we really have been better off avoiding it because we could never absolutely insure against catastrophe? I'd argue that the costs of avoiding it, of being overly risk averse, far outweigh the benefits that each and every generation has had. Worst case scenario: the telephone networks in the US fail for a couple days. That'd cause some chaos certainly and it might cost the US billions. But I'd argue that having not used it, or having forced excessively costly redundancy [back when before we had digital technology and the like], would have cost the US most of the benefits of the 20th century (i.e., trillions of dollars).

This is not to say that where affordable precautions become available they should be shunned. That factors into a risk/reward equation. Now certanly there is plenty of room for debate about what is an acceptable level of "risk", but the point is to show that even where apparently extremely costly worst case scenarios exist doesn't necessarily mean avenues of progress should be avoided. Nor do I mean to say that everyone has taken a rational approach to risk/reward; certainly there are some instances where it's been totally out of whack. In the vast majority of cases though, it's been my experience that the people that run these systems are nothing if not risk averse. Because they fear failure more than they probably should, the systems generally are acceptably safe.

Re:Java

[dragonfly_blue]

OK, where should I start. Hmmmm....

• Java does not "always compile on the user (sic) side." It is much more common for Java applets and servlets to be running as middleware on servers these days, for portability and standardization.

  The days of Java applets being downloaded and executed in a JVM in the client's browser are long gone. Microsoft killed them by "extending" the Java Virtual Machine that comes with Internet Explorer (which is bundled with Windows blah blah blah), breaking most "cross-platform" Java code in the process.

• If you're talking about the scripts that are embedded in HTML code, that is javascript, a Netscape invention, and has nothing to do with Java(tm) the Sun programming language. Embedded HTML scripts, of course, are always executed client-side, just like VBScript, ActiveX controls, and DHTML.

• You are so woefully misinformed on this topic that people are mistaking you for a troll. Why don't you go look up the difference between Java and Javascript sometime, instead of just making stuff up that you overheard somebody talking about once?

• If run-time performance is the only thing you care about, perhaps you should take a class in Assembly language. Assembly programs have runtimes on the order of magnitude of 20-100X faster than compiled C code.

I don't usually even bother to respond to people who flame whole languages without even knowing the basics of said languages.

However, your posts are written as if you think you're some sort of programming expert, which you obviously are not, if you can't even tell Java from Javascript.

Why can't code be more spec-like?

[cpeterso]

Unfortunately, there will always be a disconnect between specs and code. Programming languages have cruft and artifacts that will always force code != specs.

Here's an interesting paper called "Haskell vs. Ada vs. C++ vs. Awk vs. ..., An Experiment in Software Prototyping Productivity" about a programming language experiment. The Naval Surface Warfare Center had programmers implement a "geometric region server" (a component of the Navy's AEGIS system) in their language of choice. Program length and development time for each project are compared for the different languages. The surprise "winner" was an extemely concise program written (by the paper's author, of course) in the functional programming lanaguage Haskell. The other languages have so much cruft that eventually makes the code look nothing like the "human readable" spec.

Compare! ;-)


LANGUAGE, LINES OF CODE, LINES OF DOC, DEV TIME (HOURS)
Haskell #1, 85, 465, 10
Haskell #2, 156, 112, 8

My favorite quote from the paper:

"In conducting the independent design review at Intermetrics, there was a significant sense of disbelief. We quote from (CHJ93): "It is significant that Mr. Domanski, Mr. Banowetz, and Dr. Brosgol were all surprised and suspicious when we told them that the Haskell prototype P1 is a complete tested executable program. We provided them with a copy of P1 without explaining that it was a program, and based on preconceptions from their past experience, they had studied P1 under the assumption that it was a mixture of requirements specifications and top level design. They were convinced it was incomplete because it did not address issues such as data structure design and execution order."

oops! my table of programming languages was cutoff

[cpeterso]

LANGUAGE, LINES OF CODE, LINES OF DOC, DEV TIME (HOURS)
Haskell #1, 85, 465, 10
Haskell #2, 156, 112, 8
Ada, 787, 714, 23
Ada9X, 800, 200, 28
C++, 1105, 130
Awk, 250, 150
Lisp, 274, 12, 3

Genocide? Hardly.

[Zach Frey]

Sorry, too late. Our present level of population cannot be maintained without our present level of technology.

Sorry, but you have no idea what you're talking about.

Our present urban population densities cannot be maintained without out present technologies. That's not the same thing as our present level of population. Many of our rural areas have become depopulated over the last decades. Wes Jackson estimates that, according to the best archeological evidence, his area of Kansas supported more population in pre-Columbian times that live there today.

The corollary of this fact is that any person who advocates "rolling technology back" or "going back to the land" is advocating genocide -- and should be regarded on that level of (im)morality.

You are presenting your assertion as fact, and laying a classic ad hominem on anyone who might disagree with your "fact."

If today's technologies are unsustainable (and there's good evidence that they are), isn't it "genocidal" to advocate marching forward blindly until the crash happens? Better, I think, to look for "soft landing" scenarios -- which do involve "back to the land" and "rolling back technology."

There are several disproofs by example for your thesis, but I will simply name one -- the Amish. The Amish way supports a higher population density than the "English" techno-agri-business way, with less environmental impact at the same time. Keep in mind that the Amish do innovate technologically -- the key is that they most emphatically do not worship technology, but subordinate their technological choices to their religious and cultural priorities. If only the rest of us were so wise as to keep technology our servant rather than our master.

Re:Genocide? Hardly.

[FallLine]

Uh well. For one, most of the Amish live on some of the most fertile soil in the United States. Secondly, they're not entirely independent, they sell much of their wares so they can buy certain goods and services. Thirdly, it's a mistake to compare the Amish which support relatively small populations on large plots of land to the even small populations on even larger plots, because those farmers are generating most of the food for people in cities. The Amish simply don't need to produce nearly as much food per acre.

I don't quite get your argument about Kansas. Though I sincerely doubt the indians were able to support high population densities given their primitive agricultural skills, the fact is that Kansas EXPORTS most of their product. So that's a bogus analogy too. In any event, if the Amish lifestyle is so ideal, why don't you join them? I'm sure they'd take you.

Finish that sentence!

[TheDullBlade]

Civilizations existed quite happily for millenia without the "progess" we think new technologies will bring us

...and were then crushed when progressive civilizations decided to move into their territories.

The purpose of progress is not a more pleasurable life (as many have discovered, and enshrined in religions and philosophies that ought to be considered forms of insanity, perfect eternal happiness is simply a matter of learning to ignore your sensory input), but survival. The societies which progress and become stronger crush those which grow more slowly.

Putting the brakes on and deciding that you've progressed far enough is societal suicide.

--------

Re:Java

[dragonfly_blue]

That's weird; according to Epic Games Unreal Techology Faq, they coded the engine entirely in C++, not Java.

As for your lag problems, who knows. I have a P-II 350 with an ancient Voodoo2 card that runs Unreal just fine. I'll grant you that the Microsoft Java Virtual Machine is damned slow to start, though.

Re:One example of a website without Java.

[dragonfly_blue]

You mean,

the same Department of Defense that got rated a D+ in their security audit last week? And we're supposed to look to them for an example?

heh.

Re:population density [getting a little OT... ]

[FallLine]

Fertile soil is created, by nature and by good husbandry

Only to a very limited extent. Some land is simply too inhospitable to support extensive agriculture. What's more water is finite. Some land needs to be externally irrigated, but there simply isn't enough to go around. What you are saying is equivelent to saying that people can make themselves athletic. Sure, to a certain extent. You might see miraculous improvements, but only a few people have the necessary genetics to make it to the Olympics. The Amish are on the Olympics of land. They've done a good job preserving what they have, but they choose the most fertile land they could when they got here.

...and destroyed by bad farming practice

I don't disagree with this. One can do great damage to fertile land. But that doesn't mean the inverse is true, that we can take any land and make it sufficiently arrable.

This planet could support a population of 12,000,000,000, using essentially 19th century technology

I sincerely doubt that. Based on your previously blatent miscontruing of the facts, I doubt your premises.

I know little of Kansas

I don't live in Kansas, I live outside Philly. But I know enough about agriculture and history to know most of the Indians lacked based agricultural skills.

...but in the Australian outback one family often struggles to make a living (grow enough to feed themselves and earn enough to pay for other necessities), on land that once supported more than 200 Aboriganal Australians

Sounds awefully bubbly to me. Though I'm not terribly familiar with the Australian Aboriganess, I suspect they were more hunter-gatherer than they were agricultural, especially given the conditions. So what does it really mean to say on the "same" land. They, most likely, took an entirely different approach to the land. Exploiting and eating things that no Westerner would dream of. Furthermore, I doubt they confined their activities to a single acre...and even if they did, it's not as self-confined as, say, farming can be. You kill a wild animal on your land, and that has an impact on the surrounding area. Though there may be been clusters/tribes/city-states of these Aboriginees, their overall lack of density would allow for certain practices that couldn't be practiced today.

As for the American Indians "primitive agricultural skills", we are talking about a people who lived in harmony with their land for thousands of years.

We're also talking about a people whose population was curbed by the constant threat of infant mortality, starvation, sickness, etc. Yes, in their limited sizes I would agree that they didn't "rape" the land. Because they didn't farm any land, they couldn't easily damage any land. However, without farming they simply could never has sustained a large population. The analogy is a poor one.

I think some agricultural problems today definetly need to be addressed, but it is foolish to assert that the primitive cultures produced more food per acre [or rather that they could have sustained our populations]. You certainly have not made a convincing argument for it. You can't demonstrate that the vastly inhospitable lands in this world can suddenly be turned around into an idealistic Amish community. And you certainly can't demonstrate that ALL [or even most] land can do this.

Re:population density [getting a little OT... ]

[Paul Johnson]

It seems that the myth of the "noble savage" has been replaced by the myth of the "ecological savage". But this myth has no basis in fact.

Mammoths were hunted to extinction by my stone-age ancestors. American Indians used to drive whole herds of buffalo off cliffs merely to save the bother of isolating and killing the one they needed. They also wiped out a lot of the native american megafauna when they first arrived. The moa (large flightless New Zealand bird) was hunted to extinction in around 160 years after the first humans arrived. Tell me your ethnic origin, and I'll find a list of the species your stone-age ancestors drove to extinction.

Oh, and that beautiful speech by Chief Seattle which starts "How can you buy or sell the sky? The land? The idea is strange to us..." was never said by him. It was actually written by a film scriptwriter named Ted Perry for a 1972 documentary. Type their names into Google for more info.

To be sure, many stone age peoples have lived in "harmony" (i.e. rough equilbrium) for many thousands of years. But this is not due to any special wisdom on their part, merely the fact that the ones who didn't went extinct so we don't see them. And my 19th century ancestors who destroyed the dodo and introduced rats and rabbits to Austrailia are no better. We in the 20th century can claim exactly one point of superiority over any of them, which is that we have finally figured out what we are doing. Those who would have us forget that and go back to the stone age have seriously missed the point.

Paul.