To me, this complacency about bugs is a dark cloud over all programming work. - J. Lanier
... be that as it may, it's not the only dark cloud. Here are three more that, each in its own way, address some of the issues raised by Mr. Lanier:
Cloud 1: Information
Let's go back to the middle of the 20th century, to a very brilliant, first generation of serious hackers that included people like Alan Turing, John von Neumann, and Claude Shannon. Their primary source of coding experience involved coding information that could be sent over a wire. - J. Lanier
Claude Shannon, in his paper A Mathematical Theory of Communication
(PDF version)
begins with the oft forgotten statement that, and I'm paraphrasing here (badly),
of course he doesn't really mean "information" as that would imply a whole host of semantic issues he doesn't need to deal with but, what the heck, it's a nice word.
It's certainly an interesting word, but when it comes to what can be stored and/or transmitted, it's the wrong word. That Shannon used it and no one seems to mind speaks volumes about the "programmer's mindset" in the past 50 years and goes a long way towards explaining why GOFAI (good old fashioned AI) has failed so completely.
Information is a phenomenological experience of the mind's response to stimuli. You may find what you're reading here informative, but that does not mean this arrangement of dark and light pixels on your screen is information or that, when I posted this message, information was transmitted. The illusion of information is very real, so real that it makes it difficult to see that behind it is a mind doing some incredibly complex processing and that without mind, there is no information.
The illusion of information is borne on the wings of assumption. Call this a message or posting or whatever, but it is really a designed encoding of data, its design intended to evoke particular responses and guided by my assumption that you read and can comprehend English (one of many assumptions). We are so adept at language that most of what goes on when we speak and listen (or write and read) goes on below the level of consciousness. It takes effort to see that the illusion of information requires mind and even more effort to see that mind requires society (i.e., that minds are social -- they require the shared conventions and knowledge represented by the society they are immersed in).
Anyone who's written even a simple program has experienced the "absence of mind" in the wonderful world of Information Processing/Technology. Without mind, programs are required to simulate particular mind-like functions in order to simulate "information." We state or assume conventions and knowledge (e.g., protocols) in some subroutine so it can "think" about the "information" passed to it. We spend an inordinate amount of effort and time trying to design and follow conventions in our programs. In the "human interface" areas of our programs we expend even more effort trying to get the "stupid" user to supply information according to the conventions our programs are expecting and to decorate (format) information for human consumption.
To the extent that the knowledge and conventions within a program are complete and consistent, the current "art of programming" works surprisingly well. The larger the program, the more difficult the completeness and consistency becomes and the more dangerous the fallacy that information is storable and transmittable becomes. It becomes too easy for a programmer to assume that "information" produced or consumed by some subroutine they're working on is, and will remain informative. One can easily imagine someone at NASA working on a subroutine that transmits thrust information to a spacecraft. Would they ever imagine that what they have encoded in Pounds will be interpreted by another subroutine in Newtons if they believe their subroutine actually transmits the information? (Not that anything like this would
really happen,
of course.)
Cloud 2: System Boundaries
We need a system in which errors are more often proportional to the source of the error.
- J. Lanier
How do we know what is and isn't part of a system? We look outside and see a car, a tree, a person. The tree has a root system. The car has a brake system and an exhaust system. The person has a cardiovascular system. And yet, the best definition I have for a "system membership function" is:
Anything that affects and is a affected by a system is part of that system.
A little thought in that direction will get you to the conclusion that it's pretty much all one system and everything we deal with is a subsystem. We haven't evolved to look at the world this way. In fact, we try not to as such a holistic view is far too complex. We'd be stuck in analysis paralysis before we ever got started. We have a tendency to begin with the smallest version of "system" we can get by with and expand its domain only when it's the only way to accommodate some "outside" fact.
In my experience with programming (and technology, in general), the "source of the error" is frequently outside of the "system" we thought we were dealing with. It's not unusual to find that the destination of the error is outside of the "system" we thought we were dealing with, as well. Programmers are all very aware of the effects of system boundary errors (although most would not recognize them as such). These errors manifest themselves in usually annoying, and sometimes frustrating problems. But the consequences can be far more devastating.
In December of 1996, The Bright Field (a 763-foot freighter loaded with 56,380 long tons of corn) was positioning itself to navigate a turn in the Mississippi River when a primary oil pump failed. Automation software detected the failure and attempted to start a secondary oil pump but it wouldn't start so the automation shut down the engine.
When viewed from the perspective of the "engine system," the automation behaved in a perfectly reasonable manner (and had this occurred on the open sea, everyone would congratulate the automation designers for a job well done). But if you jump up a level you see that shutting off the engine makes it impossible to steer or stop the "ship system." Jump up another level and on that day in December you see that you now have an extremely heavy ship drifting straight towards a New Orleans wharf. (The crew was able to finally get the engine started but not in time to stop the ship. It destroyed 200 feet of dock, tore the front off of a hotel and shopping mall and injured 116 people.)
Cloud 3: Complexity
If you look at other things that people build, like oil refineries, or commercial aircraft, we can deal with complexity much more effectively than we can with software. - J. Lanier
Note: I assume (and hope) Mr. Lanier meant: "we can deal with their complexity much more effectively than we can with software's."
There is a hint in Mr. Lanier's comment of the belief that complexity is somehow undesirable or problematic. In common usage, complexity seems to share some of the semantic space occupied by complicated. Let complicated handle all things messy, difficult and hard to use. Complexity is about connections (physical, social, ideological,...) between things and it is the source of power in anything we think productive or capable of fending off entropy.
Having lived near an oil refinery that blew up I have my doubts about our ability to deal with their complexity, and no one needs reminding of what commercial aircraft are capable of. You may be tempted to come to Mr. Lanier's defense here, stating that this isn't exactly the kind of "dealing with" he meant. To which I would reply: yes I know, but that's exactly why complexity seems like such a problem, often times in very surprising ways.
People, it seems, like to forget that: a) complexity is no less productive just because we didn't intend to create it, and b) complexity is not constrained in any way by our good intentions. Programmers are also fond of thinking of their code as something that simplifies things, but new software always increases the complexity of the system. In fact, when it comes to adding complexity, nothing is faster or easier than software (and this is most likely at the root of Mr. Lanier's comment). It's rather interesting to consider how programmers try simplifying something and end up making things more complex... Some guy named Tim tries to simplify the exchange of scientific documents and, voila! out pops amazon.com.
It's worth noting that all or most of the significant "events" in human history have resulted in (or enabled or supported) huge increases in complexity. Language, agriculture, population expansion, civilization, transportation (roads, ships, etc.), writing, the printing press, electronic communication, and so on; they have all increased the connections (usually both in number and frequency) between people. (It's possible that Kurzweil's theory of exponential growth of technology is focused on an effect rather than a cause. It seems more likely to me that complexity is growing exponentially and, for the moment, technology is a parallel effect of that growth.)
Slashdot Mirror
... be that as it may, it's not the only dark cloud. Here are three more that, each in its own way, address some of the issues raised by Mr. Lanier:
Cloud 1: Information
Let's go back to the middle of the 20th century, to a very brilliant, first generation of serious hackers that included people like Alan Turing, John von Neumann, and Claude Shannon. Their primary source of coding experience involved coding information that could be sent over a wire. - J. Lanier
Claude Shannon, in his paper A Mathematical Theory of Communication (PDF version) begins with the oft forgotten statement that, and I'm paraphrasing here (badly), of course he doesn't really mean "information" as that would imply a whole host of semantic issues he doesn't need to deal with but, what the heck, it's a nice word.
It's certainly an interesting word, but when it comes to what can be stored and/or transmitted, it's the wrong word. That Shannon used it and no one seems to mind speaks volumes about the "programmer's mindset" in the past 50 years and goes a long way towards explaining why GOFAI (good old fashioned AI) has failed so completely.
Information is a phenomenological experience of the mind's response to stimuli. You may find what you're reading here informative, but that does not mean this arrangement of dark and light pixels on your screen is information or that, when I posted this message, information was transmitted. The illusion of information is very real, so real that it makes it difficult to see that behind it is a mind doing some incredibly complex processing and that without mind, there is no information.
The illusion of information is borne on the wings of assumption. Call this a message or posting or whatever, but it is really a designed encoding of data, its design intended to evoke particular responses and guided by my assumption that you read and can comprehend English (one of many assumptions). We are so adept at language that most of what goes on when we speak and listen (or write and read) goes on below the level of consciousness. It takes effort to see that the illusion of information requires mind and even more effort to see that mind requires society (i.e., that minds are social -- they require the shared conventions and knowledge represented by the society they are immersed in).
Anyone who's written even a simple program has experienced the "absence of mind" in the wonderful world of Information Processing/Technology. Without mind, programs are required to simulate particular mind-like functions in order to simulate "information." We state or assume conventions and knowledge (e.g., protocols) in some subroutine so it can "think" about the "information" passed to it. We spend an inordinate amount of effort and time trying to design and follow conventions in our programs. In the "human interface" areas of our programs we expend even more effort trying to get the "stupid" user to supply information according to the conventions our programs are expecting and to decorate (format) information for human consumption.
To the extent that the knowledge and conventions within a program are complete and consistent, the current "art of programming" works surprisingly well. The larger the program, the more difficult the completeness and consistency becomes and the more dangerous the fallacy that information is storable and transmittable becomes. It becomes too easy for a programmer to assume that "information" produced or consumed by some subroutine they're working on is, and will remain informative. One can easily imagine someone at NASA working on a subroutine that transmits thrust information to a spacecraft. Would they ever imagine that what they have encoded in Pounds will be interpreted by another subroutine in Newtons if they believe their subroutine actually transmits the information? (Not that anything like this would really happen, of course.)
Cloud 2: System Boundaries
We need a system in which errors are more often proportional to the source of the error. - J. Lanier
How do we know what is and isn't part of a system? We look outside and see a car, a tree, a person. The tree has a root system. The car has a brake system and an exhaust system. The person has a cardiovascular system. And yet, the best definition I have for a "system membership function" is: Anything that affects and is a affected by a system is part of that system.
A little thought in that direction will get you to the conclusion that it's pretty much all one system and everything we deal with is a subsystem. We haven't evolved to look at the world this way. In fact, we try not to as such a holistic view is far too complex. We'd be stuck in analysis paralysis before we ever got started. We have a tendency to begin with the smallest version of "system" we can get by with and expand its domain only when it's the only way to accommodate some "outside" fact.
In my experience with programming (and technology, in general), the "source of the error" is frequently outside of the "system" we thought we were dealing with. It's not unusual to find that the destination of the error is outside of the "system" we thought we were dealing with, as well. Programmers are all very aware of the effects of system boundary errors (although most would not recognize them as such). These errors manifest themselves in usually annoying, and sometimes frustrating problems. But the consequences can be far more devastating.
In December of 1996, The Bright Field (a 763-foot freighter loaded with 56,380 long tons of corn) was positioning itself to navigate a turn in the Mississippi River when a primary oil pump failed. Automation software detected the failure and attempted to start a secondary oil pump but it wouldn't start so the automation shut down the engine. When viewed from the perspective of the "engine system," the automation behaved in a perfectly reasonable manner (and had this occurred on the open sea, everyone would congratulate the automation designers for a job well done). But if you jump up a level you see that shutting off the engine makes it impossible to steer or stop the "ship system." Jump up another level and on that day in December you see that you now have an extremely heavy ship drifting straight towards a New Orleans wharf. (The crew was able to finally get the engine started but not in time to stop the ship. It destroyed 200 feet of dock, tore the front off of a hotel and shopping mall and injured 116 people.)
Cloud 3: Complexity
If you look at other things that people build, like oil refineries, or commercial aircraft, we can deal with complexity much more effectively than we can with software. - J. Lanier
Note: I assume (and hope) Mr. Lanier meant: "we can deal with their complexity much more effectively than we can with software's."
There is a hint in Mr. Lanier's comment of the belief that complexity is somehow undesirable or problematic. In common usage, complexity seems to share some of the semantic space occupied by complicated. Let complicated handle all things messy, difficult and hard to use. Complexity is about connections (physical, social, ideological, ...) between things and it is the source of power in anything we think productive or capable of fending off entropy.
Having lived near an oil refinery that blew up I have my doubts about our ability to deal with their complexity, and no one needs reminding of what commercial aircraft are capable of. You may be tempted to come to Mr. Lanier's defense here, stating that this isn't exactly the kind of "dealing with" he meant. To which I would reply: yes I know, but that's exactly why complexity seems like such a problem, often times in very surprising ways.
People, it seems, like to forget that: a) complexity is no less productive just because we didn't intend to create it, and b) complexity is not constrained in any way by our good intentions. Programmers are also fond of thinking of their code as something that simplifies things, but new software always increases the complexity of the system. In fact, when it comes to adding complexity, nothing is faster or easier than software (and this is most likely at the root of Mr. Lanier's comment). It's rather interesting to consider how programmers try simplifying something and end up making things more complex... Some guy named Tim tries to simplify the exchange of scientific documents and, voila! out pops amazon.com.
It's worth noting that all or most of the significant "events" in human history have resulted in (or enabled or supported) huge increases in complexity. Language, agriculture, population expansion, civilization, transportation (roads, ships, etc.), writing, the printing press, electronic communication, and so on; they have all increased the connections (usually both in number and frequency) between people. (It's possible that Kurzweil's theory of exponential growth of technology is focused on an effect rather than a cause. It seems more likely to me that complexity is growing exponentially and, for the moment, technology is a parallel effect of that growth.)