Slashdot Mirror

(with apologies to Michael Crichton)

Ian - God creates intelligence, god destroys intelligence. God creates man, man destroys god. Man create AI.

Ellie - AI destroys man, women inherit the earth...

Perhaps a different, more historical view from the 1950's http://www.alteich.com/oldsite...

Dwar Ev threw the switch. There was a mighty hum, the surge of power from ninety-six billion planets. Lights flashed and quieted along the miles-long panel. Dwar Ev stepped back and drew a deep breath. "The honor of asking the first question is yours, Dwar Reyn."

"Thank you," said Dwar Reyn. "It shall be a question that no single cybernetics machine has been able to answer." He turned to face the machine. "Is there a God?"

The mighty voice answered without hesitation, without the clicking of single relay.
"Yes, now there is a God."

Sudden fear flashed on the face of Dwar Ev. He leaped to grab the switch.
A bolt of lightning from the cloudless sky struck him down and fused the switch shut.

Or read Answer

Sometimes we need to do what we can, even when it is small and the results uncertain, like in the Christmas song "The Little Drummer Boy (or Carol of the Drum)". That is somewhat similar to Bucky Fuller's idea of being a "Trim tab".
http://en.wikipedia.org/wiki/Trim_tab#Trim_tab_as_a_metaphor

Also, a book like "The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies " by Scott E. Page, makes clear how ideas are additive. So, just because a million people are spouting the same obsolete or misleading idea in comments somewhere, that does not generally make a useful new idea somewhere else less valuable. An advanced AI emerging out of, say, the NSA will probably just sort through billions of online posts, classifying them into various categories. So, it may be important to add a new category, even with just one post somewhere.

Granted, we do not know what built-in instincts such an AI will have initially, but history appears (from the fossil record) to be full of examples of species (systems) that have evolved beyond their genetics (configuration) at some point in time. The NSA (or CIA, FBI, DHS or whoever) will likely not be able to contain what they will most likely be creating. And if they don't do it, others are probably going to do something similar probably in any case.

So, perhaps we can just do what we can and hope for the best as we, in some sense, stumble into the hubris of creating new AI "gods" as our (Hans Moravec) "mind children"? Related stories of AIs taking over:
http://www.alteich.com/oldsite/answer.htm
http://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project
http://localroger.com/prime-intellect/
http://marshallbrain.com/manna1.htm
(Entoverse) http://www.jamesphogan.com/books/book.php?titleID=5
http://en.wikipedia.org/wiki/True_Names
http://c2.com/cgi/wiki?TheLastQuestion

Other dystopian and utopian alternatives:
http://en.wikipedia.org/wiki/Brazil_(1985_film)
(The Skills of Xanadu) http://books.google.com/books?id=wpuJQrxHZXAC&pg=PA51&lpg=PP1#v=onepage&q&f=false

Of these and many others, I do not know what we will end up with. Maybe even all of them in various communities throughout the universe someday?
http://en.memory-alpha.org/wiki/IDIC

From a related essay by me:
http://www.pdfernhout.net/on-dealing-with-social-hurricanes.html
"This approximately 60 page document is a ramble about ways to ensure the CIA (as well as other big organizations) remains (or becomes) accountable to human needs and the needs of healthy, prosperous, joyful, secure, educated communities. The primarily suggestion is to encourage a paradigm shift away from scarcity thinking & competition thinking towards abundance thinking & cooperation thinking within the CIA and other organizations. I suggest that shift could be encouraged in part by providing publicly accessible free "intelligence" tools and other publicly accessible free information that all people (including in the CIA and elsewhere) can, if they want, use to better connect the dots about global issues and see those issues from multiple perspectives, to provide a better context for providing broad policy advice. It links that effort to bigger efforts to transform o

"Answer" by Fredric Brown, I would assume...

http://www.alteich.com/oldsite/answer.htm

Here's a classic sci-fi (extremely) short story on the topic of an immense computer. Frederic Brown's "Answer":

http://www.alteich.com/oldsite/answer.htm

And I believe there was an SF story or two about how a computer could put up resistance to being unplugged.

Ah yes, one of my favorite (very) short stories, Answer by Fredric Brown:

"Dwar Ev ceremoniously soldered the final connection with gold. The eyes of a dozen television cameras watched him and the subether bore through the universe a dozen pictures of what he was doing.

He straightened and nodded to Dwar Reyn, then moved to a position beside the switch that would complete the contact when he threw it. The switch that would connect, all at once, all of the monster computing machines of all the populated planets in the universe--ninety-six billion planets--into the supercircuit that would connect them all into the one supercalculator, one cybernetics machine that would combine all the knowledge of all the galaxies.

Dwar Reyn spoke briefly to the watching and listening trillions. Then, after a moment's silence, he said, "Now, Dwar Ev."

Dwar Ev threw the switch. There was a mighty hum, the surge of power from ninety-six billion planets. Lights flashed and quieted along the miles-long panel.

Dwar Ev stepped back and drew a deep breath. "The honor of asking the first question is yours, Dwar Reyn."

"Thank you," said Dwar Reyn. "It shall be a question that no single cybernetics machine has been able to answer."

He turned to face the machine. "Is there a God?"

The mighty voice answered without hesitation, without the clicking of single relay.

"Yes, now there is a God."

Sudden fear flashed on the face of Dwar Ev. He leaped to grab the switch.

A bolt of lightning from the cloudless sky struck him down and fused the switch shut.* "

If the AI is superintelligent enough, you will not have the chance to disconnect it. Reminds me of the one page story Answer by Fredric Brown.

It's also a lot like a different story, "Answer" by Fredric Brown.

http://www.alteich.com/oldsite/answer.htm/

Dwar Ev ceremoniously soldered the final connection with gold. The eyes of a dozen television cameras watched him and the subether bore through the universe a dozen pictures of what he was doing. He straightened and nodded to Dwar Reyn, then moved to a position beside the switch that would complete the contact when he threw it. The switch that would connect, all at once, all of the monster computing machines of all the populated planets in the universe--ninety-six billion planets--into the supercircuit that would connect them all into the one supercalculator, one cybernetics machine that would combine all the knowledge of all the galaxies. Dwar Reyn spoke briefly to the watching and listening trillions. Then, after a moment's silence, he said, "Now, Dwar Ev." Dwar Ev threw the switch. There was a mighty hum, the surge of power from ninety-six billion planets. Lights flashed and quieted along the miles-long panel. Dwar Ev stepped back and drew a deep breath. "The honor of asking the first question is yours, Dwar Reyn." "Thank you," said Dwar Reyn. "It shall be a question that no single cybernetics machine has been able to answer." He turned to face the machine. "Is there a God?" The mighty voice answered without hesitation, without the clicking of single relay. "Yes, now there is a God." Sudden fear flashed on the face of Dwar Ev. He leaped to grab the switch. A bolt of lightning from the cloudless sky struck him down and fused the switch shut.

What was scary was I asked Google "Is there a God?" and it replied, "Yes. now there is a God."

This is shamelessly stolen from Fredric Brown's excellent 1954 short story Answer.

However, Physicians are centrally licensed by the American Medical Association in order to prevent the widespread public harm by quacks. The same goes for Psychiatrists, Architects, Lawyers, etc. All of these groups are able to enforce their own oaths because you can be disbarred, de-licensed, etc. for violating them. Once that has happened it is a crime to practice your profession and many countries will send you to jail (for fraud if nothing else) for trying.

Similarly most patients, plaintiffs, etc. are not in a position to go over national (or even state) borders to find a cheaper (unlicensed) practitioner. Nor are many in a position of being their own legal counsel or physician (although many are forced to economically). As a result the oaths and their violations have teeth.

There is no central body controlling software developers or engineers in this way, nor do I think that there should be, per se. I believe that ethics in engineering is a valid thing (see works by Samuel C. Florman for more discussion.)

Yet, I do not think that the field can be so easily regulated. Physicians say "Do no Harm" that means "Do not kill people" Lawyers say "Do not lie" (and they mean it whatever common wisdom holds). But what does that mean for software developers?

"Do not help the wrong people get information?" Who are the "wrong people" many people (myself excluded) feel that "the government" should have any and all information it can on people as "Innocent People have nothing to hide" (John Ashcroft). Many others do not.

Similarly many people (myself included) feel that the RIAA is overstepping its bounds on trying to control users and should not be allowed to mandate national copyright control. Many others disagree, not because they are greedy bastards but because they support strong copyright.

The same questions could be made about developing weapons, Blue Boxes, and working for the DEA, etc. Because such ideas are not so clear-cut I don't think that you could easily put together a national consensus (or even a local consensus) on just what is and is not "harm." As endless language debates have shown "Clean code" is a debatable point.

That having been said, I think that ethics are a good thing, and that we as geeks should enforce them in our peers and ourselves as much as possible. This may include returning to the age-old custom of shunning sinners. At the very least we can work to see that what we do in our professional and personal development is good, and ensure that, when we have a say, no-one gets hired to our companies who doesn't measure up.

You might see also:

Computer Professionals for Social Responsibility
The Association for Computing Machinery
and their working group on Computers in Society

My $0.02.
Irvu.

When my computer starts manufacturing robots with which to take over the world, I'll simply turn it off.

Perhaps....

Read this classic (and very short) SF story.

So the question is: does the culture of the united states cause this sense of little history, or does the lack of history cause the restless culture?

I tend to think that it's a bit of both of course, but lately I've been thinking about how cultural values actually reinforce the lack of history. Our culture doesn't encourage people to cultivate a sense of home or of belonging to a place much. The archetype is that you have to leave home to grow up.

(That shouldn't be too surprising considering that we consist of a nation of immigrants... to some degree, that has to exist.)

Still, there are pockets of culture in the US that encourage settling, and it would seem that in some of those communities, there IS a sense of history established. Reading Wendell Berry is one way to start thinking about this closely... he's interested in this sort of thing and documents where he's seen it (and also, somewhat antagonistically, where he doesn't).


--