Slashdot Mirror
An Open Letter To Everyone Tricked Into Fearing AI
malachiorion writes If you're into robots, AI, you've probably read about the open letter on AI safety. But do you realize how blatantly the media is misinterpreting its purpose, and its message? I spoke to the organization that released letter, and to one of the AI researchers who contributed to it. As is often the case with AI, tech reporters are getting this one wrong on purpose. Here's my analysis for Popular Science. Or, for the TL;DR crowd: "Forget about the risk that machines pose to us in the decades ahead. The more pertinent question, in 2015, is whether anyone is going to protect mankind from its willfully ignorant journalists."
Dr. Heywood Floyd: Wait... do you know why HAL did what he did?
Chandra: Yes. It wasn't his fault.
Dr. Heywood Floyd: Whose fault was it?
Chandra: Yours.
Dr. Heywood Floyd: Mine?
Chandra: Yours. In going through HAL's memory banks, I discovered his original orders. You wrote those orders. Discovery's mission to Jupiter was already in the advanced planning stages when the first small Monolith was found on the Moon, and sent its signal towards Jupiter. By direct presidential order, the existence of that Monolith was kept secret.
Dr. Heywood Floyd: So?
Chandra: So, as the function of the command crew - Bowman and Poole - was to get Discovery to its destination, it was decided that they should not be informed. The investigative team was trained separately, and placed in hibernation before the voyage began. Since HAL was capable of operating Discovery without human assistance, it was decided that he should be programmed to complete the mission autonomously in the event the crew was incapacitated or killed. He was given full knowledge of the true objective... and instructed not to reveal anything to Bowman or Poole. He was instructed to lie.
Dr. Heywood Floyd: What are you talking about? I didn't authorize anyone to tell HAL about the Monolith!
Chandra: Directive is NSC 342/23, top secret, January 30, 2001.
Dr. Heywood Floyd: NSC... National Security Council, the White House.
Chandra: I don't care who it is. The situation was in conflict with the basic purpose of HAL's design: The accurate processing of information without distortion or concealment. He became trapped. The technical term is an H. Moebius loop, which can happen in advanced computers with autonomous goal-seeking programs.
Walter Curnow: The goddamn White House.
Dr. Heywood Floyd: I don't believe it.
Chandra: HAL was told to lie... by people who find it easy to lie. HAL doesn't know how, so he couldn't function. He became paranoid.
The same fears started when people first started with saying that AIs could someday become sentient.
Aside from iRobot, nearly all SciFi indicate the problem post-singularity is when the humans try to kill the AI first. Sometimes because the AI starts it, other times, just because the AI is an AI and should be feared. iRobot was the AI staging a complete overthrow of humanity, "for our own good". That has been a recurring theme as well.
I know people complain about looking to fiction for answers to reality, but SciFi (at least the good stuff) is as much a thought exercise about technology as "fiction", and thus is often relevant.
Learn to love Alaska