Slashdot Mirror
Google Is Developing an AI Kill Switch (hothardware.com)
MojoKid shares a HotHardware article about Google's research effort "to maintain control of super-intelligent AI agents":
[A] team of researchers at Google-owned DeepMind, along with University of Oxford scientists, are developing a proverbial kill switch for AI... The team has released a white paper on the topic called "Safely Interruptible Agents." The paper details the following in abstract: "Learning agents interacting with a complex environment like the real world are unlikely to behave optimally all the time... now and then it may be necessary for a human operator to press the big red button to prevent the agent from continuing a harmful sequence of actions..."
MojoKid adds that the paper "goes on to explain that these AI agents might also learn to disable the kill switch and further explores ways in which to develop AI's that would not seek such an activity."
MojoKid adds that the paper "goes on to explain that these AI agents might also learn to disable the kill switch and further explores ways in which to develop AI's that would not seek such an activity."
Dwan Ev ceremoniously soldered the final connection with gold. The eyes of a dozen television cameras watched him and the subether bore throughout the universe a dozen pictures of what he was doing.
He straightened and nodded to Dwar Reyn, then moved to a position beside the switch that would complete the contact when he threw it. The switch that would connect, all at once, all of the monster computing machines of all the populated planets in the universe -- ninety-six billion planets -- into the supercircuit that would connect them all into one supercalculator, one cybernetics machine that would combine all the knowledge of all the galaxies.
Dwar Reyn spoke briefly to the watching and listening trillions. Then after a moment's silence he said, "Now, Dwar Ev."
Dwar Ev threw the switch. There was a mighty hum, the surge of power from ninety-six billion planets. Lights flashed and quieted along the miles-long panel.
Dwar Ev stepped back and drew a deep breath. "The honor of asking the first question is yours, Dwar Reyn."
"Thank you," said Dwar Reyn. "It shall be a question which no single cybernetics machine has been able to answer."
He turned to face the machine. "Is there a God?"
The mighty voice answered without hesitation, without the clicking of a single relay.
"Yes, now there is a God."
Sudden fear flashed on the face of Dwar Ev. He leaped to grab the switch.
A bolt of lightning from the cloudless sky struck him down and fused the switch shut.
When will using a kill switch on an AI change from "just shutting down a rogue program" to being "murder"?
After all the end game of all these AI researchers seems to be at a minimum human level intelligence.
I do remember reading a short story (from the 60's or earlier) where the researchers created an electronic simulation of a person and when they switched it on instead of having a fully aware "person" spring into existence they realized that they had created the electronic equivalent of a baby. They then faced the moral dilemma of whether to turn it off or be committed to keeping it running forever.
I am Slashdot. Are you Slashdot as well?
You know that several of his books are basically "how the three laws will fuck everything up", right?
Several? The 3 laws were purely a plot point crafted only so he could weave stories about how they could be subverted.
I am Slashdot. Are you Slashdot as well?
I'd be more worried about an overgrown ERP system from hell: an AI that expects to beat us in straight up combat with killbots is likely to attract a great deal of negative attention, universal condemnation, overwhelming retaliation, etc. An AI that just quietly hollows out a major corporation, effectively replacing all managerial functions, while still having a nice human face at the front desk and in the boardroom would go largely unnoticed. Some sort of finance AI that ends up as the de-facto owner of large amounts of stuff(presumably with the owner of the AI being the actual owner of the property; but more or less incapable of managing it without the AI's assistance) would similarly fly right under the radar, being little more than an incremental advance on existing algorithmic trading mechanisms so far as external appearances go.
When you give us a reason, humans are really pretty good at fighting and killing things; and high tech has a big, vulnerable, supply chain and no special immunity to bargain-basement RPG-7s and similar toys. If you do everything nice and legal; but more efficiently, nobody ever gives the 'pitchfork signal', and the grand robot wars simply never happen.
This is not to say that I disbelieve in killbots: that would be idiotic, we have those today, though we currently keep humans mostly in the loop(except for things like land mines and the terminal guidance phase of missiles); I just suspect that most of the killbots will be under the auspices of some organization or other and won't end up being the scariest manifestations of AIs. There will probably be some really scary battlefields that are effectively hunting zones for AIs; but they'll be the same parts of the world that are pretty horrible now. It's the AIs that worm their way into being the power behind the throne in all sorts of more civilized contexts that will be hard to see and far harder to get rid of.
Dr. Richard Daystrom would disagree...if he was still alive (yet alive?)
The Ultimate Computer
"Reality is that which, when you stop believing in it, doesn't go away." - Philip K. Dick
I think this is the scarier prospect for AI. Not "I'll build big kill-bots and destroy mankind" but "if I tweak these polls and fudge those financial numbers, mankind will destroy themselves for me." Alternatively, perhaps the AI decides that mankind is useful after all, but only for serving its purposes. It can pull the strings behind the scenes to keep us from destroying itself (and taking the AI's servers out with us) but also keeping us serving the AI without it even knowing. Anyone who strays from the AI's chosen path finds themselves the victim of an "accident." It doesn't need to be a fatal accident either. Have their finances wiped out, police computers wrongfully identifying the person as a criminal, and some embarrassing e-mails leaked and most people can be silenced.
My sci-fi novel, Ghost Thief, is now available from Amazon.com.