Wired Founding Editor Now Challenges 'The Myth of A Superhuman AI' (backchannel.com)

Wired's founding executive editor Kevin Kelly wrote a 5,000-word takedown on "the myth of a superhuman AI," challenging dire warnings from Bill Gates, Stephen Hawking, and Elon Musk about the potential extinction of humanity at the hands of a superintelligent constructs. Slashdot reader mirandakatz calls it an "impeccably argued debunking of this pervasive myth." Kelly writes: Buried in this scenario of a takeover of superhuman artificial intelligence are five assumptions which, when examined closely, are not based on any evidence... 1.) Artificial intelligence is already getting smarter than us, at an exponential rate. 2.) We'll make AIs into a general purpose intelligence, like our own. 3.) We can make human intelligence in silicon. 4.) Intelligence can be expanded without limit. 5.) Once we have exploding superintelligence it can solve most of our problems... If the expectation of a superhuman AI takeover is built on five key assumptions that have no basis in evidence, then this idea is more akin to a religious belief -- a myth
Kelly proposes "five heresies" which he says have more evidence to support them -- including the prediction that emulating human intelligence "will be constrained by cost" -- and he likens artificial intelligence to the physical powers of machines. "[W]hile all machines as a class can beat the physical achievements of an individual human...there is no one machine that can beat an average human in everything he or she does."

Comments

Re:Well it's easy to show superhuman AI is a myth.

[Visarga]

This kind of problem will be very visible in healthcare. Human doctors will self censor for fear of contradicting AI and taking the wrong choice, that ends up badly for the patient. Because AI is right most of the time, who will have the courage of saying otherwise? Saying truth to AI could cost a person their job. Many doctors stop giving honest feedback the moment they hear another doctor has given a diagnosis out of solidarity with their colleagues or fear of the consequences of making enemies.

Re:"constrained by cost"

[angel'o'sphere]

Indeed!
And one of the myricals in this is: if an object is about to hit your eyes or comes close by, the reflext to close the eyes and raise your hands etc. is triggered _before_ that information has even reached the brain/visual cortex.
The signal processing in the eye can bypass the visual cortex to trigger protective actions.