Slashdot Mirror
Pentagon Chiefs Fear Advanced Robot Weapons Wiping Out Humanity (mirror.co.uk)
Longtime reader schwit1 writes: Huge technological leaps forward in drones, artificial intelligence and autonomous weapon systems must be addressed before humanity is driven to extinction, say chiefs of Pentagon
From a report: Air Force General Paul Selva, the Vice Chairman of the Joint Chiefs of Staff at the US Defense Department, said so-called thinking weapons could lead to: "Robotic systems to do lethal harm... a Terminator without a conscience." When asked about robotic weapons able to make their own decisions, he said: "Our job is to defeat the enemy" but "it is governed by law and by convention." He says the military insists on keeping humans in the decision-making process to "inflict violence on the enemy. [...] That ethical boundary is the one we've draw a pretty fine line on. It's one we must consider in developing these new weapons," he added. Selva said the Pentagon must reach out to artificial intelligence tech firms that are not necessarily "military-oriented" to develop new systems of command and leadership models, reports US Naval Institute News .
From a report: Air Force General Paul Selva, the Vice Chairman of the Joint Chiefs of Staff at the US Defense Department, said so-called thinking weapons could lead to: "Robotic systems to do lethal harm... a Terminator without a conscience." When asked about robotic weapons able to make their own decisions, he said: "Our job is to defeat the enemy" but "it is governed by law and by convention." He says the military insists on keeping humans in the decision-making process to "inflict violence on the enemy. [...] That ethical boundary is the one we've draw a pretty fine line on. It's one we must consider in developing these new weapons," he added. Selva said the Pentagon must reach out to artificial intelligence tech firms that are not necessarily "military-oriented" to develop new systems of command and leadership models, reports US Naval Institute News .
His concern isn't entirely unjustified. We're increasingly relying on robots to do the actual killing, but we've currently designed the systems so that humans need to be involved in the decision making. The fact that we're involving humans puts a natural bottleneck on our operations, since there's only have so much attention we can give. At some point (e.g. World War 3), it be seen as more efficient to launch a fleet of drones that vastly outnumbers our pilots, equip each with facial recognition systems and a list of targets, and tell them to kill on sight.
I'm not saying it's a good idea, but there's no denying that it would be an efficient way to get the job of killing done, and that it's the sort of measure a country might turn to in desperate times.
But at that point, we'd be just one bug away from a system that produces false positives and starts gunning down everyone in sight. We're just talking about faulty weapons, not machines that can think or understand what they're doing. But if they're deployed en masse, a single bug could have catastrophic results, in much the same way that landmines have remained a problem in many parts of the world, decades after the wars that put them there had ended. This isn't Skynet or an AI intent on world domination. This is simply a machine with a bit too much responsibility.
His concern isn't entirely unjustified. We're increasingly relying on robots to do the actual killing, but we've currently designed the systems so that humans need to be involved in the decision making.
Forget the military drones. (Or at least, they're a smaller component of the overall issue.)
We have computer-controlled cars. They will be deployed in massive numbers over the next ten years. If remote updates are possible, anyone who can update a popular model has access to a distributed weapon of mass destruction capable of causing hundreds of thousands of deaths in a matter of moments.
Warfare-oriented tech isn't the only vector for mass attacks.
Real lawyers write in C++