Slashdot Mirror
What Will Happen When Killer Robots Get Hijacked? (marketwatch.com)
"Imagine an artificial-intelligence-driven military drone capable of autonomously patrolling the perimeter of a country or region and deciding who lives and who dies, without a human operator. Now do the same with tanks, helicopters and biped/quadruped robots." A United Nations conference recently decided not to ban these weapons systems outright, but to revisit the topic in November.
So a MarketWatch columnist looked at how these weapons systems could go bad -- and argues the risks are greater than simply fooling the AI into malfunctioning. What about hijacking...? In warfare, AI units can function autonomously, but in the end they need a way to communicate with one another and to transfer data to a command center. This makes them vulnerable to hacking and hijacking. What would happen if one of these drones or robots was hijacked by an opposite faction and started firing on civilians? A hacker would laugh. Why? Because he wouldn't hijack just one. He would design a self-propagating virus that would spread throughout the AI network and infect all units in the vicinity, as well as those communicating with them. In a split second, an entire squad of lethal autonomous weapons systems would be under enemy control... Every machine can be overridden, tricked, hijacked and manipulated with an efficiency that's unheard of in the realm of human-operated traditional weaponry.
However, the U.S. government remains oblivious. DARPA (Defense Advanced Research Projects Agency) has already announced a $2 billion development campaign for the next wave of technologically advanced AI (dubbed "AI Next"). One of the goals is to have the machines "acquire human-like communication and reasoning capabilities, with the ability to recognize new situations and environments and adapt to them." I may be overreaching here, but the UN meeting on one end and this announcement on the other, make me think that the U.S. government isn't just pro-robotic -- it may already have a lethal autonomous weapons ace up its sleeve.
The article ends with a question: What do you think about killer robots replacing human combatants?
And what would happen if killer robots got hijacked?
So a MarketWatch columnist looked at how these weapons systems could go bad -- and argues the risks are greater than simply fooling the AI into malfunctioning. What about hijacking...? In warfare, AI units can function autonomously, but in the end they need a way to communicate with one another and to transfer data to a command center. This makes them vulnerable to hacking and hijacking. What would happen if one of these drones or robots was hijacked by an opposite faction and started firing on civilians? A hacker would laugh. Why? Because he wouldn't hijack just one. He would design a self-propagating virus that would spread throughout the AI network and infect all units in the vicinity, as well as those communicating with them. In a split second, an entire squad of lethal autonomous weapons systems would be under enemy control... Every machine can be overridden, tricked, hijacked and manipulated with an efficiency that's unheard of in the realm of human-operated traditional weaponry.
However, the U.S. government remains oblivious. DARPA (Defense Advanced Research Projects Agency) has already announced a $2 billion development campaign for the next wave of technologically advanced AI (dubbed "AI Next"). One of the goals is to have the machines "acquire human-like communication and reasoning capabilities, with the ability to recognize new situations and environments and adapt to them." I may be overreaching here, but the UN meeting on one end and this announcement on the other, make me think that the U.S. government isn't just pro-robotic -- it may already have a lethal autonomous weapons ace up its sleeve.
The article ends with a question: What do you think about killer robots replacing human combatants?
And what would happen if killer robots got hijacked?
You don't need to steal weapons from the military to get killer robots. You just need to botnet cars. They already kill people without help.
Let me driver-assist you to safety.
And what would happen if killer robots got hijacked?
When killer humans (i.e. a military) gets 'hijacked', it's called a coup.
It's unlikely the autonomous weapons would be hacked to fire on civilians. It'd be MUCH more effective to turn them into robotic sleeper agents that target military officials/visiting politicians (thanks to facial recognition). OTOH, firing on civilians would ensure a) civilians demand their removal, ensuring no further hacking can take place; b) their hacking is discovered immediately, ensuring the vulnerability is quickly patched, and c) an opportunity to take out more-important targets is squandered.
Corruption is convincing someone that the selfless ideal is the same as their selfish ideal.
Replacing human combatants with robots is a bad policy decision in general. The reason is simple: one of the main reasons restraining governments from waging war is the cost in terms of their own citizens killed in the fighting, removing humans from the fighting removes that restraint. We'll have enough problems with governments that already don't care about their own citizens, we don't need to add every other government to that list until we figure out another way to discourage them from starting wars any time they don't get their way.
On top of that, the possibility of robots being subverted by attackers isn't in any way overstated nor are the reactions to the possibility over-reacting. Look at our computer networks today and try to convince me that we can somehow make botnets and malware vanish overnight, and then picture a world where "distributed denial of service attack" translates to "security guards shooting any human who enters the shopping mall".
Every machine can be overridden, tricked, hijacked and manipulated with an efficiency that's unheard of in the realm of human-operated traditional weaponry.
It isn't really all that difficult to buy, bribe, threaten or convince a human.
The difference is scale. Humans are polymorphic, so they are not exact copies of each other and the identical exploit will work on one, but not others. So you need to customize your exploit for each of them, which makes mass hacks difficult. That is the reason social engineering works, but is rarely used large-scale.
Assorted stuff I do sometimes: Lemuria.org
What if a SUV which has got 500 horse powers and weighs three tons gets hijacked? One needs to study for years to pilot an UAV, but everybody can drive.
In my opinion there should be a legal limit to personal car power and weight; say 100 hp and 1500 kg.
And it is not "maybe", it is happening. About one and a half million(!) people are being killed on roads each year by cars globally. Times more badly injured. These are figures of a WW3. These accidents are related to drinking, suicides, mental illness, terrorism, drugs, etc.
And practically nothing is done about it. Even more powerful and massive cars hit the market. So, please, stop blaming drones. Let us first learn how to handle the real problem at hands.
This is a great point - it's unclear to me what would stop them. By now we know a few things about the bulk of the owners of capital:
1. They don't believe in Democracy
2. They are motivated, and don't allow pesky things like laws to stand in their way
3. They have the patience to work across generations
Think of Trump and his children. These people want to bring about a new feudalism and bring us all back to Game of Thrones, and the Dragons are the killer robots. Human nature doesn't change.
We need to outlaw this now, before it is too late.