Slashdot Mirror
What Will Happen When Killer Robots Get Hijacked? (marketwatch.com)
"Imagine an artificial-intelligence-driven military drone capable of autonomously patrolling the perimeter of a country or region and deciding who lives and who dies, without a human operator. Now do the same with tanks, helicopters and biped/quadruped robots." A United Nations conference recently decided not to ban these weapons systems outright, but to revisit the topic in November.
So a MarketWatch columnist looked at how these weapons systems could go bad -- and argues the risks are greater than simply fooling the AI into malfunctioning. What about hijacking...? In warfare, AI units can function autonomously, but in the end they need a way to communicate with one another and to transfer data to a command center. This makes them vulnerable to hacking and hijacking. What would happen if one of these drones or robots was hijacked by an opposite faction and started firing on civilians? A hacker would laugh. Why? Because he wouldn't hijack just one. He would design a self-propagating virus that would spread throughout the AI network and infect all units in the vicinity, as well as those communicating with them. In a split second, an entire squad of lethal autonomous weapons systems would be under enemy control... Every machine can be overridden, tricked, hijacked and manipulated with an efficiency that's unheard of in the realm of human-operated traditional weaponry.
However, the U.S. government remains oblivious. DARPA (Defense Advanced Research Projects Agency) has already announced a $2 billion development campaign for the next wave of technologically advanced AI (dubbed "AI Next"). One of the goals is to have the machines "acquire human-like communication and reasoning capabilities, with the ability to recognize new situations and environments and adapt to them." I may be overreaching here, but the UN meeting on one end and this announcement on the other, make me think that the U.S. government isn't just pro-robotic -- it may already have a lethal autonomous weapons ace up its sleeve.
The article ends with a question: What do you think about killer robots replacing human combatants?
And what would happen if killer robots got hijacked?
So a MarketWatch columnist looked at how these weapons systems could go bad -- and argues the risks are greater than simply fooling the AI into malfunctioning. What about hijacking...? In warfare, AI units can function autonomously, but in the end they need a way to communicate with one another and to transfer data to a command center. This makes them vulnerable to hacking and hijacking. What would happen if one of these drones or robots was hijacked by an opposite faction and started firing on civilians? A hacker would laugh. Why? Because he wouldn't hijack just one. He would design a self-propagating virus that would spread throughout the AI network and infect all units in the vicinity, as well as those communicating with them. In a split second, an entire squad of lethal autonomous weapons systems would be under enemy control... Every machine can be overridden, tricked, hijacked and manipulated with an efficiency that's unheard of in the realm of human-operated traditional weaponry.
However, the U.S. government remains oblivious. DARPA (Defense Advanced Research Projects Agency) has already announced a $2 billion development campaign for the next wave of technologically advanced AI (dubbed "AI Next"). One of the goals is to have the machines "acquire human-like communication and reasoning capabilities, with the ability to recognize new situations and environments and adapt to them." I may be overreaching here, but the UN meeting on one end and this announcement on the other, make me think that the U.S. government isn't just pro-robotic -- it may already have a lethal autonomous weapons ace up its sleeve.
The article ends with a question: What do you think about killer robots replacing human combatants?
And what would happen if killer robots got hijacked?
A hacker who identifies with Thanos' agenda could start 'solving' the overpopulation problem by themselves.
You don't need to steal weapons from the military to get killer robots. You just need to botnet cars. They already kill people without help.
Let me driver-assist you to safety.
the question we should be asking ourselves is what will happen when the ruling class doesn't need the working class to keep the military class in check? Right now we've got a bit of a balance going on. The Army protects the ruling class but the working class keeps an eye on the army and going into a decent civilian life after some time in the army gives them something to do besides run a Junta. Because of that there's a floor on how bad the ruling class can treat the army and the working class.
All that goes out the window when they've got a robot army. The robot army will never betray them. Sure, there's some engineers keeping it running, but they're nerds and typically lack the drive and charisma to overthrow the ruling class. Those kinds of coup are pulled off by charismatic generalissimos. So you're gonna have the ruling class, a small, well paid merchant class to keep the killer robots going and everybody else. That's you and my, btw. And the ruling class won't need us to buy their crap to be rich either. They'll own everything and have factories to build it. I suppose there'll be a few positions for their doctors and sex slaves. The rest of us get abandoned, sorta like how we ignore starving people in Africa.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
There is no new risk here. Virtually every modern weapon system is robotic and many are mostly autonomous with heavy usage of data channels for control. If hackers could take them over, we'd have long ago suffered these imagined catastrophes.
And what would happen if killer robots got hijacked?
When killer humans (i.e. a military) gets 'hijacked', it's called a coup.
It's unlikely the autonomous weapons would be hacked to fire on civilians. It'd be MUCH more effective to turn them into robotic sleeper agents that target military officials/visiting politicians (thanks to facial recognition). OTOH, firing on civilians would ensure a) civilians demand their removal, ensuring no further hacking can take place; b) their hacking is discovered immediately, ensuring the vulnerability is quickly patched, and c) an opportunity to take out more-important targets is squandered.
Corruption is convincing someone that the selfless ideal is the same as their selfish ideal.
Replacing human combatants with robots is a bad policy decision in general. The reason is simple: one of the main reasons restraining governments from waging war is the cost in terms of their own citizens killed in the fighting, removing humans from the fighting removes that restraint. We'll have enough problems with governments that already don't care about their own citizens, we don't need to add every other government to that list until we figure out another way to discourage them from starting wars any time they don't get their way.
On top of that, the possibility of robots being subverted by attackers isn't in any way overstated nor are the reactions to the possibility over-reacting. Look at our computer networks today and try to convince me that we can somehow make botnets and malware vanish overnight, and then picture a world where "distributed denial of service attack" translates to "security guards shooting any human who enters the shopping mall".
Every machine can be overridden, tricked, hijacked and manipulated with an efficiency that's unheard of in the realm of human-operated traditional weaponry.
It isn't really all that difficult to buy, bribe, threaten or convince a human.
The difference is scale. Humans are polymorphic, so they are not exact copies of each other and the identical exploit will work on one, but not others. So you need to customize your exploit for each of them, which makes mass hacks difficult. That is the reason social engineering works, but is rarely used large-scale.
Assorted stuff I do sometimes: Lemuria.org
Security is so far from a priority for most companies that it's not even an afterthought. Talk to administrators and you'll hear people say, "we're just not a big enough target for hackers." They will keep that attitude even when the company is a big enough target. No one cares if their system gets hacked, even credit agencies. They just put a vague fix in and move on. No questions about whether their problem is more systemic, and how to avoid those problems in the future.
"First they came for the slanderers and i said nothing."
IIRC the Iranians forced one down. The Russians have managed to jam everything, including GPS, over Syria several times. But in terms of actually hacking the drone, so you can take control of it and use it against it's official owner?
Nope.
Like most ideas about revolutionizing warfare, this is something that's very old. Many many cultures would have champions fight to the death so the army did not have to.
For that to work couple things have to be true:
1) Both sides have to have a roughly equal chance of success. That is the champions have to have comparable combat power. Back then, nobody's going to agree to wager their war on a naked guy with a pointy stick vs. a Knight in full plate. With robots, many countries won't have a robot capable of going toe-to-toe against us.
2) Both sides have to want whatever they're fighting enough to raise an army to fight for it, but not so much that they'll actually die for it. If they're gonna die for it regardless of the outcome of the fight, the fight is pointless. If the result of losing a robo-war is that the President gets executed, his top guys imprisoned, and everyone else gets fired, your worst enemies get to run the country, etc. then when the champion loses you fight anyway.
3) Both asides have to trust that the other side will abide by the deal. You're not spending $1 Billion on robo-champion if you think those bastards will fight even if their robot loses.
Just go through recent US-involved wars in your head. how many are all three things true of both sides?
The Nazis and Japanese could have made some pretty cool robots, but we'd have fought the actual war regardless of who won the robo-battle. The Koreans, Vietnamese, Iraqis, Taliban wouldn't have robots anywhere near our league, so they'll lose the war and then fight anyway.
What if a SUV which has got 500 horse powers and weighs three tons gets hijacked? One needs to study for years to pilot an UAV, but everybody can drive.
In my opinion there should be a legal limit to personal car power and weight; say 100 hp and 1500 kg.
And it is not "maybe", it is happening. About one and a half million(!) people are being killed on roads each year by cars globally. Times more badly injured. These are figures of a WW3. These accidents are related to drinking, suicides, mental illness, terrorism, drugs, etc.
And practically nothing is done about it. Even more powerful and massive cars hit the market. So, please, stop blaming drones. Let us first learn how to handle the real problem at hands.
They will kill something. It'll just be a different thing than if they hadn't got hijacked.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
There is one sinple general rule I've learned about life in general over 70+ years. It reads, "If it can be done, it will be done." It's one of Joanne's laws, I guess. I have many I've derived over the years. This one is the most generally applicable. If there is a market of people willing to pay for something and any hole of any size through which to slip product, the market will be fulfilled. Drugs are the most obvious such. Guns are another obvious market filled because there was a way to fill it, a way to do it. Scientific curiosity is another way things that can be done get done. "What happens if I ....?" "Oh, neat!" or "Oh well, I can buy a new lab."
AI weapons can be done. They will be done. That is a simple application of Joanne's Do-it law.
So my first question is, "If you make it illegal, what magic will you apply to prevent it from happening anyway?"
That leads directly to my second question. "Are your principles so strong you are willing to NOT be on the side that does it first and most effectively?"
(And as an aside, a Computer Organized Operation that takes over a government is a Coo-Coupe.)
{^_^}