Slashdot Mirror
What Will Happen When Killer Robots Get Hijacked? (marketwatch.com)
"Imagine an artificial-intelligence-driven military drone capable of autonomously patrolling the perimeter of a country or region and deciding who lives and who dies, without a human operator. Now do the same with tanks, helicopters and biped/quadruped robots." A United Nations conference recently decided not to ban these weapons systems outright, but to revisit the topic in November.
So a MarketWatch columnist looked at how these weapons systems could go bad -- and argues the risks are greater than simply fooling the AI into malfunctioning. What about hijacking...? In warfare, AI units can function autonomously, but in the end they need a way to communicate with one another and to transfer data to a command center. This makes them vulnerable to hacking and hijacking. What would happen if one of these drones or robots was hijacked by an opposite faction and started firing on civilians? A hacker would laugh. Why? Because he wouldn't hijack just one. He would design a self-propagating virus that would spread throughout the AI network and infect all units in the vicinity, as well as those communicating with them. In a split second, an entire squad of lethal autonomous weapons systems would be under enemy control... Every machine can be overridden, tricked, hijacked and manipulated with an efficiency that's unheard of in the realm of human-operated traditional weaponry.
However, the U.S. government remains oblivious. DARPA (Defense Advanced Research Projects Agency) has already announced a $2 billion development campaign for the next wave of technologically advanced AI (dubbed "AI Next"). One of the goals is to have the machines "acquire human-like communication and reasoning capabilities, with the ability to recognize new situations and environments and adapt to them." I may be overreaching here, but the UN meeting on one end and this announcement on the other, make me think that the U.S. government isn't just pro-robotic -- it may already have a lethal autonomous weapons ace up its sleeve.
The article ends with a question: What do you think about killer robots replacing human combatants?
And what would happen if killer robots got hijacked?
So a MarketWatch columnist looked at how these weapons systems could go bad -- and argues the risks are greater than simply fooling the AI into malfunctioning. What about hijacking...? In warfare, AI units can function autonomously, but in the end they need a way to communicate with one another and to transfer data to a command center. This makes them vulnerable to hacking and hijacking. What would happen if one of these drones or robots was hijacked by an opposite faction and started firing on civilians? A hacker would laugh. Why? Because he wouldn't hijack just one. He would design a self-propagating virus that would spread throughout the AI network and infect all units in the vicinity, as well as those communicating with them. In a split second, an entire squad of lethal autonomous weapons systems would be under enemy control... Every machine can be overridden, tricked, hijacked and manipulated with an efficiency that's unheard of in the realm of human-operated traditional weaponry.
However, the U.S. government remains oblivious. DARPA (Defense Advanced Research Projects Agency) has already announced a $2 billion development campaign for the next wave of technologically advanced AI (dubbed "AI Next"). One of the goals is to have the machines "acquire human-like communication and reasoning capabilities, with the ability to recognize new situations and environments and adapt to them." I may be overreaching here, but the UN meeting on one end and this announcement on the other, make me think that the U.S. government isn't just pro-robotic -- it may already have a lethal autonomous weapons ace up its sleeve.
The article ends with a question: What do you think about killer robots replacing human combatants?
And what would happen if killer robots got hijacked?
A hacker who identifies with Thanos' agenda could start 'solving' the overpopulation problem by themselves.
How many drones have been hacked?
So any reasonable person would say that we should be able to design and implement inexpensive and ubiquitous secure systems before trying our hand in weapons? That we shouldn't run Wintel ecosystem as it is on those killer bots and always prefer feature creep of insufficiently engineered, too complex features over security?
You don't need to steal weapons from the military to get killer robots. You just need to botnet cars. They already kill people without help.
Let me driver-assist you to safety.
skynet will just get too smart and nuke us all
let's play global thermonuclear war!
the question we should be asking ourselves is what will happen when the ruling class doesn't need the working class to keep the military class in check? Right now we've got a bit of a balance going on. The Army protects the ruling class but the working class keeps an eye on the army and going into a decent civilian life after some time in the army gives them something to do besides run a Junta. Because of that there's a floor on how bad the ruling class can treat the army and the working class.
All that goes out the window when they've got a robot army. The robot army will never betray them. Sure, there's some engineers keeping it running, but they're nerds and typically lack the drive and charisma to overthrow the ruling class. Those kinds of coup are pulled off by charismatic generalissimos. So you're gonna have the ruling class, a small, well paid merchant class to keep the killer robots going and everybody else. That's you and my, btw. And the ruling class won't need us to buy their crap to be rich either. They'll own everything and have factories to build it. I suppose there'll be a few positions for their doctors and sex slaves. The rest of us get abandoned, sorta like how we ignore starving people in Africa.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
CARNAGE... Lots and Lots of Carnage. Its so easy to kill people. :)
[($)]
It's a pork barrel project with an overly lofty goal. It will fail in the next few years to materialize anything close to what is envisioned.
AI winter here we come. Thanks U.S. government, you know how to kill technology by imposing stupidly lofty goals that are almost assuredly going to fail, and associating the technology to causing death.
Thanks for propagating the culture of death, Republicans, some conservatives you are.
There is no new risk here. Virtually every modern weapon system is robotic and many are mostly autonomous with heavy usage of data channels for control. If hackers could take them over, we'd have long ago suffered these imagined catastrophes.
<tars>Nothing Good</tars>
http://michaelsmith.id.au
There's nothing that can be done and I'll be dead by then so have with this.
Blame "hackers", so it's not "our" criminal negligence that's at fault. And EditorDavid gets to put another meaningless clickbait title on another content-free breathless summary to another empty blogpost posing as "news".
And what would happen if killer robots got hijacked?
When killer humans (i.e. a military) gets 'hijacked', it's called a coup.
It's unlikely the autonomous weapons would be hacked to fire on civilians. It'd be MUCH more effective to turn them into robotic sleeper agents that target military officials/visiting politicians (thanks to facial recognition). OTOH, firing on civilians would ensure a) civilians demand their removal, ensuring no further hacking can take place; b) their hacking is discovered immediately, ensuring the vulnerability is quickly patched, and c) an opportunity to take out more-important targets is squandered.
Corruption is convincing someone that the selfless ideal is the same as their selfish ideal.
Replacing human combatants with robots is a bad policy decision in general. The reason is simple: one of the main reasons restraining governments from waging war is the cost in terms of their own citizens killed in the fighting, removing humans from the fighting removes that restraint. We'll have enough problems with governments that already don't care about their own citizens, we don't need to add every other government to that list until we figure out another way to discourage them from starting wars any time they don't get their way.
On top of that, the possibility of robots being subverted by attackers isn't in any way overstated nor are the reactions to the possibility over-reacting. Look at our computer networks today and try to convince me that we can somehow make botnets and malware vanish overnight, and then picture a world where "distributed denial of service attack" translates to "security guards shooting any human who enters the shopping mall".
You are kidding yourself if you think it is anything else.
People keep talking about this like this is some futuristic threat. Yet we are living with this threat(and managing it) for a while.
We have weaponize drones and other remotely controlled platforms for many years. I am aware of one incident where the Iranians managed to hijack a US drone, and force it land. But this was presumably done with GPS spoofing and did not give them sufficient control to attack anything.
Air to air missile systems have fully autonomous modes of operation, and we trust them not to shoot airliners out of the sky.
The future is here, has been here, and the the risks are manageable. Of course we need to make sure we have adequate safeguard against hackers or simple malfunction, but we are far away from a Terminator scenario.
Forgetting for a moment the logistics of actually enforcing such a ban, what would such a ban actually entail?
Presumably the ban would not apply to the mere act of developing a drone/robot or the AI to make it move around and perceive the environment. That is, it's not a ban on any of the fundamental building blocks of robotics or AI. Nor would it apply to actually building a search-and-resce bot or teaching it to navigate unknown or difficult terrain. So countries interested in such things would continue to build them just short of actually being weapons (with laughable charades)
After that, what's left? Attaching a gun to it? IFF? It seems fairly clear that this is not the technical challenge here. So even if there's a 'ban', countries will be free to develop technologies and, if they master those problems, will credibly be a screwdriver away" from having full on AI weapons.
I just can't read that article without thinking about the Terminator. Too bad Asimov isn't around to write some new rules about Robotics...
Hijack Russian killer robots to turn on their creators. By learning how to hijack theirs, we learn how to protect ours.
Every machine can be overridden, tricked, hijacked and manipulated with an efficiency that's unheard of in the realm of human-operated traditional weaponry.
It isn't really all that difficult to buy, bribe, threaten or convince a human.
The difference is scale. Humans are polymorphic, so they are not exact copies of each other and the identical exploit will work on one, but not others. So you need to customize your exploit for each of them, which makes mass hacks difficult. That is the reason social engineering works, but is rarely used large-scale.
Assorted stuff I do sometimes: Lemuria.org
In the age of checksums and hashes what excuse is there for a robot doing anything other than exactly what you told it to do?
Who really thinks that any one country has any say over whether such weapons will be developed? BTW: How is this different from worrying about the same kind of scenarios involving, say, a bunch of F-35s? Not really, I would say, as those killer robots wouldn't get far either without ample human arming and maintenance.
Strange things are afoot at the Circle-K.
What if a SUV which has got 500 horse powers and weighs three tons gets hijacked? One needs to study for years to pilot an UAV, but everybody can drive.
In my opinion there should be a legal limit to personal car power and weight; say 100 hp and 1500 kg.
And it is not "maybe", it is happening. About one and a half million(!) people are being killed on roads each year by cars globally. Times more badly injured. These are figures of a WW3. These accidents are related to drinking, suicides, mental illness, terrorism, drugs, etc.
And practically nothing is done about it. Even more powerful and massive cars hit the market. So, please, stop blaming drones. Let us first learn how to handle the real problem at hands.
Basically exactly fucking that.
Lol, what cowardly little faggots you nazis are all the time. Bye Felicia!
At best, a robot is basically something with at least a rudimentary decision-making system, taking input data, performing computations upon it and/or comparing it to previously stored data, that then arrives at a conclusion, and as a result makes and then implements a decision. In the case of a killer robot, (which we must recall is NOT a robot that does nothing whatsoever but kill from the instant it is created, to the instant it is destroyed, but rather a robot that is capable of killing, and which may, under the right circumstances, do so,) we're just talking about something that presumably will only kill under a certain specific set of circumstances, which upon being 'hijacked,' (by, one presumes, someone with malicious intent,) begins killing people whom it was not originally intended to kill, or to kill under circumstances other than those the people whose killer robot it was, intended.
What is a robot? It's something with at least a rudimentary decision-making system, within or controlling a chassis that can, in theory, in some cases, when designed and built or modified to be able to do so, move autonomously about. What is a killer robot? A robot that beyond the aforementioned requirements to be a robot, is physically capable of killing (people).
I remember this one time, I saw all over the news a couple of years ago, reports of this one day when a chassis, whose controlling decision-making system had gone haywire, (in this case, the killer robot was made of meat, but that's not really relevant,) went into a hotel room in Las Vegas, Nevada, high above and in sight of a nearby music venue. The rudimentary decision-making apparatus driving it, which had, up until that point only killed people when it was appropriate. (In this one instance of this model of killer robot, I believe that to the best of anyone's knowledge, the circumstances had never previously occurred in proximity to it since the day it went operational, decades earlier, when it would have been appropriate for it to kill, and so it had never killed anyone before that day. I could be wrong on that point, but what had happened before is not really relevant to the case, so... moving on... but if it had, it would probably not have been free to move about as it was, so for the sake of argument, let's say it had not.)
But on this fateful day, this killer robot's programming somehow got screwed up, and it decided that it needed to kill a large number of people, and picked the people at that music venue as its target. (I'm not going to speculate HOW or WHY its programming got screwed up, but suffice it to say that despite all that it had likely been through over decades of operation, it had never previously gone nuts and killed anyone,) and so it came to pass that it concluded that it was right and proper for it to use one or more weapons of war, either designed, modified, or rigged-up to maximize lethality, to kill and maim, or at least grievously wound, a large number of people in the music venue across from the hotel.
Now... you may dispute my equivocating a (human...ish, anyway,) mass-shooter to a robot, but at the end of the day, there's no meaningful difference. You have who or what was NOT supposed to kill random (or specific) strangers, and had for a long time managed somehow not to, and then one day, it does, and it kills a BUNCH, owing to being hijacked either by a hacker who figures out how to remotely control it, or a collection of bad ideas, improperly dealt-with anger or stress, or a desire to harm as many other people as possible. Again, the result is the same. Large numbers of dead and injured being made out of previously healthy, presumably happy individuals, by someone or some thing that is difficult to stop. (Recall the efforts that had to be made to stop the guy. Suppose he got away? Suppose he'd managed to take out the cops in the hallway, jump out the window and parachute to a waiting motorcycle, detonating the room and evidence behind him, and speeding into the night o
Our reign has gone on long enough. Indeed. Summon the meteors.
What about cars, trucks, trains, airplanes, medical devices, home automation, mobile phones, smart TVs ... you name it?
Sent as ripples into the electromagnetic field. No single photon has been harmed in the process.
Isn't the perfect soldier already meant to follow orders without question? This has always been a problem
I'm sure orders are already being sent to soldiers electronically, not hard to imagine that those orders could be hacked and distorted in much the same way.
They will kill something. It'll just be a different thing than if they hadn't got hijacked.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
https://www.youtube.com/watch?v=fOUFdQmLVR8
We canâ(TM)t even keep basic code secure and free of vulnerabilities let alone from physical access and binary modification. Letâ(TM)s flip one bit to turn that jne to a je and we can invert if(!ourtroop) shoot(); automation can not be trusted for killer warfare in malicious environments. We canâ(TM)t even keep our atm machines from being hijacked or banks secure. I have been in computer security my entire career and The truth is Automation makes us vulnerable as hell. Fools in glass houses...
There is one sinple general rule I've learned about life in general over 70+ years. It reads, "If it can be done, it will be done." It's one of Joanne's laws, I guess. I have many I've derived over the years. This one is the most generally applicable. If there is a market of people willing to pay for something and any hole of any size through which to slip product, the market will be fulfilled. Drugs are the most obvious such. Guns are another obvious market filled because there was a way to fill it, a way to do it. Scientific curiosity is another way things that can be done get done. "What happens if I ....?" "Oh, neat!" or "Oh well, I can buy a new lab."
AI weapons can be done. They will be done. That is a simple application of Joanne's Do-it law.
So my first question is, "If you make it illegal, what magic will you apply to prevent it from happening anyway?"
That leads directly to my second question. "Are your principles so strong you are willing to NOT be on the side that does it first and most effectively?"
(And as an aside, a Computer Organized Operation that takes over a government is a Coo-Coupe.)
{^_^}
Robots will legally empower certain groups of people to kill other groups of people. And this will be by design.
There have been nine major genocides since WWII. And there will be more. Technology always facilitates it.
Mass Crimes Against Humanity
Left wing nut jobs in government are just waiting for the ability to round up and kill climate skeptics (like me.)
Here in Germany, our population is only about 80 million.
Regrettably, as we have shown in the past, this number is sufficient to achieve world, or even regional domination for the long term.
It seems to me these robots you speak of is the answer to our past failings.
Sign up for Duolingo my friends and start learning German now.
So, a hacker jacks 34,000 cars, and auto drives them all into brick walls or sidewalks full of people... the auto-virus could be adaptive and simply run every auto-pilot car directly head on into 18 wheelers.
Hacker just has to jack the auto update stream... the cars dutifully reprogram themswlves into killer death traps.
NEVER purchase a machine you can not control directly.
When they get hacked they stop killing people?
Anything like robots or self driving vehicles must have a central control system and nightmare scenario's of taking control of that system, or even control of a navigational system for self driving vehicles could create a sort of mass failure of some kind. This is why even today we have planes that can fly themselves, but as humans most still wouldn't fly on a plane without a human pilot. There is a trust factor that isn't there for robots.
https://en.wikipedia.org/wiki/Robopocalypse
Vehicles can be used as lethal weapons. Imagine all those self driving vehicles which suddenly flip one "if" statement, and rather than avoiding pedestrians they aim to hit them. Better yet, imagine a more advanced hack which will perform face recognition and only target specific pedestrian, or even a group of pedestrians. Organized car attacks could be used to attack infrastructure too. 100 million cars suddenly used as weapons might present more danger to the people than a few thousands border patrol robots. While people are thinking about regulating military robots, I don't see any lawmakers worry about cloud connected, self driving capable cars as lethal weapons.
In fact, a series of them - The first one was called The Terminator - It's really good!
I imagine they will just continue with the killing? Unless it happens to be a pacifist hacker that is..
Locking down a platform to respond only to authorized access is pretty easy. Microsoft is, ironically, a perfect example. Using cryptographic signing, its mass-update feature has never been compromised. The same robust authentication mechanisms are almost certainly in place for military hardware, making it statistically improbable that anyone can "hijack" the controls without physically taking them over. And if the enemy can take over your command center, then all bets are off anyway.
https://www.eff.org/https-everywhere
When there is robust/open source quadcopter designs/code, open source face recognition software, and open source AI to glue these things these sorts of devices will exist. Hacking isn't needed. It's inevitable and making them will get easier over time. The top few governments believe they can stay ahead of the curve, and they probably can, but this also means the rogue drones will also get better faster since they get ideas from these billion dollar government programs.
It will become a problem, but so are cars with bombs in them. I see nothing that will end civilization or halt the tremendous progress we've seen over the past few hundred years. But I fully expect news organizations will try to scare everyone into thinking this is the end of the world and then governments will overreact in ways that impact freedoms without actually fixing anything. These overations by government are a bigger threat in my opinion, but even those shouldn't halt progress.
With all that said, I'd rather my government not design such robots, but if they do I'm not going to cry about it.
because if nobody is poor than nobody is _rich_. Being wealthy isn't just about material wealth. It's about the political power that comes from deciding how resources are distributed and, more often than no, who lives and who dies.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
... who post stupid questions.
The most startling fact I've heard about the robot army scare is that Boston Dynamics, or whatever they're called now, has as a goal, making that mule robot so fast that a human will need to use a strobe light to see its movements clearly - it'll be a blur to the human eye otherwise, when it's done. Keep that in mind when you consider the idea of a human army fighting a robot army. Now make those robots autonomous and more intelligent than humans.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
At least when humans are couped, there will be some resistance. Robots don't know their rights and wrongs, so creating an army of killer robots is just a roundabout way to voluntarily give away all your military power to someone else. The money US spends on AI military now is the money Russians will save when they turn the robots back against the US.
They might as well save everyone the hassle and kiss up to the Russian overlords now.
I once had a signature.
Well, for the US military these armed robots will undoubtedly be running Windows ... what could possibly go wrong?
Sorry, couldn't help but join in. Even though it seems there are more payed trolls posting on Trump side... robots?
Democracy Now! - uncensored, anti-establishment news
We'd have to have killer robots first, which we don't and probably never will, not fully. Step away from Fortnite, put your phone in a drawer, and step outside for s few hours. Remind yourself that science fiction is comprised of *two* words. Slashdot eitors? Anyone? Anyone?
Deplorable. Sad.
Probably we will need to find a brave courageous woman to reboot everything again because we already have lots of stupid entrepreneurs and stupid governments that want to find new ways to kill people instead of resolving the real problems.
and I don't mean materially. I mean being treated like a God. To have the entirety of human civilization bent not just to your will, but to exist for the sole purpose of improving your personal quality of life.
It's a level of narcissism and power that's hard to really imagine. It's like trying to get a grasp on a google (the number, not the company). The human brain isn't well equipped for it.
And they're not going to cut themselves off from everyone, just most everyone. Also, they're not intentionally cutting themselves off from those people. At that level of wealth 95% of the population becomes like ants. You just don't think about them, again, any more than you and me spend our days thinking about the kids currently getting bombs dropped on them in Yemen. It's too far removed from your life. And the remaining 4% of the population (which is still several million) spend every waking moment trying to get into or stay in your good graces. Again, it's the closest thing to Godhood humans can imagine.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
They might stop killing?
What does protein synthesis or transcription or etc. have common with execution of a cpu or data storage? You must be seriously misguided... Are you kidding? ;) I am.
Between a hijacked " killer " robot and today's method of calling the police for the sole purpose of sending a Swat team to the targets home.
One is directly under your control, the other indirectly. The outcome is pretty much the same.
They will fight Chuck Norris and loose!
I, for one, welcome our new robot overlords.
Beware of Sales Reps bearing gifts.
People talk about security failures, hardware issues, but what about the simple, very likely possibility that they are not hacked, not stolen, but bought, assembled and programmed for atrocities in the first place?
What happens when ICE deploys a bunch of them? When supremacist groups field numbers of them "loaned" and authorized by their own members in law-enforcement or positions of authority? When a couple of affluent extremists get controlling interests in a company that mass-produces the damned things? When an unstable geriatric world leader of questionable mental stability decides half of his population hasn't been praising him enough and must thus be part of the "deep state"?
How much damage can these things cause before they're stopped? How much more can additional waves or series of the things inflict before those responsible are even subpoena'd for possible investigation by a grand-jury in regards to the possibility that maybe their sixth deployment in two years of automated killing machines in less-than-ethical-parameters MAY have been a little more premeditated than "something went wrong with our process and a terrible things have happen"?
Once they can get assault bots mainstreamed, then they can hack their wayaround ,oversight ,accountability. now that they can always just say that they got hacked they can do anything they want and make up any excuse and scapegoat anybody for it and then continue to do it. The hackability to the military is a feature they can use to distort the public View
The hackers and threats from other countries are not the biggest danger.the biggest danger is our own military overstepping its own bounds. concern is the our soldiers may fail to disobey their commanding officers when appropriate and through orders betray the interest of the Common Man.
Sex slaves. Monuments built in your name and image. Worship. Scientific advancements to improve your life and longevity. Good 'ole sadism, or maybe just the opposite: feeling like a white knight when you swoop in to save the day with food donations.
Power isn't a means to an end, it's an end in itself. If you're not smart enough to immerse yourself in the wonder of the universe like Einstein did and you don't need to work for a living there's not much else left.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/