Slashdot Mirror
Debating a Ban On Autonomous Weapons (thebulletin.org)
Lasrick writes: A pretty informative debate on banning autonomous weapons has just closed at the Bulletin of the Atomic Scientists. The debate looks at an open letter, published In July, 2015, in which researchers in artificial intelligence and robotics (and endorsed by high-profile individuals such as Stephen Hawking) called for 'a ban on offensive autonomous weapons beyond meaningful human control.' The letter echoes arguments made since 2013 by the Campaign to Stop Killer Robots, which views autonomous weapons as 'a fundamental challenge to the protection of civilians and to international human rights and humanitarian law.'
But support for a ban is not unanimous. Some researchers argue that autonomous weapons would commit fewer battlefield atrocities than human beings—and that their development might even be considered morally imperative. The authors in this debate focus on these questions: Would deployed autonomous weapons promote or detract from civilian safety; and is an outright ban the proper response to development of autonomous weapons?
But support for a ban is not unanimous. Some researchers argue that autonomous weapons would commit fewer battlefield atrocities than human beings—and that their development might even be considered morally imperative. The authors in this debate focus on these questions: Would deployed autonomous weapons promote or detract from civilian safety; and is an outright ban the proper response to development of autonomous weapons?
Covered this topic exceptionally well, including at what point an autonomous weapon could be trusted...
And the answer is when it becomes human.
GrpA
Enjoy science fiction? "Turing Evolved" - AI, Mecha, Androids and rail-gun battles. What more could you want?
War is all about winning at virtually all cost. In today's world that will increasingly mean autonomous weaponry. Aircraft in particular have passed the ability of a human pilot to control. The biggest limiting factor in warplane development is the pilot.
If a gun can shoot itself: you don't need to strap a meat puppet to it.
If there's no meat puppets aiming guns at you: there's no need to aim guns at meat puppets.
Autonomous weapons are "world peace" via gunboat diplomacy. Nobody is willing to die to kill a robot so when they run out of robots to die on their behalf: they will naturally surrender.
War will become a competition to see who can build better robots at destroying other robots and eventually we do away with war entirely as computer simulations become a more cost effective means of settling disputes. Explosive slave collars would be a prerequisite to avoid the loser refusing to "pay up" but in the end that's a small price to pay for peace!
When autonomous weapons are outlawed, only outlaws will have unstoppable armies of soulless killing machines.
Don't waste your vote! Vote for whoever you want, unless you live in a swing state it won't matter anyways
-OR-
/. forgot about the nasty downside of an autonomous doomsday devie.
how
But support for a ban is not unanimous. Some researchers argue that autonomous weapons would commit fewer battlefield atrocities than human beings—and that their development might even be considered morally imperative.
In particular, Dr. Miles Dyson and his associates, Drs. Skyler Natalya and Keel Lbot, Ph.D.
Has few if any 'atomic' scientists on it nowadays.. It does have a lot of left wing political pundits.
You can't put it back. In fact, it just gets bigger every year.
If every government in the world banned them, private individuals have the means to make them anyway with cheap materials nowadays.
Personally, I'd rather see robots kill each other as opposed to humans slaying each other. Not only does it make sense for the militaries involved (robots can't desert, they will never be afraid of gunfire, and you don't need to tell their family that they died in the war), but the civilians would prefer it too (you don't have to risk your life being shot, you won't have to abandon your home, etc.). Autonomous machines are an advantage for everyone involved, and would be a much more humane way to solve wars.
However, from a practical perspective, that may not be possible. The elephant in the room is that most wars are caused by resource or territorial disputes, and those are often a result of overpopulation. Autonomous wars don't kill people, and hence nothing is done about the core problem. Furthermore, traditional wars are oftentimes waged for... maybe a few decades? At most? I know of several exceptions, but I don't think we see contiguous conflicts last longer than that very often, because war is such a draining activity to be engaged in. Everyone dying tends to end wars because of public exhaustion, but with machines exploding instead, that never happens, and so we may potentially have to worry about wars dragging on for far longer than they used to.
"Set a man a fire, he'll be warm for the rest of the night. Set a man afire, he'll be warm for the rest of his life."
What's the difference between a search-and-rescue bot and a kill bot? The function is going to pretty much identical right up to the point the target is located, just duct tape a gun to point in same direction as the camera and wire the "person located" signal to pull the trigger. It's one thing to ban ABC weapons because they're very specific technologies, but this is way too generic to work. And it's not like the military is going to avoid developing it for intelligence gathering and decision support systems, even if you keep a human in the loop it's literally going to be one flip of the switch to full automatic where the computer's recommendations are implemented by itself.
The primary reason to keep soldiers in the loop today is because you're trying to fight a "good war" and avoid antagonizing the civilians so you want manual confirmation of each target, if you take the gloves off and say if you're found outside after curfew we'll shoot to kill and live with the collateral you could automate much more. And don't get up on the high horse, when the US nuked Hiroshima and Nagasaki they knew there's be about 100-200k civilian casualties. In a real war nobody's going to give a fuck if the robots are just 99% or 95% right, if it can save our troops and civilians and end the war for sure we're going to let them fight for us.
Live today, because you never know what tomorrow brings
Arguably while war is all about winning, it's not at "all cost". We (ignoring GOP presidential candidates) will not bomb an urban civilian center these days in order to kill a few hostile bad actors. Arguably the availability of precision munitions changed the moral balance from "bomb a city into submission" into the modern sensibility of "only bomb specific targets of military importance."
Even though war is a terrible and bloody affair, we as societies have constantly been moving towards more humane and less deadly conflict. It's one thing to shoot someone shooting at you. It's quite another to kill someone in cold blood. War is arguably largely about self defense today: "I have to shoot you in order to not die." The shift I see happening with autonomous weapons is that there is no imperative to shoot people shooting at you. It might be expensive or costly to lose an autonomous infantryman, but if you can capture without killing I suspect we'll expect our autonomous soldiers to exert "self control". Obviously if it's a shooter killing civilians you would be morally justified in stopping them using violent force if necessary but otherwise autonomous troops effectively become more akin to police officers than soldiers in their relationship to the population.
No, it's just that if an autonomous weapon does it, it would be more difficult to call it an "atrocity". If a dozen villagers are killed because of a minefield that some idiot decided should go near where they live, the only reason you can't call that a "massacre" is that there was no human making the targeting decision.
In the 1920s, there were some who argued that aerial bombing would be more humane because they could be far more precise than field artillery, hitting only the target that you want to hit. Look how well that worked out.
sub f{($f)=@_;print"$f(q{$f});";}f(q{sub f{($f)=@_;print"$f(q{$f});";}f});
Landmines kill little kids without asking. Do we want more things killing automatically?
My ism, it's full of beliefs.
Many restricted military technologies are fairly easy to detect. Nuclear weapons require a massive industrial input that has well known signatures, for example.
A robot is different. It can be something that's dual use. One day it's a regular robot. Look at the software. All civilian and a nice strong optics mount point on it.
Change the software and swap some of the camera gear for a small machine gun and use the rest for aiming, it's a killbot. This is just one example. Another obvious one is putting an autonomous drone software package into the flight computers of an airplane that can also be manned. This game can go on and on with just about any weapons system you can think of.
It doesn't take industrial facilities that are different from usual ones to make them. If you can make versatile robots for civilian use, and separately make weapons you just have to put them together at the last minute. They don't have any particular signatures the way chemical weapons and their precursors do. Most nations are already making ordinance, so who's to say whether a human is going to be in the loop to fire it or if it's triggered by an AI?
If people want to cheat on this, it'll be pretty easy to do so.
So far, the landmine bans haven't seemed to have slowed down the planting of them a bit in various wars. We have to have demining teams, not just for cleaning up old wars, but the very ones that are going on now.
I don't expect this to have a much greater effect.
and you end up with no possibility for revolutions. Tyranny will be rampage. Rich will afford such weapon, poor will not. As simple as that.
That's something of a question, isn't it? What does "meaningful human control" translate to? Does it mean that a human has to okay each weapon discharge? Does it mean that we aren't supposed to release a swarm of von neumann kill-bots with 'destroy everything' as a goal? What if we release them in an area with orders to kill any humans with weapons that don't have a valid IFF signal?
In addition, I've seen with UN weapon ban treaties that they're sometimes used as a 'we don't have them, so you shouldn't either' tool. Who's closest to these sorts of weapons? The USA. Who's NOT going to agree with any treaty limiting the effective use of these weapons? The USA. Rendering the ban useless.
I don't read AC A human right
Prohibitions against everything from the MAOB to the rifle have been tried at one time or another. The three that have stuck are chemical, biological, and (mostly) nuclear. Why just these three out of all the way we have of killing each-other? Why is white phosphorous still used but Sarin isn't? It's not because one is more horrible. It's because one is prohibitively expensive and dangerous to "safely" develop, use and transport. Why moab and not nuke? because one is prohibitively expensive and dangerous to develop, use and transport. And why landmines but not smallpox? It's not the number of civilian casualties.
There will come a time when fully automated weapons systems are less expensive to deploy, and safer for one's own side, than a soldier. We already have some fully automated weapons systems out there, for example those guarding the Korean DMZ. But when that day comes no prohibition will prevent widespread deployment of fully autonomous weapons.
If video games influenced behavior the Pac Man generation would be eating pills and running away from their problems.
Autonomous weapons don't kill people. People with autonomou ... wait, wrong thread!
The original Robocop movie correctly predicted how these things might turn out. Infallible is not the adjective to apply.
Still, at least it shot a company exec during a demo, so not a total failure.
"Some researchers argue that autonomous weapons would commit fewer battlefield atrocities than human beings"
,e.g. the generals handling the command, would allow them. Therefore it would still be HUMAN being declaring what's the ROE. 5% civilian casualty allowed. 20%. 100%. The number would be set by human. At least in the case of human we can have other human balking at atrocities and rebelling against order or getting taken to task after the war. With machine it would be a "I am not liable the machine misinterpreted etc...." bullshit shitfest. And with no easy way to demonstrate the falseness of it. And if it is machine there is FAR LESS incentive to say no to a war, when there is nobody from your country which will see the consequence. Body bags on TV are a very strong politic brake to wars, when they are "yours". Heap of scrap don't. So it is much easier to decide to go for a war. Which is why by the way we see so much bombing by the US using drone : they know there is nobody protesting much because no US body bags. So they bomb more and more. There is no incentive to slow that down. THIS is the real danger of autonomous or semi autonomous weapons : the lowering of the barrier of moral usage to almost nothing.
They would commit only as much atrocities as their master giving the order
C. Sagan : A demon haunted world:
http://www.amazon.com/gp/product/0345409469/
visit randi.org
But support for a ban is not unanimous. Some researchers argue that autonomous weapons would commit fewer battlefield atrocities than human beings—and that their development might even be considered morally imperative.
I hope the people suggesting this have taken into account the responsibility factor. *When* an autonomous weapons platform commits a war crime, who gets sent to Hague? The CEO of the manufacturer? The brogrammers who made the code?
-SR
Drones can be armed, so yes, it totally is related to autonomous weapons.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
I think we all know not to build Terminators.
What a shame that in this relatively late stage of human development we should still be debating the moral pluses and minuses of applying technology to kill each other. Talking about "offensive autonomous weapons beyond meaningful human control" and "autonomous weapons would commit fewer battlefield atrocities than human beings" - shouldn't we be raising the level of discourse? A lot?
'The Economy' is a giant Ponzi scheme whose most pitiable suckers are the youngest among us and the yet-unborn.
This 'autonomous weapons systems' debate is under attack, a hostile takeover by radical factions of the Artificial Intelligence research community. When I first glimpsed Hawking's phrase offensive autonomous weapons beyond meaningful human control I immediately thought, he's talking about land mines, right? --- but no... it appears they had some hideously complicated Futurist Thing in mind. Okay... perhaps we're just talking about AI because it is fun to talk about it and it takes our minds away from other unpleasant things.
What are we really afraid of? Let's shelve AI for just a moment and make a list.
Things unable or unwilling to identify and spare civilians.
Things that could 'turn' on their masters.
Things that don't know or care that the war ended years ago.
Things too dumb to realize that they were made to do evil.
Things with lithium batteries which are harmful to the environment unless disposed of properly.
A short list! Since LAND MINES meet these criteria (except the last, they are better for the environment than cell phones) without the tiniest glimmer of artificial intelligence --- I would suggest that this fixation on AI is hyper-specific and a little obsessive.
It's like any other piece of technology we might make. If it is designed, engineered and made well by humans who make a reasonable effort to at least consider my short list and take every step to mitigate these items, no matter how smart it is, we'll be as well off as we could possibly be, given that people are making such things.
If we must produce artificial intelligences for war, they should be made by the same companies that make those automotive jumper cables you see for sale at 24 hour convenience stores. The human race would have nothing to worry about.
It also appears the 'autonomous weapons systems' debate has been invaded by people who hate war itself. Who let them in? I would expect the Bulletin of the Atomic Scientists to advise them sternly, this is not the place to justify or condemn the existence of war itself, this tribunal has been convened to discuss things with processors and neural nets and instruction sets and big fat dual-use research contracts and stuff. We cannot allow you to disrupt these proceedings. (No fighting in the war room!) But sadly --- the Bulletin is permitting people to express their distaste for war. Those who love war find this rude and insensitive. Don't they have feelings,too?
THE BULLETIN: "But no, you're the one who's out of order, bringing up land mines as if they are in any way relevant to this 'autonomous weapons systems' discussion. Land mines are cheap, mass-produced, inevitable, deployed, funded, signed sealed and delivered. There is no way to prevent their use because bad people use them all the time. With land mines we have no choice. But we still have time to choose not to make artificial intelligences. For war. We have a choice."
HOCUSLOCUS: "Okay... so, what if you get your wish and all you AI Play Nice Boy Scouts sign a treaty or something... and we are not afraid any more... so a self-ware AI robot is IS produced, not for war, but to travel the world (autonomously!) to clean toilet bowls. Because you can be sure that in the future we'll be playing among the stars and harnessing neutrinos but no one will ever be able to male a toilet bowl that doesn't crust up with shit. In order to really get toilets clean you need the kind of weaponry the military is only dreaming about today. So this AI has some serious big honking space gun WOTAN stuff. What do you think will happen on the day this machine figures out where all that shit is coming from?"
THE BULLETIN: "We concede you have a point. But at present no one is concerned about the specific scenario of toilet cleaning robots. We may some day convene a symposium to discuss them, specifically, but not land mines. Is that understood?"
In Blade Runner
<blink>down the rabbit hole</blink>
I will give up my killbot when you pry the controller from my cold, dead hands! Because the Constitution!
The US does not have a population large enough to fight a traditional war against nations like China who can arm many millions of soldiers. We would be in the position of having to go to nuclear, germ or gas warfare in a land war with China and some other nations as well. The very nature of that kind of war would mean the death of many millions of noncombatants. It would also mean the slaughter of vast numbers of American troops and a devastation of our economy. But autonomous and drone types of weapons can take out specific targets with far less slaughter of innocents. They can also be quite intelligent and do things like only respond to people carrying a weapon. They can also give us ways of avoiding the limits of foreign powers. For example, there are nations that will not allow any armed ship to dock in their ports. Drones could transfer to a vessel standing near- by and thus allow our ships to dock and still have vast protection, such that any enemy attack would result in a high- speed attack with immediate consequences, for any enemy attacks upon our ships in port.
"Some researchers argue that autonomous weapons would commit fewer battlefield atrocities than human beings—and that their development might even be considered morally imperative."
Fuck your moral imperative, if you're going to remove humans from the chain then remove them entirely, no more human participation in war.
All they are really talking about here is a weapon that will never say no, like humans sometimes do.
"If any question why we died, Tell them because our fathers lied."
But robots could be turned on and ordered to do atrocities that no human would agree to do.
If a rogue general wanted to convince US soldiers to kill everyone in the White House, or to kill everyone in Congress, he would find it very difficult. It would take almost nothing for a rogue General to convince a squadron of drones to do the exact same thing.
excitingthingstodo.blogspot.com
does not approve of this message!
As mentioned elsewhere in this story and thread:
Landmines are autonomous, durable, and persistent. Very dangerous. And easily forgotten by their employers.
Claymore mines ditto, though less appreciated for their durability, and found so much more quickly.
Technology has produced much more interesting autonomous weapons, mobile, with greater range, but if you're all worked up because they are being controlled by operators half a world way that high-five each other when they obliterate a wedding party, well, you've missed the old white men and women in a room full of screens showing blobs moving and some going cold...
And before that, old white men in subway tunnels moving little wooden icons around a map, listening to telephone reports, and lighting up a cigar to celebrate that they have, once more, survived the night.
War is more and more fought elsewhere. This may be offensive to your delicate sensibilities, but be thankful you are not shooting from your bathroom window. Or not. Just don't ask me to pretend you have any moral high ground. We must change things or accept the world as it becomes.
deleting the extra space after periods so i can stay relevant, yeah.
In The Day the Earth Stood Still, Gort is a member of an autonomous robot police force built by aliens, programmed to preserve peace in the universe by destroying any aggressor. Thus forcing fickle and emotional sentient beings to behave.
...unless you're also going to halt *all* AI development because any automated weapon without an AI can be controlled by any sufficiently advanced AI.
For instance, all the UAVs could be controlled by an AI, thereby taking a non-AI weapon and making it an autonomous weapon. As long as you have remotely controlled weaponry and AI development - however disconnected they may be - you have the potential for an autonomous weapon that could be outside of human control.
That said, a claymore is a very simple autonomous weapon - albeit one that can be easily disabled, but it's autonomous nonetheless.
Truth is like the sun. You can shut it out for a time, but it ain't goin' away. - Elvis Presley (source: imdb.com)
Great idea, lets start with land mines, ban their production in the US (US-based companies build and sell more land mines than any other country, most without auto-deactivation or other features that let them stop being a problem after the war ends), sales, and shipment through US controlled territory. Oh, wait, that would impact profits, nevermind.
Dear Umbrella Corp Customer,
We rolled out an automatic firmware update last night that seems to have a few bugs. If your AutoGunner 3000 killed your entire village, please follow these steps to restore it to factory mode:
1) Using a paper clip, press the small reset button (located on the front of your AutoGunner 3000's turret, directly below the .308 barrel).
2) Continue depressing the button for approximately 5 seconds. If the turret begins to move, release and start the procedure over.
3) If you survived^H^H^H^H^H^Hhear a slight beep, your AutoGunner 3000 should begin a warm boot process. Any remaining shells or cartridges in the hopper or magazine will be emptied. DO NOT STAND IN FRONT OF YOUR AutoGunner 3000 DURING WARM BOOT.
4) Re-apply the firmware update (available on our website).
Looked at from a certain perspective, the overwhelmingly bad threats we face going forward are all variations on a single theme- huge power over life and death is devolving into the hands of smaller and smaller groups of individuals. The size of the group needed to pull off an attack is shrinking, while the number of people harmed and the severity of that harm are both simultaneously going through the roof.
This characterizes the nature of the threat of terrorism - small arms fire is bad, automatic rifles are much worse and small groups of people flying planes (a high technology) into skyscrapers (another high technology) is worse yet. So this is yet another example of that phenomena, albeit a more obvious example.
We're going to bestow obscene weapons of ultimate destruction upon ourselves then, we're going to use them, just as the critics are saying. At some point it's got to happen that we wake up and realize that only path to survival for us is to attack the root cause of the problem- human thought and behavior- specifically terrorism comes from just two sources.
One is the obvious, the infliction of real economic and power injustices by a tiny ruling minority upon the majority. The minority essentially treat the lives and fortunes of the majority as an asset to be strip mined for their personal benefit. Call it what you want, dehumanization, oppression, exploitation, enslavement, rape it's all the same thing, the same strain of thought and impulse: over there is that other person- let me find a way to separate him from anything I may want and then turn the very act of his living, his existence, into an event which benefits only me.
This breeds huge criminal and extremist political movements all over the globe- in the Middle East, in Russia which is effectively a kleptocracy, and even in the US which is effectively a plutocracy.
The reason for a lot of the "crime" is because "crime" and the "black market" is the only rational way for people to put bread on their table. So also with extremist political movements. People know when they're treated unfairly, when their lives have been stripped of meaning and hope. Not everyone thinks the best choice is to set himself on fire, and if that widespread general malaise gets harnessed by a unifying narrative which promises to solve all those problems, then you have something like ISIS or al Queda.
Besides economic and social injustice, the other source of terrorism is sociobiologically driven impulses which for some reason are immune to satiety or attenuation of any form. They just keep driving people to *more* of the behavior without limit. More money, more power, more land, more resources, more control over other people and what their lives will be like. From a sociobiological perspective, all those drives serve one purpose- to advance the "cause" of your genes, to increase your personal reproductive fitness at the expense of your alleged competitors for the same reproductive-enhancing resources and opportunities.
And in fact, control over and access to all that stuff DID, over the course of evolutionary time, prove to be the deciding factors in who lived long enough to spread their genes and who was genetically shut out. As a consequence, we are all to a greater or lesser extent possessors of those impulses. But for some people, or for all people in some environments, those drives just never experience satiety or attenuation by other life events. They brutally and callously determine their bearers' behaviors and dominate their world view.
So bin Laden, who had many wives, and many children and tons of money STILL felt insulted, dismissed and humiliated when his offer to defend Saudi Arabia from Saddam Hussein was rebuffed by the Saudi royal family. So Saddam Hussein who had effective control over the land wealth and political power of a sovereign state only used it to aggrandize himself and seek yet more power. So John Rockefeller, Andrew Carnegie, and Bill Gates - who are known now by many for their charitable trus
I guess it depends on how you define autonomous weapons.
Back in the 1980s there were several weapons systems that would be programmed to lock in on a certain target characteristic (sound signature, EM signature, heat signature) and autonomously maneuver to strike their programmed target. Later tech could be programmed to automatically acquire a target based on signature parameters. Digi-cam actually makes it more difficult for certain tech to automatically acquire a shape and heat signature for a human body.
Many would think of the hunter-seeker drones when they hear "autonomous weapons" but there are many more variants that could be included in that term.
What needs a ban is attaching weapons to surveillance cameras. If man can network it, man can hack it. To my knowledge; only a few localized insane installations have weaponized their security surveillance system. But, wow, if we had those in schools we could prevent school shootings! Or, you could have school shootings with the whack job not even having to enter the building.
This curmudgeonly old fart thinks of the movie "Runaway" every time I read a reference to the "Internet of Things". The death by weaponized, networked, vacuum cleaner in that movie was a sphincter clenching meme when you start thinking of automating to a "smart house" level of tech.
http://www.imdb.com/title/tt0088024/?ref_=nv_sr_6
NRRPT/RCT
I've developed a DRONE that can TRACK each and every BRAHMIN in the World;
If drones have Facial recognition software they'll become ultimate killing machines;
Casteism