Slashdot Mirror
Human Rights Watch: Petition Against Robots On the Battle Field
New submitter KublaCant writes "'At this very moment, researchers around the world – including in the United States – are working to develop fully autonomous war machines: killer robots. This is not science fiction. It is a real and powerful threat to humanity.' These are the first words of a Human Rights Watch Petition to President Obama to keep robots from the battlefield. The argument is that robots possess neither common sense, 'real' reason, any sense of mercy nor — most important — the option to not obey illegal commands. With the fast-spreading use of drones et al., we are allegedly a long way off from Asimov's famous Three Laws of Robotics being implanted in autonomous fighting machines, or into any ( semi- ) autonomous robot.
A 'Stop the Killer Robots' campaign will also be launched in April at the British House of Commons and includes many of the groups that successfully campaigned to have international action taken against cluster bombs and landmines. They hope to get a similar global treaty against autonomous weapons. The Guardian has more about this, including quotes from well-known robotics researcher Noel Sharkey from Sheffield University."
http://en.wikipedia.org/wiki/Berserker_(Saberhagen)
Fred Saberhagen's "Beserker" series.
Aside from touching on the subject at hand, it's just some crackin' good sci-fi. :)
I don't know if we'd ever reach that point ourselves, but in that series, an unknown (and now extinct) alien race, losing a war and desperate, created "doomsday" machines that were simply programmed to kill all life. They were self-replicating, self-aware AIs that took their task seriously, too.
Then again, I ask myself what some jihadist might do, if given half the chance ... . .. ..
Cogito, igitur comedam pizza.
In robot drone murders and you morons think he will sign something? Obama, Nobel Peace Prize winner that has killed the most innocent women and children yet!
As far as I can tell, we need to focus on dealing with the presence of drones and "killer robots," not how to prevent them. Like it or not, 'progress marches on'.
There is no satisfactory definition, just as there is no definition for "Artificial Intelligence". Things change and so does our acceptance of what is commonplace vs. what is considered novel.
This message is sponsored by Sarah and John Connor. With special consideration from Morpheus, Trinity, and Neo.
sudo make me a sandwich
Hey, James Cameron, are you the submitter??
The automomous Terminator-style robots the summary refers to are far from becoming a battlefield standard, much to the disappointment of the /. crowd and sci-fi nerds.
Predator drones et al., like all current robotic devices in the battlefield, still have a human being in charge making all the decisions, so the points raised are completely moot.
I thought that all current systems still have a human-in-the-loop who makes the use-of-force decision? Or has this changed in recent years? I know last I heard, it was still being discussed in military ethics circles. Have we moved on from this?
The drones America uses are piloted by humans. The other robot in use by the military is the one that disables bombs. It also is remote controlled by a human. I don't think the military has any non piloted robots deployed in combat. Even a turret would be too dangerous. An automated turret could kill our own troops. Closest thing we have is landmines.
if we could just get the robots to only fight other robots...
loyalty above all, save honor
in the 1890's Tesla staged naval battles in Madison Square Garden where remote-controlled boats did battle against each other. His goal was to have robots fighting in wars as our proxies, so men wouldn't have to die. But eventually, it will be man vs machine, Terminator-style.
Remember kids, if you're not paying for the service, YOU ARE THE PRODUCT THAT IS BEING SOLD.
His machines weren't "robots" any more than Predator drones are: they were remote controlled by radio.
If you think about a virus for a second, it's the same thing. You can't reason with a virus. It doesn't make moral decisions. It just does what its DNA programs it to do, and it's even more dangerous because it's self-replicating. We need to deal with autonomous robots the same way we deal with bio-warfare.
"I have never let my schooling interfere with my education." - Mark Twain
Depending on how one defines "robot", this will be extremely unlikely.
I don't mean to be the dark figure in this conversation, but I think it's inevitable that robots will be used on the battlefield, just like people are going to continue to use cluster bombs, land mines, dum-dum bullets and other horrible devices. The reason is that they're effective.
War is a measurement of who is most effective at holding territory. It is often fought between uneven sides, for example the Iraqi army in their 40-year-old tanks going out against the American Apaches who promptly slaughtered them. Sometimes, there are seeming upsets but often there's an uneven balance behind the scenes there as well.
Robots are going to make it to the battlefield because they are effective not as killing machines, but as defensive machines. They're an improvement over land mines, actually. The reason for this is that you can programmatically define "defense" where offense is going to require more complexity.
Already South Korean is deploying robotic machine gun-equipped sentries on its border. Why put a human out there to die from sniper fire when you can have armored robots watching the whole border?
Eventually, robots may make it to offensive roles. I think this is more dubious because avoiding friendly fire is difficult, and using transponders just gives the enemy homing beacons. In the meantime, they'll make it to the battlefield, no matter how many teary people sign petitions and throw flowers at them.
What kind of war? How about we replace the godamned shitheads in all the godamned government agencies with a functional, law-biding computer program? If we did this in a way that everyone agreed with (haha) then maybe we could have social sanity again. Think about all the computers, not wanting money, not being able to be bribed, not being effected by personal, or religious ideals... Yeah, these war-robots sound good, so long as the war is in the proper place...
Given that technology has a tendency to end in its "simplest" state due to the forces of the competitive market, I never quite understood how it was realistic that the Laws of Robotics were ever actually expected to be implemented in the first place. By the time robots were capable of processing them in any meaningful sense, they'd already be bypassed by suppliers willing to skip them for a cheaper product that doesn't worry about such things.
Legislation might be useful to force manufacturers to implement such for civilian use once robots were capable of "understanding" these laws, but was there ever a time that we assumed that military and law enforcement 'bots would be required to obey them also?
It's funny, between robots and FTL drives, teleporter technology, replicators, etc... I always thought the Three Laws (or later amended) were the most unrealistic.
These days it's everywhere. Whether predicting good or bad to come, they all act like it's imminent. Calm down you guys, both you folks who think the Terminator is nigh, and you guys on the edge of your seat waiting for techno-rapture; neither are coming in your lifetime, or mine, or for generations for that matter
Get on with your lives.
Also, these petitions are a joke now.
I fail to see the difference between a robot killing off a group of people and a religious extremist strapping explosives to his four year old son before he tells him to "go say 'hi' to those nice soldiers over there."
How many times must it be said? Asimov's 3 "laws" have nothing to do with real robotics, future or present. They were a _plot device_, designed to make his (fictional) stories more interesting. Even mentioning them at all in this context implies ignorance of actual robotics in reality. In reality, robot 'brains' are computers, programmed with software. Worry more about bugs in that software, and lack of oversight on the people controlling them.
Killer robots can't be a government only option =D
This led to clever people developing submachine guns.
Give it a couple decades and you'll be able to download plans for your own battlebot and then create it on your printer
This article is absolute garbage. Almost everything in that Guardian article is misinformed and sensationalist.
"fully autonomous war machines"? Care to give an example? I've follow this stuff pretty closely in the news on top of researching AI myself. And from what I have seen no one is working on this. Hell, we've only just started to crack autonomous vehicles. They site X-37 space plane for gods' sake. Everything about that is classified so how do they know it is autonomous?
My favourite gem has to be this one: "No one on your side might get killed, but what effect will you be having on the other side, not just in lives but in attitudes and anger?". Pretty sure that keeping your side alive while attacking your opponent has been the point of every weapon that has ever been developed.
How about we replace all soldiers on all battlefields with these robots? That way there's no issue with robots killing people.
robots killing robots
wars settled in a clash of machinery without any humans for miles around
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
There are all indications that the coming robotic revolution will usher in a new era of human peace and prosperity. Robots have no emotion, no bias. Imagine deploying a few hundred (or thousand) semi-autonomous robotic peacekeepers into a conflict zone. They maintain the peace 24/7, they never tire, they are alert and objective in their duties. War is traditionally an incredibly wasteful and expensive exercise. Look at Iraq and Afghanistan! $1 trillion and thousands of allied casualties. Deploy a robot army and watch the costs come down. No need for living quarters, no need of food or water, logistics becomes cheaper in every aspect.
Like them or loath them, Drones are incredibly efficient in what they do. They are very lethal, but they are precise. How many innocents died in the decades of embargo on Iraq and the subsequent large scale bombings under Bush? Estimates run into over 100,000. Use of drones in Libya, Mali, Yemen, Pakistan have reduced costs by hundreds of millions and prevented thousands of needless casualties. Drones are the future and the US has an edge that will not give up.
Robots are not alive.
There is no true sacrifice of blood and souls when robots take the place of soldiers in battle. In my opinion, that brings them up to WMD in terms of being able to inflict loads of casualties with little risk to the aggressor.
Only humans should be on the battlefield killing each other, robots killing robots is just so inhumane.
I haven't thought of anything clever to put here, but then again most of you haven't either.
I agree! No robots in the battlefield! I also propose that all battles from now on be fought by avatars in a game. Humans should be prohibited to engage in real battles, where they act as merciless robots, "just following orders", even "illegal" ones. A new Geneva convention should be summoned immediately to enforce these new rules of engagement!
can america please do something about mental health in the US? Americans are seeing Obama as a killer, clearly you need mental help because George Bush signed all those laws...
Without "autonomous war machines" we've managed to firebomb cities (with a nice 3 hour gap between bombing runs so that fire fighters and so on would be putting out the first run's fires when the second run hit), mass murder civilians, drop atomic bombs on cities, use chemical weapons, and everything in between. I don't think feelings of mercy and pity and an ability to not follow illegal orders makes much of a difference.
My main problem with using robots (or, more likely, remotely piloted-semi-autonomous war machines) on the battlefield is that it makes war too easy. Right now, drones aside, war is a costly matter. You need to put actual lives at risk and that acts as a check on what generals/politicians would want to use troops for. Want to invade North Korea and Iran to stop them from being a threat once and for all? Well, that's going to wind up costing tons of lives which is going to make it harder to sell to the public. (Lives on the other side count too, but - let's face it - they don't count as much because it is all too easy to dehumanize the enemy.)
However, let's assume millions of soldiers were seated at a "video game console" while their robotic avatars were out in the battlefield killing North Koreans or Iranians. Suddenly, the only cost is money spent on damaged avatars. Spending billions on war can eventually cause public sentiment to shift against the war, but not the same way as the prospect of thousands of body bags coming home does. I fear that, once war becomes a "real life video game", we'll become a much more war-mongering nation due to the reduced "cost" of war.
My sci-fi novel, Ghost Thief, is now available from Amazon.com.
I didn't think, after watching "Terminator", that this would start happening until about 2029 or later.
are coming
There have been many situations where you've had humans in on the ground, one gets killed, and the slain soldier's buddies snap and decide to massacre an entire village. I'm not really sure what part of merciful warfare autonomous robots are threatening.
War is terrible. The people starting any war are stupid. I don't care if it is the USA (my country), some other country or just organizations like al-Qaeda. Don't start something you cannot finish.
Missiles are robots.
Bombs are robots.
Any stand-off weapon is basically a robot.
Drones have a human "somewhere" making the decision. Sometimes that decision is wrong, just like when real people are on a battle field and shot at the wrong people.
Sometimes the equipment malfunctions.
War is terrible and should be avoided. Only idiots think that breaking things really solves issues. Often, it only delays the issue.
For deep beliefs where there is no way to change the belief, yet the other side wants to kill their opponents (us?), then the only answer seems to be to kill everyone over age 7 and reteach all their children. Killing just the people on the front has proven to be ineffective. More will be grown to be "warriors" and continue the fight. This seems to be a way of life in the middle east. They've been complaining about the same issue for thousands of years. Get over it already. War is stupid. If you don't agree to let the other-side live, there is no way to resolve the issue.
Argentina has been complaining about the Falklands for hundreds of years. Get over it already. War is stupid. If you don't agree to let the other-side live, there is no way to resolve the issue.
War is terrible, but doing a half-assed job is just as terrible. It leaves an entire culture wasting time thinking about retribution instead of becoming productive, wealthy members of a world-wide society.
Kill them all. End THAT war forever. Don't let the children of the dead live on to believe that they own some vengence to their ancestors. Kill them all. That would be kinder to the current people and for the generations to come.
Whatever happened to neutron bombs?
War is terrible whether it is close up or remote.
His machines weren't "robots" any more than Predator drones are: they were remote controlled by radio.
Yes. That is probably why he stated that they were remote-controlled.
http://en.wikipedia.org/wiki/Samson_RCWS
These turrets count I think. Israel has at times said they are keeping a man in the loop, but the technology doesn't require it, and at times they have said they are in
"see-shoot" mode. This is essentially indiscriminate area denial that is easier to turn off than mines. It does have the computer vision and targeting aspects of a killer robot, just not the path finding and obstacle avoidance parts.
refactor the law, its bloated, confusing and unmaintainable.
wanna kill all humans?
Here today, gone tomorrow
People really get very preferrential about their mode of death. Robots seem to offend them more than the more civilized death of a soldier. Me? Hell, I'd prefer to die in the glory of combating a metal beast. That would be far more glorious than fighting a mere mortal.
We should learn what we need to know about issues, before we decide what we need to feel about them.
The 3 laws of robotics were a brilliant development as a plot device. But I don't think there's even a theoretical way to implement them on actual technology that wouldn't be trivial to circumvent. There is not a single organization that makes "robot brains" that can build in these laws. There isn't a way to build this into a processor, and even if there were, you could modify the data sent to the processor. Also, there would always be a manufacturer willing to leave them out once the price got high enough, my guess is that, for most manufacturers, that price would be around the cost of a current processor plus a dollar.
The author gave the following reasons against autonomous war robots:
|| Robots possess neither common sense... ||
Me: This is true, but isn't that the point? Someone behind the curtain has common sense? For example, the current generation of drones in use aren't intelligent, but the people flying them are making the decisions (or rather, their superiors). We need to separate the ED-209 vs. drone conversation as I'm pro drone, anti-ED-209 style military robot.
|| 'real' reason ||
Me: See above. Our current drones do have "real reasoning" aka, there is a real human operating them.
|| any sense of mercy ||
Me: Actually, "mercy" can cause problems. Offering mercy, only to have that terrorist return (with additional knowledge). Also, wouldn't an operator show more mercy with a drone? AKA, if they "kill" the drone, nobody dies. I see this as a Win/Win for the use of drones.
|| nor — most important — the option to not obey illegal commands. ||
Actually, this is where you lost me. Our current generation of drones are scrutinized in a number of ways. Everything is recorded, the operators are in direct communication with their superiors and their peers (other operators), and the fog of war is very different. I could argue that the "illegal commands" could actually be reduced. What's more likely, a person under the stress of survival making an illegal/horrible decision or the person sitting in an air conditioned building operating a robot 7000+ miles away making an illegal/horrible decision, when being recorded?
All that said, there is something very real about us having real humans on the ground in many places. That said, if we could minimize human casualties by implementing limited drones, why wouldn't we do that?
If the US and Great Britain refuse to produce these machines, somebody else will. And good luck trying to get every freaking country in the world to sign your "global agreement" (Iran and North Korea, anyone?). If we outlaw killer robots, only the outlaws will have killer robots.
snip...
We are afraid of body bags coming home and we're afraid of collateral damage.
snip...
Hmm
I don't believe the military is afraid of collateral damage maybe reporting of it and gaining negative publicity, but certainly not of inflicting it. One reason that collateral damage figures in Iraq (inflicted by the alliance), were not reported, indeed they lied at the time and said the figures were not even recorded.
...over humans.
Here's hoping we eventually get to the point where both sides just deploy robots, and whichever has robots standing at the end wins.
Lets stop wasting young lives.
What do I know, I'm just an idiot, right?
People keep going on about the robocaplypse and the "friendly AI problem" but the real problem has, for millenia, been the "friendly NI problem" or friendly natural intelligence problem. Whether the drones being manipulated by the natural intelligences are made of silicon, metal and composites powered by electricity, or they are flesh and blood constructs powered by chemistry is beside the point.
Seastead this.
NPR had a piece that same kinds of arguments have been made against every new escalation of military technology back to metal swords and the gun itself. That technology increases the soldiers killing capacity and makes him/her more removed from man-to-man combat.
The real jump would be machine-decided (A.I.) killing. For the most part there is a man in the decision loop. Even with the new Israeli "Iron Dome" missiles where operator has seconds to decide to launch. (More of a financial decision because they cost $75K apiece.)
I have put a lot of thought into combat robots, particularly airborne ones. I think they're really an inevitable development.
I don't have a problem with robots maneuvering themselves over a battlefield. I don't have a problem with a robot killing someone. I don't even have a problem with giving it a target, and letting it decide the best way to eliminate it.
The only provision I would require is that we not have it select its own targets. There should be a human operator somewhere telling it what it should be shooting at. Or, for some scenarios, a "shoot everything that moves that isn't broadcasting an IFF signal" button, but that would be useful mainly for aircraft or fixed defenses, and should only be used for actual large-scale warfare, not counter-insurgency stuff. I'm talking "there are 200 MiGs over DC, if it isn't USAF, kill it". And *that's* only necessary because I think air warfare is going to become a numbers game, with hundreds or thousands of cheap (~$10,000) drones forming a "swarm", so a pure "drone pilot/human gunner" solution just won't work. Land robots are intrinsically more complex - both the human-like designs and the tank-like designs (the only two worth a damn IMO) are by their nature expensive, so the numbers will be small enough for "drone pilot/human gunner" to work.
Not a good idea
This led to clever people developing submachine guns.
Give it a couple decades and you'll be able to download plans for your own battlebot and then create it on your printer
Not true. See http://en.wikipedia.org/wiki/Combat_shotgun
"It can't be bargained with. It can't be reasoned with. It doesn't feel pity, or remorse, or fear. And it absolutely will not stop, ever, until you are dead."
Sounds like the perfect fucking soldier to me.
You don't fight wars except for one reason - to win. To win, you have to kill the enemy. To kill the enemy, you need efficient soldiers who are committed to the mission.
Robots are also absent of malice which unfortunately is not the case on the human battlefield.
... have already contributed 'Skynet'. http://en.wikipedia.org/wiki/Skynet_(satellite) refers...
The first robots we build in large quantities are designed specifically to violate all three laws.
Believe it or not, philosophically-minded roboticists and robotically-minded philosophers are on this.
May I suggest, for example, this book from MIT Press?
The drones are already in the air. Do we really need robots when it's just easier to kill from the air?
And if the drones are too pin-point, there are still enough nukes out there to take out huge portions of population if that is the need.
It's all going to be death by air, drone, ICBM or bunker buster.
Bryan
Stop these robots from fingering my wife!
I am very small, utmostly microscopic.
Robots are expendable. This can be bad because it takes away the risks that often prevent military action. This can also be a good thing because a robot doesn't need to defend itself. It wont fire at an angry civilian mob. A robot under fire can wait for confirmation from it's operator before firing back. People don't have this luxury, they need to take quick action to defend themselves, often without enough information. Everything the robot sees can be recorded, which makes the operator accountable for the commands they send.
To Welcome Our Robotic Overlords!
Surely we can't deprive future battlefields of these wondrous autonomous machines! Oh, the humanity! No, wait ...
http://en.wikipedia.org/wiki/Bolo_(tank)
http://www.whkeith.com/graphics/bolo-mark-xx.jpg
but what if you're the bad guy? strength doesn't make you right.
Just another second banana
For the US at least, there are currently no programs researching autonomous weapons systems. There is no chance in hell that the Pentagon is iwlling to do that. A human will ALWAYS be in the loop. someone please tell these human right watch buffoons to go sip some chait and be quiet.
I wonder how many college kids will feel like this once they learn that their autonomous vehicle designs used in the DARPA competitions have been used in this manner.
http://www.youtube.com/watch?v=26YLehuMydo
At least we can be thankful that the world is now free of cluster bombs and landmines.
Dunno about 'fully autonomous' as opposed to remote control, but I think that over in Iraq and Afghanistan, if a robot replaces and therefore saves the life one American soldier, I say, "Bring on the bots!" Same for cluster bombs and land mines.
It's why the U.S has never signed the convention on land mines. And shouldn't.
Do I want to see that in use in America? Not at all. But I know what nation I am a citizen of, and whose troops I want to have the overwhelming advantage on any battlefield. Carbon or silicon. As a veteran, the only thing I'm surprised at is the lack of them to date - a remote control equivalent to a T-100 could probably do the sentry work of a whole platoon in the hands of one operator and two armed ground troops for precision rifle work.
Just put all wars on the moon, make sure all sides have robots (and the Pentagon will he happy to sell them to you if you don't have access to your own.) Then let them have at it. No more dead soldiers, no more collateral damage, looser picks up brewskies after the fight. A new era of world peace ensues.
Let's tie the politicians voting for this onto the front line of tanks during our first strike. Sure by this point the enemies will mostly have been dealt with by airstrikes, covert operations, or propoganda, but this way at least they'll get to ride into battle gallantly leading their men.
Just don't put them INSIDE the tanks. We don't want them stinking it up for those poor tanker crews :)
So they want to ban robots from wars, so that only real people die in the battlefield? What's the "game theory" behind a robot-only war? Whatever it is, it has to be better than sending 18 year old kids to fight.
--
Stay tuned for some shock and awe coming right up after this messages!
The very idea that robotic weapons murder people randomly belongs to the world of nut jobs and lunatics. In war conventional bombs wipe out hundreds of thousands in an air raid. Everything gets blown to bits including schools, hospitals, senior centers, moms, factory workers and the entire area of conflict. Any high school kid should be well aware of Berlin or Tokyo or Hiroshima as examples. A smart weapon that takes out one car or one room in a hotel is hardly an instrument of terror. It is a life saver for innocents in a war zone. I know that protest is in fashion but idiotic protests serve no one. Compare taking out one bunker with a smart bomb (which is a robot) to smacking the area with a nuclear bomb. So far robotic devices are the greatest idea in the peace movement. I look forward to the day when we use these armed robots for internal crime control as well as military uses. Imagine a burglar in the night trying to break into a car and an intelligent, small, flying robot sticks a fork in his ear. Given enough technology crime may become a thing of the past.
Decades? Arm a landmine on top of a roomba. Instant killer robot.
You make some great points I agree with about cruise missiles and indoctrination ("brainwashing") already being around for a long time in warfare. I agree we need to think more deeply about this, and your proposal is a start in that direction. One issue with your suggestion is that these days even invading a country like Iraq that posed the US no immediate danger was labelled "defensive". The best of ideas can just get spun around when core values are lost. That has given us "free speech zones" that are literally cages miles from any events. And it has given us "border zones" that extend 100 miles inland and cover 75% of the population where citizens rights are essentially suspendable whenever desired by law enforcement calling in the border patrol. Thus, your suggestion might never be invoked because leaders would just label any war of aggression as "defensive" -- and who is to stop them?
You might like two related links. The first about something written by Marine Major General Smedley Butler in the 1930s called "War is a Racket", where he concludes only by taking the profit-motive out of warfare can it be ended:
http://www.warisaracket.com/
The second is by me, and is the product of more than a quarter century of thinking about this issue since I spent about a year as a visitor/volunteer to two heavily-military-funded CMU robotics labs: ... There is a fundamental mismatch between 21st century reality and 20th century security thinking. Those "security" agencies are using those tools of abundance, cooperation, and sharing mainly from a mindset of scarcity, competition, and secrecy. Given the power of 21st century technology as an amplifier (including as weapons of mass destruction), a scarcity-based approach to using such technology ultimately is just making us all insecure. Such powerful technologies of abundance, designed, organized, and used from a mindset of scarcity could well ironically doom us all whether through military robots, nukes, plagues, propaganda, or whatever else... Or alternatively, as Bucky Fuller and others have suggested, we could use such technologies to build a world that is abundant and secure for all. ..."
http://www.pdfernhout.net/recognizing-irony-is-a-key-to-transcending-militarism.html
"Military robots like drones are ironic because they are created essentially to force humans to work like robots in an industrialized social order. Why not just create industrial robots to do the work instead?
I signed the petition anyway (and included a link to my essay in the "sincerely" closing line which was the only part of the letter that was editable besides my name). But I feel that only by addressing the issues Butler raises and I raise and you raise will we all move towards a real long-term solution on this for humanity (and AIs) as a whole.
A 21st century issue: the irony of technologies of abundance in the hands of those still thinking in terms of scarcity.
When they're not demanding a new Holocaust they're screaming at the people calling them on it
Well, let's make sure these Robot Protesters' families are first in line for a military draft. And make sure they talk to the widows, parentless children and amputees from the past wars and ask their opinion on it.
I wouldn't mind robots on the battlefield as long as no humans were on the battlefield. If all wars end up being fought between robots, and no humans / animals involved etc, then I'm all for robots being on the battlefield. However, the cynic in me thinks once they get on the battlefield, they'll end up in the city streets and elsewhere that humans congregate.
Sure enough, the cow costume was hanging up next to the superhero outfit and sailors uniform. (S,Spud)
This will lead to so called clever-printer devices being able to download plans for a next gen battlebot and create it on itself, denying _you_ access to the print button.
this sounds like one of those projects that americans spend billions developing it while the russians employ a kitten with a dynamite stick strapped on it?
See Forever Peace by Joe Haldeman for the natural conclusion of drone development.
See the Black Magic M-66 anime movie for the natural conclusion of purely robotic kill bot development. (That anime is one of the closest I've seen to capturing the feeling of Saberhagen's Berserker series (already mentioned in this thread). The Terminator series runs a very distant second place.)