Slashdot Mirror
Pentagon Chiefs Fear Advanced Robot Weapons Wiping Out Humanity (mirror.co.uk)
Longtime reader schwit1 writes: Huge technological leaps forward in drones, artificial intelligence and autonomous weapon systems must be addressed before humanity is driven to extinction, say chiefs of Pentagon
From a report: Air Force General Paul Selva, the Vice Chairman of the Joint Chiefs of Staff at the US Defense Department, said so-called thinking weapons could lead to: "Robotic systems to do lethal harm... a Terminator without a conscience." When asked about robotic weapons able to make their own decisions, he said: "Our job is to defeat the enemy" but "it is governed by law and by convention." He says the military insists on keeping humans in the decision-making process to "inflict violence on the enemy. [...] That ethical boundary is the one we've draw a pretty fine line on. It's one we must consider in developing these new weapons," he added. Selva said the Pentagon must reach out to artificial intelligence tech firms that are not necessarily "military-oriented" to develop new systems of command and leadership models, reports US Naval Institute News .
From a report: Air Force General Paul Selva, the Vice Chairman of the Joint Chiefs of Staff at the US Defense Department, said so-called thinking weapons could lead to: "Robotic systems to do lethal harm... a Terminator without a conscience." When asked about robotic weapons able to make their own decisions, he said: "Our job is to defeat the enemy" but "it is governed by law and by convention." He says the military insists on keeping humans in the decision-making process to "inflict violence on the enemy. [...] That ethical boundary is the one we've draw a pretty fine line on. It's one we must consider in developing these new weapons," he added. Selva said the Pentagon must reach out to artificial intelligence tech firms that are not necessarily "military-oriented" to develop new systems of command and leadership models, reports US Naval Institute News .
Surprised no one mentioned this yet. Why people think the Air Force guy is crazy is beyond me. Of course autonomous systems that kill are a threat to humanity.
HBI's Law: Frequency of calling others Nazis is directly correlated with the likelihood of the accuser being Communist.
I'm wondering if humans will ever shake off their extremely violent ancestry and wind down the war and militarism. The US is the greatest exporter of weapons and the most militarily aggressive country in the world with military action in over 100 countries.
https://www.thenation.com/arti...
If we can't lead by example in toning down endless warfare, and instead provide the cover that other countries need to justify and build their own drone and robot armies, then the world of the future is going to be a very dismal place indeed.
A brain is a terrible thing to waste... Mind? That's debatable.
Another, and I think larger, problem with the increased use of technology rather than human boots-on-the-ground is that it makes it easier, from a political standpoint, to go to war. You don't have mothers, fathers, and spouses of all those people being put in harms way making trouble because their loved ones are dying. This is one reason I'm a fan of bringing back the draft, without all the loopholes that allowed rich-white-boys (I'm looking at YOU, W!) to dodge serving. If your constituents have skin in the game, it's harder to vote on a war resolution.
War and violence are inherent to the profitability of the military industrial complex. You cannot have greatness and executive bonuses without corporate profits.
I'll see your senator, and I'll raise you two judges.
... about 7 billion people find themselves in agreement with the Pentagon chiefs...
Log in or piss off.
Until repair/refuel/replenish is automated...you don't have to worry about more than a first strike. Of course, you aren't hooking the nukes up to these, are you? (nervous silence)
His concern isn't entirely unjustified. We're increasingly relying on robots to do the actual killing, but we've currently designed the systems so that humans need to be involved in the decision making. The fact that we're involving humans puts a natural bottleneck on our operations, since there's only have so much attention we can give. At some point (e.g. World War 3), it be seen as more efficient to launch a fleet of drones that vastly outnumbers our pilots, equip each with facial recognition systems and a list of targets, and tell them to kill on sight.
I'm not saying it's a good idea, but there's no denying that it would be an efficient way to get the job of killing done, and that it's the sort of measure a country might turn to in desperate times.
But at that point, we'd be just one bug away from a system that produces false positives and starts gunning down everyone in sight. We're just talking about faulty weapons, not machines that can think or understand what they're doing. But if they're deployed en masse, a single bug could have catastrophic results, in much the same way that landmines have remained a problem in many parts of the world, decades after the wars that put them there had ended. This isn't Skynet or an AI intent on world domination. This is simply a machine with a bit too much responsibility.
Come and see the violence inherent in the system. Help! Help! I'm being repressed!
"National Security is the chief cause of national insecurity." - Celine's First Law
It's a legitimate concern in an all out arms race that someone will let AI guided weapons make their own attack decisions if only to circumvent the decision cycle of the enemies systems. It's not hard to end up with scenarios like Philip K. Dick's "Second Variety" or the movie Screamers (based on the book).
We have nothing to fear but fear itself! And Spiders!
Apple is the wealthiest company that has ever existed.
Did they use violence to get there?
Agriculture is arguable the greatest thing mankind ever invented.
It doesn't need violence.
In other words, you're a tough guy behind a gun. But your mind is can't see what actual greatness looks like.
There are two types of people in the world: Those who crave closure
He says the military insists on keeping humans in the decision-making process to "inflict violence on the enemy. [...] That ethical boundary is the one we've draw a pretty fine line on. It's one we must consider in developing these new weapons," he added.
I'm sure that's what he tells himself. As a non-techie, he might even be able to believe it. However, just about all hardware in existence has been experiencing creeping "AI" for decades. Does the pilot make every decision for the position of every aileron during flight? Of course not. There are lots of little decisions to be made in the piloting of aircraft and ordinance that are getting more and more computerized every year. At some point there will be anti-drone weaponry, and defensive weaponry on the drones, and when that day comes having to wait for an Ethernet packet to go from Kandahar to Virginia, a human to process it, and then back, is going to be seen as a mission-threatening liability. At that point they'll have the computer make the firing decisions too, but they'll justify it by saying the human's role was to start the mission in the first place.
Here's a question for you: When some other nation (eg: Russia or China) starts making these drones and deploying them over countries in ways we don't agree with, perhaps even over countries friendly to the USA, how is the USA going to feel about them then?
In the old Trek TOS there was an episode where they found a planet where large amounts of people just reported to extermination centers because the warring states' computers told them to based on their warfare simulations. As I get older, I'm finding that less and less implausible.
Yeah, people frequently quote facts like "Russia or the US has enough nukes to destroy the world 5 times over".
Even if that is true, they won't have the ability to fire all of them- they'll send off a dozen before all their launch sites would be nuked by the enemy. Even if they fired all of them- they're not going to fire them to cover the surface area of the earth, they're going to fire multiple at strategic sites.
Po Dunk, West Dakota, is not going to be a target. It makes no sense sending a $x-zillion rocket to target a town with a population of 700. Rural areas all around the world wouldn't be hit.
There would be survivors of any nuclear conflict. Maybe millions of survivors. It would suck to be a survivor, life would be really hard under a nuclear winter with all distribution networks destroyed. Humanity would survive though at a much diminished rate.
"That's the way to do it" - Punch
Well, a bug will produce collateral damages, however it will not lead to human race extinction or not even near it. An autonomous killing robot still need to refuel, replenish ammo, etc. So, it is not entirely autonomous. It still relies on external systems to keep going. Beside that, if this kind of race toward an autonomous killing machine is launched, there will be also development for counter measures against it. It is not like your ennemy will sit and rest waiting for being killed by your machine. No matter how wonderful may your killing machine be, there will be efficient counter measures against it.
Achille Talon
Hop!
If you haven't seen the counterargument, here you go.
https://app.box.com/WitthoftResume Code: https://github.com/cellocgw
But at that point, we'd be just one bug away from a system that produces false positives and starts gunning down everyone in sight. We're just talking about faulty weapons, not machines that can think or understand what they're doing. But if they're deployed en masse, a single bug could have catastrophic results, in much the same way that landmines have remained a problem in many parts of the world, decades after the wars that put them there had ended.
Star Trek: The Next Generation, Arsenal of Freedom was my first thought...
Ezekiel 23:20
His concern isn't entirely unjustified. We're increasingly relying on robots to do the actual killing, but we've currently designed the systems so that humans need to be involved in the decision making.
Forget the military drones. (Or at least, they're a smaller component of the overall issue.)
We have computer-controlled cars. They will be deployed in massive numbers over the next ten years. If remote updates are possible, anyone who can update a popular model has access to a distributed weapon of mass destruction capable of causing hundreds of thousands of deaths in a matter of moments.
Warfare-oriented tech isn't the only vector for mass attacks.
Real lawyers write in C++
I'd like to say that toning down endless warfare would be nice, but I think we have the dangerous idea that we are in a post-violence world.
This may be the future, but it isn't the Star Trek future. Being an example to others is honestly not yet at the point where it will lead to a cascading effect and end violence.
What happens when the US ceases to patrol off the Horn of Africa without the Somali's having their people prosperous and their country stable? Very simply, more piracy, because the Somalis are still poor and fighting each other.
What happens when the US leaves the Gulf while the Arab States and Iran have not sworn off violent fundamentalist ideologies? Escalation and threats to the current production of a great deal of the world's supply of energy and no moderating force, because no one else is interested in ignoring sectarian goals for peace and unity.
Yes, our mission in those areas acts as an irritant in some ways, and certainly bad decisions in that area can cause dangerous moments, but this is not yet a situation where picking up and unilaterally swearing off war or international military missions is going to have the effect you have hoped for. Places like Iran and the Arab States and India and Pakistan, and other areas need to treat each other like Canada, Europe and the US treat each other to get to the point where stepping away will result in anything but war and serious instability.
Actually obtaining democracy peace to the point where most of the West have gotten today, as imperfect as it is, took centuries of brutal warfare and bloody revolutions, and that is AFTER most of those Western countries had agreed that representative and government and the rule of international law was a good idea. The US had to pretty much fight a brutal Civil War and years of tensions with Europe to get there, and Western Europe itself didn't clean up its act until WWII. The rest of the world has a long way to go on that front and while it would be nice to say that they just have to adopt our institutions, we have already seen what happens when an uneducated population, unused to peace and democratic institutions is forced to turn into a democracy. It becomes a "Democratic Republic" which works exactly like a dictatorship or oligarchy.
There would be survivors of any nuclear conflict. Maybe millions of survivors. It would suck to be a survivor, life would be really hard under a nuclear winter with all distribution networks destroyed. Humanity would survive though at a much diminished rate.
Nice assertion, but we still might not survive. Starvation and disease could take the rest. Human-kind has been close to extinction before. We could make it there again. And maybe this time, not be quite as lucky.
That is all.
That was a great TNG episode, and not too far out of our reach even now. Imagine a swarm of 50,000 solar powered 100lb flying wing drones capable of staying aloft indefinitely that are mass produced and cost $5000 each. Leave your soldiers at home and deploy them over a war zone with orders to kill anything that fires at them, wears the enemy uniform or matches other criteria using bullets fired from 5000 feet up (firing in a vertical dive with software ballistic correction for crosswinds after the first shot should give pinpoint accuracy). Give each drone 200 rounds of ammunition and an on-board explosive charge. With the right targeting system, you can reduce the cost to kill an entire army of soldiers to $0.50/kill with an initial investment of $250M, which is pennies for the military. The swarm of drones could kill 5M enemies before reloading or being used as cruise missiles. Further, any aircraft engaging the swarm gets blown up by 20-30 simultaneous kamikazi attacks and the drone's explosive charges. You can take out non-hardened targets like cars and buildings with the same method. I guarantee you that Iran and China are both working on this now, as it is an effective means for them to challenge the US for air supremacy.
If something like that were to go haywire, you could create a massive no-go kill zone for years.
If you disagree, please post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like
It is for geopolitical control and great profit. Even our own government admits that we have created more terrorists than we kill, which I assume is not because our government and military are incompetent, it is just a form of job security. There wasn't nearly as much bloodshed and civil war in the Middle East until we went in with our military and intelligence agencies to institute "regime change" by way of war. Now there are civil wars (e.g., Libya, Syria and Iraq) where there had not been before we intervened militarily. We were not attacked by any of those countries, and it is an international war crime to commit unprovoked military aggression. Millions of refugees are fleeing the fighting. None of it had to be, and none of it has brought about any type of peace or stability, not even in Afghanistan where we have been the longest (who also did not attack the US).
You know full well that the US is the most aggressive country on the planet. We are not keeping the peace, we are making sure that peace can not happen and that the wars will go on indefinitely, thus keeping the region in turmoil, and keeping the profits flowing. Please point to one place where our military has produced "peace" since the first Gulf War. I just pointed to a number of places where we undid the peace, and created endless war.
A brain is a terrible thing to waste... Mind? That's debatable.
I disagree. Biological and chemical warfare is much more efficient at killing with the added bonus of not destroying infrastructure. Biological warfare probably has the highest likelihood of wiping all of humanity out due to a screw up since it is potentially self-replicating and infectious. Basically, we're more prone to go Resident Evil than Fallout.
Yes it's an anecdote! Were you expecting original research in a Slashdot comment?
... ...
Don't build them.
Perhaps? Maybe?
Just sayin'.
We suffer more in our imagination than in reality. - Seneca
blah-blah, white-man's burden, blah-blah...
Not saying the west should pull out or that we are in a post-violence world, but it might be a good idea to step back and see if the west is helping or hurting.
Too often we are jumping in to protect our "interests" instead of helping the "situation". It may be we are doing too much short term reacting...
FWIW, the US Civil War wasn't really much about instability as it was a conflict over the future of the recently annexed territory and the power of the central government. The small guy lost in the end (after lots of bloodshed) their right to secede from the union and were basically adopt the institutions of the union. Is that what you are saying what happens when an uneducated population, is forced to turn to a "Democratic Republic" (e.g., the USA's current form of govt)? ;^)
It's so much clearer now...
The so called "peace" we have today in the west is of course illusory (as seen by recent events like Crimea). The waves of immigrants to Europe fleeing the *real* instability in Syria and the economically challenged countries the middle east is showing the cracks in European stability.
Let's face the sad truth, stability that everyone desires seems to only draw on the wealth of a nation. Given the current assortment of "wealthy" nations historically used mercantilism to create much of their wealth from these in-stable countries, is it no wonder that we continue to attempt to project stability in a region to protect our interests. But what of *their* interests? Hence we return to the white man's burden argument... ;^)
I hate to use China as an example, but it used to be a dumping ground for European and Japanese influence peddling (e.g., opium war, concession ports, forced trade, occupation, etc), until they managed to get everyone to leave them alone for a few decades. Sure it was brutal (great-leap-forward, cultural revolution, etc), but they managed to dig themselves out of a hole into some reasonable form of stability mainly because they simply got wealthier without interference. Now they look like they might take over the world. Perhaps this is what people fear the most and keeps the west involved in other countries...