Slashdot Mirror
EFF: Google Should Not Help the US Military Build Unaccountable AI Systems (eff.org)
The Electronic Frontier Foundation's Peter Eckersley writes: Yesterday, The New York Times reported that there is widespread unrest amongst Google's employees about the company's work on a U.S. military project called "Project Maven." Google has claimed that its work on Maven is for "non-offensive uses only," but it seems that the company is building computer vision systems to flag objects and people seen by military drones for human review. This may in some cases lead to subsequent targeting by missile strikes. EFF has been mulling the ethical implications of such contracts, and we have some advice for Google and other tech companies that are considering building military AI systems.
The EFF lists several "starting points" any company, or any worker, considering whether to work with the military on a project with potentially dangerous or risk AI applications should be asking:
1. Is it possible to create strong and binding international institutions or agreements that define acceptable military uses and limitations in the use of AI? While this is not an easy task, the current lack of such structures is troubling. There are serious and potentially destabilizing impacts from deploying AI in any military setting not clearly governed by settled rules of war. The use of AI in potential target identification processes is one clear category of uses that must be governed by law.
2.Is there a robust process for studying and mitigating the safety and geopolitical stability problems that could result from the deployment of military AI? Does this process apply before work commences, along the development pathway and after deployment? Could it incorporate the sufficient expertise to address subtle and complex technical problems? And would those leading the process have sufficient independence and authority to ensure that it can check companies' and military agencies' decisions?
3.Are the contracting agencies willing to commit to not using AI for autonomous offensive weapons? Or to ensuring that any defensive autonomous systems are carefully engineered to avoid risks of accidental harm or conflict escalation? Are present testing and formal verification methods adequate for that task?
4.Can there be transparent, accountable oversight from an independently constituted ethics board or similar entity with both the power to veto aspects of the program and the power to bring public transparency to issues where necessary or appropriate? For example, while Alphabet's AI-focused subsidiary DeepMind has committed to independent ethics review, we are not aware of similar commitments from Google itself. Given this letter, we are concerned that the internal transparency, review, and discussion of Project Maven inside Google was inadequate. Any project review process must be transparent, informed, and independent. While it remains difficult to ensure that that is the case, without such independent oversight, a project runs real risk of harm.
The EFF lists several "starting points" any company, or any worker, considering whether to work with the military on a project with potentially dangerous or risk AI applications should be asking:
1. Is it possible to create strong and binding international institutions or agreements that define acceptable military uses and limitations in the use of AI? While this is not an easy task, the current lack of such structures is troubling. There are serious and potentially destabilizing impacts from deploying AI in any military setting not clearly governed by settled rules of war. The use of AI in potential target identification processes is one clear category of uses that must be governed by law.
2.Is there a robust process for studying and mitigating the safety and geopolitical stability problems that could result from the deployment of military AI? Does this process apply before work commences, along the development pathway and after deployment? Could it incorporate the sufficient expertise to address subtle and complex technical problems? And would those leading the process have sufficient independence and authority to ensure that it can check companies' and military agencies' decisions?
3.Are the contracting agencies willing to commit to not using AI for autonomous offensive weapons? Or to ensuring that any defensive autonomous systems are carefully engineered to avoid risks of accidental harm or conflict escalation? Are present testing and formal verification methods adequate for that task?
4.Can there be transparent, accountable oversight from an independently constituted ethics board or similar entity with both the power to veto aspects of the program and the power to bring public transparency to issues where necessary or appropriate? For example, while Alphabet's AI-focused subsidiary DeepMind has committed to independent ethics review, we are not aware of similar commitments from Google itself. Given this letter, we are concerned that the internal transparency, review, and discussion of Project Maven inside Google was inadequate. Any project review process must be transparent, informed, and independent. While it remains difficult to ensure that that is the case, without such independent oversight, a project runs real risk of harm.
Can these questions be answered in the affirmative for any advanced weapons system? Seems sort of an impossibly high bar they've set.
Irony: Agile development has too much intertia to be abandoned now.
Well if you can get China and Russia to stop improving their own militaries than by all means don't let the US do it. There is a big fallacy today that says if the US didn't build advanced weapons then there would no reason for others to create their own. The world is heading towards a melt down and I want the US to have the weapons needed to come out on top. The US already has to put up with people clamoring for total transparency in it's intelligence and counter intelligence agencies. These are the same morons who evidently don't know what the word "covert" actually means. We are on the cusp of another world war because the generation that experienced the last one are dying off. People in Europe and the US who refuse to understand that all the rights they enjoy today are the direct result of the allies winning WW2 and costs roughly 60 million dead and trillions of dollars worth of damage. What would the world look like today had Germany and Japan had won WW2. What would the world look like if the US had not kept the USSR from imposing it's will across the planet. You only have to look at the state of eastern Europe when the Russians were running the place. Any one who has a problem with working on anything related to advanced military technology are free to find another job that doesn't offend their delicate sensibilities. We use technology that started life as military sponsored projects. Modern satellite communications, aviation technologies, computer technologies , advanced materials, medical related technologies, rocket technologies, information packet protocols and routers needed for creating distributed and redundant electronic communications that paved the way for today's Internet. And the computer processing, jet engines, avionics, materials, and HMI technologies that were in the science fiction realm 20 years ago were realized when building today's 5th generation jet fighters. So EFF should identify all the companies involved and come up with reasons you should not work at.
You mentioned a lot of non-violent uses of technology that has been funded by the military, and military resources being used to deliver food, medical supplies, and other relief. That's all true and good. Versus violent uses, you say, which are bad.
ALSO there are countries who want to wipe us out. There are countries with the ability to kill millions of Americans. What has happened before will happen again - there will be a country who *wants* to attack us and *can*. The US response to Japanese surprise attack at Pearl Harbor was very much violent - as it needed to be. They were bombing us - by surprise, pretending to negotiate trade agreements with us while their ships were underway to attack us. Swift and violent action to protect ourselves was the right action, and the only option.
I most certainly don't agree with every use of the US military. I AM very glad for its primary use - being a massive deterrent to anyone who might think about attacking us. You may think "no military would ever attack the United States". That's true, at the moment. But why? Why wouldn't North Korea, or Iran, Russia, or China*, send bombers to the US? Because we would crush them, that's why. The REASON we don't have to fight off an attack today is precisely because of our military capability.
That's the main use of a superpower military - making an attack on us inconceivable by simply having the *capability* to win decisively and quickly if we were attacked. That's a good thing. I don't want our country to be defenseless, a tempting target. Our capacity for overwhelming violence is a large part of why other countries don't initiate violence against us or our friends.
* The situation with China specifically is a bit more complex at the moment. Trade is important to them, and they have some significant military power. They have also noticed that they can attack us via cyber warfare and we don't treat it as an attack, we let them get away with that.
It's not that the US develops those weapons, as much as they US gets involved in a lot of other countries. Reducing that would be a good start, but unfortunately it looks unlikely under the current administration.
Advanced weapons don't make a huge difference really. The US still has enough nukes to maintain MAD. No missile shield is reliably enough to defend against that arsenal, and the same goes for current Russian ICBMs. All this stuff about hypersonic nuclear cruise missiles and torpedo drones is largely posturing, adding nuclear warheads to technologies developed for other kinds of warfare.
The big danger now is from the new cold war. Cyber attacks, interference with democracy and supplying arms that can't be traced. Basically the same as the old cold war, with newer tech.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
Most technology can be used for non-violent purposes that are overwhelmingly beneficial to people.
In addition, violence itself is merely another tool, one which can be put to good purpose. Military forces are important tools of public policy. They can be used to end horrific suffering and they can be used to maintain peace, by explicitly threatening anyone who would break the peace with violent consequences.
The underlying assumption of your post seems to be that military capability is an unalloyed evil. I'll grant that in an ideal world it would be completely unnecessary, but that is not the world in which we live. If we're concerned about misuse of military power, it seems to me that the armed forces already have more than enough capability to have us shaking in our boots, and it's not clear to me that adding AI to the mix (assuming the AI doesn't get out of control) significantly changes anything.
To make military forces "safe", we need to (a) ensure that they remain subject to civilian control and (b) ensure that civilian control acts responsibly. I'll grant that we seriously undermined (b) in the 2016 election, but that's a repairable problem.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
AI driven defensive and offensive weapons systems are crucial to the survival of any power in the future world. We need to redouble efforts into making them more efficient. It's as simple as if we don't, they will and we will be lost to what little time is left for mankind to be written into any history books.
That said, we could focus hard solving the problems of differentiating between legitimate and illegitimate targets. We could focus on systems to save lives and win the hearts and minds of local populations. The only way an enemy is truly defeated is if you either killed them all (which is possible) or win over their hearts and minds (which is harder).
Above all, it would be extremely beneficial to focus on non-lethal weapons systems. For example, small drones with tranquilizer darts or slime bombs that make an area so slick that enemy troops cannot traverse enabling a battle win by maneuver. Catch enemy soldiers with nets... Whatever it is--war technologies require extreme innovation and creativity, be they lethal or not. The non-lethal approaches add the advantages of:
1. Capturing provides people to interrogate, leading to information that is key to more wins.
2. Non-harm is far more effective at winning the hearts and minds of an enemy.
3. Non-harm is far better for Public Relations.
4. Non-harm is morally superior, when and where it is reasonably possible.