Slashdot Mirror
EFF: Google Should Not Help the US Military Build Unaccountable AI Systems (eff.org)
The Electronic Frontier Foundation's Peter Eckersley writes: Yesterday, The New York Times reported that there is widespread unrest amongst Google's employees about the company's work on a U.S. military project called "Project Maven." Google has claimed that its work on Maven is for "non-offensive uses only," but it seems that the company is building computer vision systems to flag objects and people seen by military drones for human review. This may in some cases lead to subsequent targeting by missile strikes. EFF has been mulling the ethical implications of such contracts, and we have some advice for Google and other tech companies that are considering building military AI systems.
The EFF lists several "starting points" any company, or any worker, considering whether to work with the military on a project with potentially dangerous or risk AI applications should be asking:
1. Is it possible to create strong and binding international institutions or agreements that define acceptable military uses and limitations in the use of AI? While this is not an easy task, the current lack of such structures is troubling. There are serious and potentially destabilizing impacts from deploying AI in any military setting not clearly governed by settled rules of war. The use of AI in potential target identification processes is one clear category of uses that must be governed by law.
2.Is there a robust process for studying and mitigating the safety and geopolitical stability problems that could result from the deployment of military AI? Does this process apply before work commences, along the development pathway and after deployment? Could it incorporate the sufficient expertise to address subtle and complex technical problems? And would those leading the process have sufficient independence and authority to ensure that it can check companies' and military agencies' decisions?
3.Are the contracting agencies willing to commit to not using AI for autonomous offensive weapons? Or to ensuring that any defensive autonomous systems are carefully engineered to avoid risks of accidental harm or conflict escalation? Are present testing and formal verification methods adequate for that task?
4.Can there be transparent, accountable oversight from an independently constituted ethics board or similar entity with both the power to veto aspects of the program and the power to bring public transparency to issues where necessary or appropriate? For example, while Alphabet's AI-focused subsidiary DeepMind has committed to independent ethics review, we are not aware of similar commitments from Google itself. Given this letter, we are concerned that the internal transparency, review, and discussion of Project Maven inside Google was inadequate. Any project review process must be transparent, informed, and independent. While it remains difficult to ensure that that is the case, without such independent oversight, a project runs real risk of harm.
The EFF lists several "starting points" any company, or any worker, considering whether to work with the military on a project with potentially dangerous or risk AI applications should be asking:
1. Is it possible to create strong and binding international institutions or agreements that define acceptable military uses and limitations in the use of AI? While this is not an easy task, the current lack of such structures is troubling. There are serious and potentially destabilizing impacts from deploying AI in any military setting not clearly governed by settled rules of war. The use of AI in potential target identification processes is one clear category of uses that must be governed by law.
2.Is there a robust process for studying and mitigating the safety and geopolitical stability problems that could result from the deployment of military AI? Does this process apply before work commences, along the development pathway and after deployment? Could it incorporate the sufficient expertise to address subtle and complex technical problems? And would those leading the process have sufficient independence and authority to ensure that it can check companies' and military agencies' decisions?
3.Are the contracting agencies willing to commit to not using AI for autonomous offensive weapons? Or to ensuring that any defensive autonomous systems are carefully engineered to avoid risks of accidental harm or conflict escalation? Are present testing and formal verification methods adequate for that task?
4.Can there be transparent, accountable oversight from an independently constituted ethics board or similar entity with both the power to veto aspects of the program and the power to bring public transparency to issues where necessary or appropriate? For example, while Alphabet's AI-focused subsidiary DeepMind has committed to independent ethics review, we are not aware of similar commitments from Google itself. Given this letter, we are concerned that the internal transparency, review, and discussion of Project Maven inside Google was inadequate. Any project review process must be transparent, informed, and independent. While it remains difficult to ensure that that is the case, without such independent oversight, a project runs real risk of harm.
Can these questions be answered in the affirmative for any advanced weapons system? Seems sort of an impossibly high bar they've set.
Irony: Agile development has too much intertia to be abandoned now.
You mentioned a lot of non-violent uses of technology that has been funded by the military, and military resources being used to deliver food, medical supplies, and other relief. That's all true and good. Versus violent uses, you say, which are bad.
ALSO there are countries who want to wipe us out. There are countries with the ability to kill millions of Americans. What has happened before will happen again - there will be a country who *wants* to attack us and *can*. The US response to Japanese surprise attack at Pearl Harbor was very much violent - as it needed to be. They were bombing us - by surprise, pretending to negotiate trade agreements with us while their ships were underway to attack us. Swift and violent action to protect ourselves was the right action, and the only option.
I most certainly don't agree with every use of the US military. I AM very glad for its primary use - being a massive deterrent to anyone who might think about attacking us. You may think "no military would ever attack the United States". That's true, at the moment. But why? Why wouldn't North Korea, or Iran, Russia, or China*, send bombers to the US? Because we would crush them, that's why. The REASON we don't have to fight off an attack today is precisely because of our military capability.
That's the main use of a superpower military - making an attack on us inconceivable by simply having the *capability* to win decisively and quickly if we were attacked. That's a good thing. I don't want our country to be defenseless, a tempting target. Our capacity for overwhelming violence is a large part of why other countries don't initiate violence against us or our friends.
* The situation with China specifically is a bit more complex at the moment. Trade is important to them, and they have some significant military power. They have also noticed that they can attack us via cyber warfare and we don't treat it as an attack, we let them get away with that.
Most technology can be used for non-violent purposes that are overwhelmingly beneficial to people.
In addition, violence itself is merely another tool, one which can be put to good purpose. Military forces are important tools of public policy. They can be used to end horrific suffering and they can be used to maintain peace, by explicitly threatening anyone who would break the peace with violent consequences.
The underlying assumption of your post seems to be that military capability is an unalloyed evil. I'll grant that in an ideal world it would be completely unnecessary, but that is not the world in which we live. If we're concerned about misuse of military power, it seems to me that the armed forces already have more than enough capability to have us shaking in our boots, and it's not clear to me that adding AI to the mix (assuming the AI doesn't get out of control) significantly changes anything.
To make military forces "safe", we need to (a) ensure that they remain subject to civilian control and (b) ensure that civilian control acts responsibly. I'll grant that we seriously undermined (b) in the 2016 election, but that's a repairable problem.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.