Slashdot Mirror
EFF: Google Should Not Help the US Military Build Unaccountable AI Systems (eff.org)
The Electronic Frontier Foundation's Peter Eckersley writes: Yesterday, The New York Times reported that there is widespread unrest amongst Google's employees about the company's work on a U.S. military project called "Project Maven." Google has claimed that its work on Maven is for "non-offensive uses only," but it seems that the company is building computer vision systems to flag objects and people seen by military drones for human review. This may in some cases lead to subsequent targeting by missile strikes. EFF has been mulling the ethical implications of such contracts, and we have some advice for Google and other tech companies that are considering building military AI systems.
The EFF lists several "starting points" any company, or any worker, considering whether to work with the military on a project with potentially dangerous or risk AI applications should be asking:
1. Is it possible to create strong and binding international institutions or agreements that define acceptable military uses and limitations in the use of AI? While this is not an easy task, the current lack of such structures is troubling. There are serious and potentially destabilizing impacts from deploying AI in any military setting not clearly governed by settled rules of war. The use of AI in potential target identification processes is one clear category of uses that must be governed by law.
2.Is there a robust process for studying and mitigating the safety and geopolitical stability problems that could result from the deployment of military AI? Does this process apply before work commences, along the development pathway and after deployment? Could it incorporate the sufficient expertise to address subtle and complex technical problems? And would those leading the process have sufficient independence and authority to ensure that it can check companies' and military agencies' decisions?
3.Are the contracting agencies willing to commit to not using AI for autonomous offensive weapons? Or to ensuring that any defensive autonomous systems are carefully engineered to avoid risks of accidental harm or conflict escalation? Are present testing and formal verification methods adequate for that task?
4.Can there be transparent, accountable oversight from an independently constituted ethics board or similar entity with both the power to veto aspects of the program and the power to bring public transparency to issues where necessary or appropriate? For example, while Alphabet's AI-focused subsidiary DeepMind has committed to independent ethics review, we are not aware of similar commitments from Google itself. Given this letter, we are concerned that the internal transparency, review, and discussion of Project Maven inside Google was inadequate. Any project review process must be transparent, informed, and independent. While it remains difficult to ensure that that is the case, without such independent oversight, a project runs real risk of harm.
The EFF lists several "starting points" any company, or any worker, considering whether to work with the military on a project with potentially dangerous or risk AI applications should be asking:
1. Is it possible to create strong and binding international institutions or agreements that define acceptable military uses and limitations in the use of AI? While this is not an easy task, the current lack of such structures is troubling. There are serious and potentially destabilizing impacts from deploying AI in any military setting not clearly governed by settled rules of war. The use of AI in potential target identification processes is one clear category of uses that must be governed by law.
2.Is there a robust process for studying and mitigating the safety and geopolitical stability problems that could result from the deployment of military AI? Does this process apply before work commences, along the development pathway and after deployment? Could it incorporate the sufficient expertise to address subtle and complex technical problems? And would those leading the process have sufficient independence and authority to ensure that it can check companies' and military agencies' decisions?
3.Are the contracting agencies willing to commit to not using AI for autonomous offensive weapons? Or to ensuring that any defensive autonomous systems are carefully engineered to avoid risks of accidental harm or conflict escalation? Are present testing and formal verification methods adequate for that task?
4.Can there be transparent, accountable oversight from an independently constituted ethics board or similar entity with both the power to veto aspects of the program and the power to bring public transparency to issues where necessary or appropriate? For example, while Alphabet's AI-focused subsidiary DeepMind has committed to independent ethics review, we are not aware of similar commitments from Google itself. Given this letter, we are concerned that the internal transparency, review, and discussion of Project Maven inside Google was inadequate. Any project review process must be transparent, informed, and independent. While it remains difficult to ensure that that is the case, without such independent oversight, a project runs real risk of harm.
When is it acceptable to help the military? There are a lot of applications that could be used for surveillance and non-offensive purposes, but could also be used to attack or kill people. As a civilian researcher developing technology with military funding, it's not clear how the work will eventually be used.
I was involved with a project that was funded by a US military office. To remain anonymous, I won't say exactly where my funding came from or what project I was working on, but I've seen calls to fund this research from the Air Force Office of Scientific Research and also from the Army Research Laboratory. Atmospheric wind shear can be exploited by aircraft to converse power through dynamic soaring. During the day, when the surface is being heated by solar radiation, the aircraft can fly in thermals and other areas of ascent in the planetary boundary layer, usually in the lowest 1-2 km, and exploit static soaring. Autonomous systems such as drones can use this information in planning their flight path and conserve power, which allows them to stay in flight longer and extend the missions they can carry out.
Although there are civilian uses for this technology, my work was funded by a grant from the US military. I had no role in designing the project or soliciting funding, but I was employed with funds from the grant. There are non-violent uses for this technology, even in military applications. But they can also be used to attack people.
Drones could be used to deliver supplies including food or medical supplies. Drones could be used to locate people in search-and-rescue missions. Drones could be used by the Coast Guard to patrol smugglers bringing contraband and drugs in the US. Drones could also be used to patrol the southern border of the US and would probably be quite a bit more useful than a wall. They could be used to gather surveillance of enemy combatants who may pose a risk both to US troops and civilians, to allow people time to evacuate or find shelter. None of these are violent, and many of these applications are not controversial at all. However, drones can also carry weapons and be used to attack and kill people.
As a researcher, I have no control over how my research is used by the military. I can use the results in other projects for civilian use to benefit people. A meteorologist might use drones to collect data around severe thunderstorms to improve weather forecasting and provide better warnings to people. This technology could be used to extend the flight of those drones and help gather data that can save lives. However, the research is funded by the military, and the military could use it to kill people.
Is it wrong to accept the funding and conduct research that can benefit civilians but can also be used for harm? Most technology can be used for non-violent purposes that are overwhelmingly beneficial to people. Even nuclear weapons could be used to benefit humanity if, for example, they were used to destroy a large near Earth asteroid that might collide with Earth. As a researcher, I have no control over how the military would use the results of my work. But that work could be used for both beneficial and harmful purposes. Is it wrong to accept that funding and do research for the military? When is it acceptable to do research with military funding and when is it not? Where do you draw the line?
The internet itself is based on intellectual property paid for and developed by the US military. GPS systems which are at the core of many computer applications we all love and use is run off a system originally used developed for the US military... there's hundreds more examples.
Technology is just that. It can be used for multiple purposes, very often the original intent of the technology can end up being used in completely ways, meaning that technology intended for military use can end up becoming something like the world wide web and technology not intended for military use can end up being used to take lives e.g.chlorine gas is used widely within industry for thousands of purposes... other than gassing people in idlib