Slashdot Mirror
EFF: Google Should Not Help the US Military Build Unaccountable AI Systems (eff.org)
The Electronic Frontier Foundation's Peter Eckersley writes: Yesterday, The New York Times reported that there is widespread unrest amongst Google's employees about the company's work on a U.S. military project called "Project Maven." Google has claimed that its work on Maven is for "non-offensive uses only," but it seems that the company is building computer vision systems to flag objects and people seen by military drones for human review. This may in some cases lead to subsequent targeting by missile strikes. EFF has been mulling the ethical implications of such contracts, and we have some advice for Google and other tech companies that are considering building military AI systems.
The EFF lists several "starting points" any company, or any worker, considering whether to work with the military on a project with potentially dangerous or risk AI applications should be asking:
1. Is it possible to create strong and binding international institutions or agreements that define acceptable military uses and limitations in the use of AI? While this is not an easy task, the current lack of such structures is troubling. There are serious and potentially destabilizing impacts from deploying AI in any military setting not clearly governed by settled rules of war. The use of AI in potential target identification processes is one clear category of uses that must be governed by law.
2.Is there a robust process for studying and mitigating the safety and geopolitical stability problems that could result from the deployment of military AI? Does this process apply before work commences, along the development pathway and after deployment? Could it incorporate the sufficient expertise to address subtle and complex technical problems? And would those leading the process have sufficient independence and authority to ensure that it can check companies' and military agencies' decisions?
3.Are the contracting agencies willing to commit to not using AI for autonomous offensive weapons? Or to ensuring that any defensive autonomous systems are carefully engineered to avoid risks of accidental harm or conflict escalation? Are present testing and formal verification methods adequate for that task?
4.Can there be transparent, accountable oversight from an independently constituted ethics board or similar entity with both the power to veto aspects of the program and the power to bring public transparency to issues where necessary or appropriate? For example, while Alphabet's AI-focused subsidiary DeepMind has committed to independent ethics review, we are not aware of similar commitments from Google itself. Given this letter, we are concerned that the internal transparency, review, and discussion of Project Maven inside Google was inadequate. Any project review process must be transparent, informed, and independent. While it remains difficult to ensure that that is the case, without such independent oversight, a project runs real risk of harm.
The EFF lists several "starting points" any company, or any worker, considering whether to work with the military on a project with potentially dangerous or risk AI applications should be asking:
1. Is it possible to create strong and binding international institutions or agreements that define acceptable military uses and limitations in the use of AI? While this is not an easy task, the current lack of such structures is troubling. There are serious and potentially destabilizing impacts from deploying AI in any military setting not clearly governed by settled rules of war. The use of AI in potential target identification processes is one clear category of uses that must be governed by law.
2.Is there a robust process for studying and mitigating the safety and geopolitical stability problems that could result from the deployment of military AI? Does this process apply before work commences, along the development pathway and after deployment? Could it incorporate the sufficient expertise to address subtle and complex technical problems? And would those leading the process have sufficient independence and authority to ensure that it can check companies' and military agencies' decisions?
3.Are the contracting agencies willing to commit to not using AI for autonomous offensive weapons? Or to ensuring that any defensive autonomous systems are carefully engineered to avoid risks of accidental harm or conflict escalation? Are present testing and formal verification methods adequate for that task?
4.Can there be transparent, accountable oversight from an independently constituted ethics board or similar entity with both the power to veto aspects of the program and the power to bring public transparency to issues where necessary or appropriate? For example, while Alphabet's AI-focused subsidiary DeepMind has committed to independent ethics review, we are not aware of similar commitments from Google itself. Given this letter, we are concerned that the internal transparency, review, and discussion of Project Maven inside Google was inadequate. Any project review process must be transparent, informed, and independent. While it remains difficult to ensure that that is the case, without such independent oversight, a project runs real risk of harm.
We’re all working really hard to burn the world down. How dare that pack of libtard snowflakes get in the way of our fun. Bring on the SkyNet!
1. no
2. no
3. no
4. no
Just look at current international agreements and how often they are ignored.
-- Will program for bandwidth
Answers to the questions: 1. No. Nothing international is binding, nor can it ever be. 2. Maybe, but it wouldn't change anything so it's not worth the expense. Let some universities throw donor dollars at the question, argue it philosophically, and encourage their snowflakes to skip class and protest it while it gets implemented anyway. 3. Not in a free market. 4. That's not how our government works when it comes to secret programs. And if you want ethics, the folks who leaked this info should go to jail.
When is it acceptable to help the military? There are a lot of applications that could be used for surveillance and non-offensive purposes, but could also be used to attack or kill people. As a civilian researcher developing technology with military funding, it's not clear how the work will eventually be used.
I was involved with a project that was funded by a US military office. To remain anonymous, I won't say exactly where my funding came from or what project I was working on, but I've seen calls to fund this research from the Air Force Office of Scientific Research and also from the Army Research Laboratory. Atmospheric wind shear can be exploited by aircraft to converse power through dynamic soaring. During the day, when the surface is being heated by solar radiation, the aircraft can fly in thermals and other areas of ascent in the planetary boundary layer, usually in the lowest 1-2 km, and exploit static soaring. Autonomous systems such as drones can use this information in planning their flight path and conserve power, which allows them to stay in flight longer and extend the missions they can carry out.
Although there are civilian uses for this technology, my work was funded by a grant from the US military. I had no role in designing the project or soliciting funding, but I was employed with funds from the grant. There are non-violent uses for this technology, even in military applications. But they can also be used to attack people.
Drones could be used to deliver supplies including food or medical supplies. Drones could be used to locate people in search-and-rescue missions. Drones could be used by the Coast Guard to patrol smugglers bringing contraband and drugs in the US. Drones could also be used to patrol the southern border of the US and would probably be quite a bit more useful than a wall. They could be used to gather surveillance of enemy combatants who may pose a risk both to US troops and civilians, to allow people time to evacuate or find shelter. None of these are violent, and many of these applications are not controversial at all. However, drones can also carry weapons and be used to attack and kill people.
As a researcher, I have no control over how my research is used by the military. I can use the results in other projects for civilian use to benefit people. A meteorologist might use drones to collect data around severe thunderstorms to improve weather forecasting and provide better warnings to people. This technology could be used to extend the flight of those drones and help gather data that can save lives. However, the research is funded by the military, and the military could use it to kill people.
Is it wrong to accept the funding and conduct research that can benefit civilians but can also be used for harm? Most technology can be used for non-violent purposes that are overwhelmingly beneficial to people. Even nuclear weapons could be used to benefit humanity if, for example, they were used to destroy a large near Earth asteroid that might collide with Earth. As a researcher, I have no control over how the military would use the results of my work. But that work could be used for both beneficial and harmful purposes. Is it wrong to accept that funding and do research for the military? When is it acceptable to do research with military funding and when is it not? Where do you draw the line?
You mentioned a lot of non-violent uses of technology that has been funded by the military, and military resources being used to deliver food, medical supplies, and other relief. That's all true and good. Versus violent uses, you say, which are bad.
ALSO there are countries who want to wipe us out. There are countries with the ability to kill millions of Americans. What has happened before will happen again - there will be a country who *wants* to attack us and *can*. The US response to Japanese surprise attack at Pearl Harbor was very much violent - as it needed to be. They were bombing us - by surprise, pretending to negotiate trade agreements with us while their ships were underway to attack us. Swift and violent action to protect ourselves was the right action, and the only option.
I most certainly don't agree with every use of the US military. I AM very glad for its primary use - being a massive deterrent to anyone who might think about attacking us. You may think "no military would ever attack the United States". That's true, at the moment. But why? Why wouldn't North Korea, or Iran, Russia, or China*, send bombers to the US? Because we would crush them, that's why. The REASON we don't have to fight off an attack today is precisely because of our military capability.
That's the main use of a superpower military - making an attack on us inconceivable by simply having the *capability* to win decisively and quickly if we were attacked. That's a good thing. I don't want our country to be defenseless, a tempting target. Our capacity for overwhelming violence is a large part of why other countries don't initiate violence against us or our friends.
* The situation with China specifically is a bit more complex at the moment. Trade is important to them, and they have some significant military power. They have also noticed that they can attack us via cyber warfare and we don't treat it as an attack, we let them get away with that.
The internet itself is based on intellectual property paid for and developed by the US military. GPS systems which are at the core of many computer applications we all love and use is run off a system originally used developed for the US military... there's hundreds more examples.
Technology is just that. It can be used for multiple purposes, very often the original intent of the technology can end up being used in completely ways, meaning that technology intended for military use can end up becoming something like the world wide web and technology not intended for military use can end up being used to take lives e.g.chlorine gas is used widely within industry for thousands of purposes... other than gassing people in idlib
AI driven defensive and offensive weapons systems are crucial to the survival of any power in the future world. We need to redouble efforts into making them more efficient. It's as simple as if we don't, they will and we will be lost to what little time is left for mankind to be written into any history books.
That said, we could focus hard solving the problems of differentiating between legitimate and illegitimate targets. We could focus on systems to save lives and win the hearts and minds of local populations. The only way an enemy is truly defeated is if you either killed them all (which is possible) or win over their hearts and minds (which is harder).
Above all, it would be extremely beneficial to focus on non-lethal weapons systems. For example, small drones with tranquilizer darts or slime bombs that make an area so slick that enemy troops cannot traverse enabling a battle win by maneuver. Catch enemy soldiers with nets... Whatever it is--war technologies require extreme innovation and creativity, be they lethal or not. The non-lethal approaches add the advantages of:
1. Capturing provides people to interrogate, leading to information that is key to more wins.
2. Non-harm is far more effective at winning the hearts and minds of an enemy.
3. Non-harm is far better for Public Relations.
4. Non-harm is morally superior, when and where it is reasonably possible.