Slashdot Mirror
Why Engineers Must Consider the Ethical Implications of Their Work
An anonymous reader writes "An article by Abbas El-Zein at The Guardian explores the ethical responsibilities for engineers who create and maintain 'technologies of violence.' He says, 'Engineers who see themselves as builders of the shelter and infrastructure for human needs also use their expertise in order to destroy and kill more efficiently. When doctors or nurses use their knowledge of anatomy in order to torture or conduct medical experiments on helpless subjects, we are rightly outraged. Why doesn't society seem to apply the same standards to engineers? There is more than one answer to the question of course, but two points are especially pertinent: the common good we engineers see ourselves serving and our relationship to authority. ... Our ethics have become mostly technical: how to design properly, how to not cut corners, how to serve our clients well. We work hard to prevent failure of the systems we build, but only in relation to what these systems are meant to do, rather than the way they might actually be utilised, or whether they should have been built at all. We are not amoral, far from it; it's just that we have steered ourselves into a place where our morality has a smaller scope.'"
The existence of nuclear bombs have saved millions of lives over the last 60 years. Sounds pretty ethical to me.
While it is hard to draw exact parallels, society already holds engineers to similar standards to doctors. The outrage over doctors experimenting on helpless test subjects is pretty similar to, say, when engineers use live subjects for testing weapons.
Technology is a tool, and a tool can be used a weapon. You should blame the one who wields the weapon. Do we blame Pasteur for biological warfare? I do not, but without him much of what we know about making bio-weapons would not exist.
You can study rockets to go to the moon, but eventually someone is going to shoot them at their neighbor.
You can study a way to get cheap energy for everyone, but eventually someone will make a bomb.
You can create a large forum for the people that is resistant to people stopping you from communication, but someone will eventually create a global spy system that watches everything you do.
It is unfortunate, but I would place the blame not on the person who makes the technology, but the one who decides how to use it. When we complain about doctors helping with torture, we are complaining about the ones there to extend the pain, not the ones who came up with ways keep people alive.
A lot of the engineers I've known who worked on military equipment do consider the ethical implications of their work. They feel they are helping protect our troops (see the beginning of Iron Man 1, where Tony Stark uses a similar justification), or something similar.
Other guys I know are just happy to have a job. Some people consider it unethical to work in the corporate world at all, so just because you consider something unethical, doesn't mean everyone considers it unethical. The NSA has the purpose of catching terrorists, which is a good goal. The reason we don't like them is because of the abuses, not because of their goal.
"First they came for the slanderers and i said nothing."
While it is hard to draw exact parallels, society already holds engineers to similar standards to doctors. The outrage over doctors experimenting on helpless test subjects is pretty similar to, say, when engineers use live subjects for testing weapons.
Yeah. The article's author is making a poor analogy. Blaming engineers would be more akin to blaming the scalpel designer for the doctor's experimentation. Its not the scalpel or the gun that is the problem, it is the mind and the intentions behind the hand holding the scalpel or gun. Both can be used for good or bad.
Short of WMD the issue is not as simple as the author suggests.
I'm not sure he saw it that way:
In fact, he was later very much opposed to its use:
Lost at C:>. Found at C.
Every single person needs to do this. If you work in the weapons industry and don't feel bad about it, you are a psychopath. Simple.
So your view is that good guys produce no weapons, and bad guys produce lots of weapons...and it's that simple? What happens when the bad guys decide to be bad with their weapons by turning them on the unarmed, defenseless good guys?
Or, is it that some people should work in the weapons industry, but feel really really bad about it. And of course, those of us who have happy jobs are the better people, since we took the "high road" by forcing someone else to be bad. Perhaps we could have a lottery (Shirley Jackson's version), to decide who among us has to be bad so that our good lives can continue safely. Then we can all sit back and bask in the shiny, sunny warmth of just how good we all are, unlike those bad, bad people who make weapons that we can defend ourselves with...
For your security, this post has been encrypted with ROT-13, twice.
It's from the Bhagavad Gita, quoted in this context by Robert Oppenheimer. Nothing to do with Confucius.
Playing the devil's advocate, when was the last time you got sued for malpractice for bugs in your code?
Support the EFF and Creative Commons. The war is coming, and they're supporting you...
I work in "the weapons industry". I sleep well knowing that my projects are being used so that my baby girls can sleep well at night. But then, I've seen first hand what motivated, evil people do when nobody stops them. But then, I've been to some nasty parts of Africa. At the risk of Godwining the thread, I strongly encourage you to go tour one of the concentration camps in Germany. Those people exist, and to paraphrase, "People sleep peaceably in their beds at night only because rough men stand ready to do violence on their behalf."-Orwell,
Playing the devil's advocate, when was the last time you got sued for malpractice for bugs in your code?
What is wrong with that concept, really?
I've no problem hearing Kaching! every time I get a blue screen.
If Pandora's box is destined to be opened, *I* want to be the one to open it.
Submitter doesn't like humanity very much. He wishes there were laws, rules, regulations, and guide lines for everything. He wants to hold engineers responsible for their discoveries. He wants to judge each discovery as "good" or "bad", then reward or punish the engineers, scientists, and the craftsmen for whatever results.
Sad as it is, I prefer the world we have, in which men and woman exercise free will.
https://www.youtube.com/watch?v=YpCASVFyQoE
"Windows is like the faint smell of piss in a subway: it's there, and there's nothing you can do about it." - Charlie Br
When doctors or nurses use their knowledge of anatomy in order to torture or conduct medical experiments on helpless subjects, we are rightly outraged. Why doesn't society seem to apply the same standards to engineers?
When a doctor tortures a patient there is a direct cause and effect from the doctor's actions to the pain and suffering of the victim.
When an engineer designs a weapon, he's not actually causing the pain and suffering. Once you get away from "complete responsibility", the rest is easy:
1) If I don't do it, someone else will
2) I need to feed myself and my family
3) It'll only be used on the bad guys
4) It helps protect my country
5) It's the user's responsibility, not mine
6) The boss thinks it's a good idea
7) It has significant non-evil uses
8) No one will ever know it was me
For a concrete example, consider the Collateral Murder video from a couple of years back. Who was responsible for these deaths?
The helicopter pilots got the go-ahead from their commanders, the commanders [probably] got the go-ahead from intelligence services, the services made the correct decision based on the information they had, and the information was somehow "wrong".
Who's to blame for the collateral murder incident? By deftly distributing blame among many players, it changes from personal responsibility to "a failure of the system", or "a tragic accident".
For a second example, consider Bush's Iraq war: he was on TV stating that he had convincing evidence of WMDs in Iraq. A couple of years later it came out that the intelligence services had never said this and tried to convince the president of the opposite. Bush's response was: "We [the administration] didn't get the message". (Note the use of "we" in his statement.)
Who's responsible for the war? The President says he got bad intelligence, the intelligence services say they never gave bad intelligence. It's impossible to lay the blame on someone, it's a "failure of the system".
But don't worry, the problem is fixed - it'll never happen again.
(Epilogue: The Gulf oil spill was largely enabled by failures of the Minerals Management Service, who is responsible for overseeing the safety procedures of off-shore drilling. The problems were largely fixed by renaming the service to Bureau of Ocean Management. The problem is fixed, now we won't have any more disasters. Sorry about that...)
Strictly speaking coders are not engineers. We use that term colloquially but I definitely got the impression that the article was speaking primarily of PEs.
Very few engineers of any sort are PEs, at least in the US. Whether you are a classical mechanical/electrical engineer, or a coder, seems to be purely a subjective distinction. Some coders I know are definitely engineers, some are not (by my world view).
I think the article applies to any of the above, but the cited examples may be the domain of PEs - which as stated in the article - are not funded at all like corporate engineers, and thus have concerns more relevant to their funding model. Plenty of coders are involved in making missiles, and in fact some distruptive things (like bitcoin) were created by coders. There are huge ramifications to bitcoin, should it become successful: tax evasion, illegal trug trafficking, import/export bypass, etc. All things our government was asked to interfere with, by someone, for some reason, that are bypassed by someone's experiment. You may not support "the war on drugs", but the freely elected government of the US chose to take it on in response to various pressures. There are ethical implications to providing a mechanism to easily bypass this to others.
This topic always comes up, but the bottom line is: if it can be done, someone will do it. The world is not populated by exclusively ethical people, engineers are no different. Having a thing and then choosing not to use it probably causes less actual destruction than not having it at all.
Strictly speaking coders are not engineers. We use that term colloquially but I definitely got the impression that the article was speaking primarily of PEs.
Strictly speaking coders are not engineers.
They may or may not have an engineering degree/license but what coders are doing is most assuredly engineering. I'm an industrial engineer by training but I also do work almost daily that could be described as electrical engineering and sometimes mechanical engineering. Just because you don't have a document hanging on the wall saying you are an engineer doesn't mean you aren't one in real life.
We use that term colloquially but I definitely got the impression that the article was speaking primarily of PEs.
There are relatively few PEs compared to the number of engineers out there. Having a PE license doesn't mean you are better at engineering than someone who doesn't have one. The the sort of engineering I do it would have been a complete waste of my time for me to go get a PE license. It simply isn't necessary for many engineers. A PE is only required in certain circumstances and primarily for liability and statutory reasons. (Though it must be said that people with PE licenses tend to be good engineers in my experience)
I'm not sure he saw it that way:
In fact, he was later very much opposed to its use:
Oppenheimer was always a very conflicted individual.
Remember that he wasn't an elected politician or military commander, he was a civilian scientist who was tasked to develop the atomic bomb. It was never his job to decide if the bombs should be used and he knew that. In fact, he was very much motivated to both develop these weapons during WWII when he was terrified of the Nazi's developing a nuclear capability and using it on the Allies.
It's unfortunate that he felt bad about it later on. But guess what? A lot of people felt bad about many of their war actions later on. However, it was war. People tend to make different decisions when they are under an extreme amount of stress from a looming predator as compared to when they are relaxing in their vacation house.
Try to envision the time period. The Nazis had annihilated most of Europe and were gassing civilian Jews to death because they didn't think they were the perfect race. They were making lightshades out of Jewish skin for fun. The Japanese were waging a particularly vicious war on the Pacific. The US was stuck between these two insanities and tried to stay out of things for as long as possible. 12 MILLION people died in WWII. After the Nazis surrendered, the US had to start shipping war weary troops to the other side of the world for more fighting.
Was dropping two atomic bombs on Japan a nice thing to do? No. But I think the US was prepared to keep making nukes and dropping them on Japan until Japan surrendered rather than lose more US troops invading Japan.
We still haven't used up the Purple Heart medals we manufactured in the 1940s in anticipation of the casualties we expected in the invasion of Japan. In the context of WWII, the atomic bomb undoubtedly created a net savings of both allied and Japanese lives.
I was brought in to a government contractor's project as consultant during the Vietnam War. They were having severe problems with building their software system, and expected me to help them identify the root causes. For two weeks, they hemmed-and-hawed, trying to keep from telling me the true purpose of the system. Finally, when it was clear they couldn't understand the root problems themselves, they briefed me on what the system was ACTUALLY intended to accomplish. They did this on a Friday.
I was appalled that American citizens could dream up such an incredibly horrible intention: I can't say more, but the goal (in part) was to efficiently kill innocent civilians.
My choice was clear: I packed up, went to the airport, and bought a ticket home. On Monday, I was back at my regular desk. There was simply no way my conscience would allow me to optimize the schedule and effectiveness of such a project. There was never any repercussion, from anyone. I understand the contract was cancelled for non-performance several months later.
We who understand technology need to make value judgements: Do YOU want to write code that disadvantages fellow citizens? Do YOU want to create systems that transfer wealth from middle-class to rich folks? Do YOU want to write code that has secrets that could harm someone in the future buried inside? Do YOU want to make money by cheating ordinary citizens (think High-frequency "trading")? Do YOU want to see more systems, like NSA's, that violate the constitution, the law, and common decency?
I didn't, and I don't. Stand up for what you believe.
I find it extraordinary that you believe engineers have the choice to "shelve their own projects" in any significant frequency, given that nearly all of the perform work for hire, and don't own what they create. And don't say "patents" to me - my name is on a small number of patents, and I have fuck-all say in how those things are used; I didn't even get my dollar for the patent rights!
Some of us choose not to design weapons. It isn't theoretical.
Actually it is in a way. This strategy only works because you are protected by other engineers who design the weapons that protect you, whether you approve of those weapons or not.
There is nothing wrong with your moral choice but lets face facts. Pacifists can only exits in absolute isolation or where they are protected by friendly non-pacifists. In the real world there will be unfriendly non-pacifists who will subjugate, enslave or kill you. Regrettably this is the way some humans are wired.
In the book "Guns, germs, and steel" a warlike group of pacific islanders are mentioned. A subgroup colonizes a new island, loses contact with the original group and in isolation becomes pacifist. When contact is reestablished the subgroup is enslaved. This was done to blood relatives separated by only a small amount of time (in generational terms) with a common culture, language and religion.
I think George Orwell summed it up thusly: People sleep peaceably in their beds at night only because rough men stand ready to do violence on their behalf. In other words don't sit around in a nice 1st world country and pretend it got that way by accident.
So a physician or nurse should just go ahead and help the CIA torture some poor goat herder in Gitmo, because they're not 'uniquely clever' and someone else could do it? Sorry, doesn't fly.
If I don't feel something is ethical I won't do it. I don't care if the guy at the next desk over can do it or not, that has no bearing on my decision at all. **I** won't do it because I feel it's wrong.
"Think about how stupid the average person is. Now, realise that half of them are dumber than that." - George Carlin
You end up with the engineering equivalent of pharmacists refusing to sell contraceptives because they think that contraceptives are immoral.
It's amazing how everyone who says "so-and-so profession must consider the ethical implications of their work" always imagines that the person considering the ethics happens to be considering ethics that they agree with. They never think that they might consider unethical contraceptives, abortion, gay sex, miscegenation, etc.
We *want* apartment owners to say "If you use that apartment for sodomy, I am not responsible just because I rented you the apartment you used to do it in."
There are also a lot of areas where there isn't a clear ethical line.
You work in petrochem. Are you killing the planet or providing necessary energy?
You are an interior ballistics expert working for a swedish arms manufacturer (Hi Oerjan). The weapons can be used to stomp ruthless murderers and terrorists or it can be shot into crowded buses. Do you own all of that?
You build dams. They provide power but they flood habitat. Are you doing something ethically good or bad?
You develop car engines. Cars get people around. Your engine might save on greenhouse gases. But it furthers the automotive culture. Ehtical or not?
I have written software for: Large HR system, massive multi-user gaming platform, 3G/4G network policy enforcement, massive online gaming platform where many of the games were slots, poker, etc, major federal police force dispatch and mobile computing software, point of sale systems, network management systems, RCAF Tactical Navigation Trainer, etc.
I'd say at least half of the projects had some dodgy aspect somewhere in them (the online gambling one was the most distasteful but it was legal and thus within general public acceptance although I felt a bit slimy as did many developers). You can pick ethical issues out in most projects both in terms of the final product, the marketing, or the underlying model of operations that the invention supports.
Do engineers own all of that? That's a lot to expect. If none of us worked on projects in any way dodgy ethically or morally, society would not have most of what it has today. I'd image a good 70% of technology would not exist.
-- Mal: "Well they tell you: never hit a man with a closed fist. But it is, on occasion, hilarious."
it wasn't just Germany developing atomic weapons, Japan was as well
Among their plans was one to dirty-bomb the US west coast, San Francisco in particular.
They also understood the potential for a fission bomb and were working on it. This is why they recognized the A-bomb right away.
That's also why it was important to bomb Nagasaki the week after Hiroshima. This discredited the generals' claim that making these things was hard and the US couldn't have very many of them. Dropping two in as many weeks raised the spectre of one a week forever.
And the generals were right. As I understand it, the US had one in the pipeline and enough material for one or two more. Then there'd have been a several month pause, followed by about one a month.
This dearth of material, combined with the fact that the first one dropped was an untested design so failure WAS an option, was why there wasn't a demo to try to convince the Japanese to knuckle under without an actual bombing. The less-than-a-handful were too precious to be spent on other than actual targets.
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way