Slashdot Mirror
Why Engineers Must Consider the Ethical Implications of Their Work
An anonymous reader writes "An article by Abbas El-Zein at The Guardian explores the ethical responsibilities for engineers who create and maintain 'technologies of violence.' He says, 'Engineers who see themselves as builders of the shelter and infrastructure for human needs also use their expertise in order to destroy and kill more efficiently. When doctors or nurses use their knowledge of anatomy in order to torture or conduct medical experiments on helpless subjects, we are rightly outraged. Why doesn't society seem to apply the same standards to engineers? There is more than one answer to the question of course, but two points are especially pertinent: the common good we engineers see ourselves serving and our relationship to authority. ... Our ethics have become mostly technical: how to design properly, how to not cut corners, how to serve our clients well. We work hard to prevent failure of the systems we build, but only in relation to what these systems are meant to do, rather than the way they might actually be utilised, or whether they should have been built at all. We are not amoral, far from it; it's just that we have steered ourselves into a place where our morality has a smaller scope.'"
The existence of nuclear bombs have saved millions of lives over the last 60 years. Sounds pretty ethical to me.
While it is hard to draw exact parallels, society already holds engineers to similar standards to doctors. The outrage over doctors experimenting on helpless test subjects is pretty similar to, say, when engineers use live subjects for testing weapons.
We engineers got our historical start by building WAR engines.
In Rowanda the slaughter was committed using nothing more than machetes. However the people where whipped into a fervor by people on the radio inciting to violence. Marconi should have seen it coming. He had a responsibility.
More seriously, to paraphrase stephenson and others. human beings are at the top of the food chain because we are the most effective and fearsome killing machines currently known. We will find new ways to kill things, and each other regardless of the intended purpose of a tool, and how many safe guards are built into it. It is what we do and why we are where we are.
Technology is a tool, and a tool can be used a weapon. You should blame the one who wields the weapon. Do we blame Pasteur for biological warfare? I do not, but without him much of what we know about making bio-weapons would not exist.
You can study rockets to go to the moon, but eventually someone is going to shoot them at their neighbor.
You can study a way to get cheap energy for everyone, but eventually someone will make a bomb.
You can create a large forum for the people that is resistant to people stopping you from communication, but someone will eventually create a global spy system that watches everything you do.
It is unfortunate, but I would place the blame not on the person who makes the technology, but the one who decides how to use it. When we complain about doctors helping with torture, we are complaining about the ones there to extend the pain, not the ones who came up with ways keep people alive.
A lot of the engineers I've known who worked on military equipment do consider the ethical implications of their work. They feel they are helping protect our troops (see the beginning of Iron Man 1, where Tony Stark uses a similar justification), or something similar.
Other guys I know are just happy to have a job. Some people consider it unethical to work in the corporate world at all, so just because you consider something unethical, doesn't mean everyone considers it unethical. The NSA has the purpose of catching terrorists, which is a good goal. The reason we don't like them is because of the abuses, not because of their goal.
"First they came for the slanderers and i said nothing."
Saying an engineer shouldn't design a better weapon is like saying a doctor shouldn't treat a wounded soldier.
While it is hard to draw exact parallels, society already holds engineers to similar standards to doctors. The outrage over doctors experimenting on helpless test subjects is pretty similar to, say, when engineers use live subjects for testing weapons.
Yeah. The article's author is making a poor analogy. Blaming engineers would be more akin to blaming the scalpel designer for the doctor's experimentation. Its not the scalpel or the gun that is the problem, it is the mind and the intentions behind the hand holding the scalpel or gun. Both can be used for good or bad.
Short of WMD the issue is not as simple as the author suggests.
Every single person needs to do this. If you work in the weapons industry and don't feel bad about it, you are a psychopath. Simple.
So your view is that good guys produce no weapons, and bad guys produce lots of weapons...and it's that simple? What happens when the bad guys decide to be bad with their weapons by turning them on the unarmed, defenseless good guys?
Or, is it that some people should work in the weapons industry, but feel really really bad about it. And of course, those of us who have happy jobs are the better people, since we took the "high road" by forcing someone else to be bad. Perhaps we could have a lottery (Shirley Jackson's version), to decide who among us has to be bad so that our good lives can continue safely. Then we can all sit back and bask in the shiny, sunny warmth of just how good we all are, unlike those bad, bad people who make weapons that we can defend ourselves with...
For your security, this post has been encrypted with ROT-13, twice.
It's from the Bhagavad Gita, quoted in this context by Robert Oppenheimer. Nothing to do with Confucius.
Biological and chemical weapons were used, are used. It is just that now the world decries their use. If you look at WWI for instance. So it wasn't always the case that doctors were persecuted for using their knowledge for war. With engineers, creating conventional weapons, that is something accepted by the world. There is no moral outrage(on a large enough scale to matter)against a 500lbs bomb. When it comes to conventional weapons, everyone accepts the risks. We realize we need defense, so they are good. If someone uses them for offense, or evil instead, then that person is blamed, not the engineer. Should people that create steak knives also think about the ethical implications if someone gets stabbed with their knife? What if a car is used for violence? I realize these aren't the best examples, but my point is intended use. That is what matters. If an engineer creates a single weapon that will destroy the planet, then you will have your outrage. There is no point to such a weapon. Nuclear weapons are close to that, but they have been used to save lives as well.(est. that over 1 million Americans would have died invading Japan, along with millions of Japanese) There is nothing wrong with stepping back and saying, "Am I morally Ok with what I am creating?" But when it comes to conventional and nuclear weapons, if someone says no, then there will be hundreds to take your place. Military technology also trickles down to the general public and improves their lives as well.
"Cowardice in a race, as in an individual, is the unpardonable sin." --Teddy Roosevelt
Designing a missile system to kill lots of brown people on the other side of the world is not very ethically ambiguous. Thing is, there are plenty of technologies that are.
For example, DARPA has been doing lots of research on robots. They point out how self-driving cars can save lives, robots can find and defuse bombs and rescue victims, etc. But these technologies can be used for war just as easily.
There's no -1 for "I don't get it."
Playing the devil's advocate, when was the last time you got sued for malpractice for bugs in your code?
Support the EFF and Creative Commons. The war is coming, and they're supporting you...
I work in "the weapons industry". I sleep well knowing that my projects are being used so that my baby girls can sleep well at night. But then, I've seen first hand what motivated, evil people do when nobody stops them. But then, I've been to some nasty parts of Africa. At the risk of Godwining the thread, I strongly encourage you to go tour one of the concentration camps in Germany. Those people exist, and to paraphrase, "People sleep peaceably in their beds at night only because rough men stand ready to do violence on their behalf."-Orwell,
Probably because a Doctor helping torture someone is directly involved. The engineer's creation may be used in a way he/she never invisioned and with no involvement from him./her
Nor even with the knowledge of the ultimate use of his/her creation. Weapons can be used for good, its all about the intentions of those operating the weapons. For example those engineers who worked on weapons used to destroy the Third Reich, many of these were designed during peace time, ex M1 Garand Rifle of the US Army and Marine Corp. Short of WMD things are not so clear.
Playing the devil's advocate, when was the last time you got sued for malpractice for bugs in your code?
What is wrong with that concept, really?
I've no problem hearing Kaching! every time I get a blue screen.
If Pandora's box is destined to be opened, *I* want to be the one to open it.
in the US Military-Industrial Complex for most of the last 35 years.
If that doesn't match your ethics, that's OK.
Strictly speaking coders are not engineers. We use that term colloquially but I definitely got the impression that the article was speaking primarily of PEs.
Check out my lame java blog at www.javachopshop.com
What about the scientists and generals and governments that worked to design nuclear bombs? Didn't they have any ethics?
As someone going for a PhD in Civil and Environmental Engineering, my basic response to Mr. El-Zein is ...
Sod off.
Now stop thinking that the world needs to fix the Middle East or care about your "problems".
Oil is over. Nobody cares about you anymore.
-- Tigger warning: This post may contain tiggers! --
Submitter doesn't like humanity very much. He wishes there were laws, rules, regulations, and guide lines for everything. He wants to hold engineers responsible for their discoveries. He wants to judge each discovery as "good" or "bad", then reward or punish the engineers, scientists, and the craftsmen for whatever results.
Sad as it is, I prefer the world we have, in which men and woman exercise free will.
https://www.youtube.com/watch?v=YpCASVFyQoE
"Windows is like the faint smell of piss in a subway: it's there, and there's nothing you can do about it." - Charlie Br
Most engineers I've met who work in defense do not wake up every morning thinking about more efficient ways to kill women and children. They wake up, believing that what they do furthers the protection of their families, fellow citizens and their homeland. Doesn't matter if the engineer is an American, Chinese, Russian, Israeli, Iranian, etc., most pretty much think that what they do is going to create a better and safer world for their loved ones. The engineers at the NSA, and I would even argue their most senior leadership, likely believe that what they do is for the benefit of the United States. I think there's plenty of room to argue whether or not their assumptions and ethical standards are correct, but to imply that they're not thinking about this at all or simply creating superweapons for sport with no care about their end uses is overly simplistic.
When doctors or nurses use their knowledge of anatomy in order to torture or conduct medical experiments on helpless subjects, we are rightly outraged. Why doesn't society seem to apply the same standards to engineers?
When a doctor tortures a patient there is a direct cause and effect from the doctor's actions to the pain and suffering of the victim.
When an engineer designs a weapon, he's not actually causing the pain and suffering. Once you get away from "complete responsibility", the rest is easy:
1) If I don't do it, someone else will
2) I need to feed myself and my family
3) It'll only be used on the bad guys
4) It helps protect my country
5) It's the user's responsibility, not mine
6) The boss thinks it's a good idea
7) It has significant non-evil uses
8) No one will ever know it was me
For a concrete example, consider the Collateral Murder video from a couple of years back. Who was responsible for these deaths?
The helicopter pilots got the go-ahead from their commanders, the commanders [probably] got the go-ahead from intelligence services, the services made the correct decision based on the information they had, and the information was somehow "wrong".
Who's to blame for the collateral murder incident? By deftly distributing blame among many players, it changes from personal responsibility to "a failure of the system", or "a tragic accident".
For a second example, consider Bush's Iraq war: he was on TV stating that he had convincing evidence of WMDs in Iraq. A couple of years later it came out that the intelligence services had never said this and tried to convince the president of the opposite. Bush's response was: "We [the administration] didn't get the message". (Note the use of "we" in his statement.)
Who's responsible for the war? The President says he got bad intelligence, the intelligence services say they never gave bad intelligence. It's impossible to lay the blame on someone, it's a "failure of the system".
But don't worry, the problem is fixed - it'll never happen again.
(Epilogue: The Gulf oil spill was largely enabled by failures of the Minerals Management Service, who is responsible for overseeing the safety procedures of off-shore drilling. The problems were largely fixed by renaming the service to Bureau of Ocean Management. The problem is fixed, now we won't have any more disasters. Sorry about that...)
Strictly speaking coders are not engineers. We use that term colloquially but I definitely got the impression that the article was speaking primarily of PEs.
Very few engineers of any sort are PEs, at least in the US. Whether you are a classical mechanical/electrical engineer, or a coder, seems to be purely a subjective distinction. Some coders I know are definitely engineers, some are not (by my world view).
I think the article applies to any of the above, but the cited examples may be the domain of PEs - which as stated in the article - are not funded at all like corporate engineers, and thus have concerns more relevant to their funding model. Plenty of coders are involved in making missiles, and in fact some distruptive things (like bitcoin) were created by coders. There are huge ramifications to bitcoin, should it become successful: tax evasion, illegal trug trafficking, import/export bypass, etc. All things our government was asked to interfere with, by someone, for some reason, that are bypassed by someone's experiment. You may not support "the war on drugs", but the freely elected government of the US chose to take it on in response to various pressures. There are ethical implications to providing a mechanism to easily bypass this to others.
This topic always comes up, but the bottom line is: if it can be done, someone will do it. The world is not populated by exclusively ethical people, engineers are no different. Having a thing and then choosing not to use it probably causes less actual destruction than not having it at all.
Interestingly enough, according to canon this actually wasn't true. The Star Wars canon actually provides my favorite (fictional) example of an engineer who completely failed to consider what the project they were working on would be used for. There was a whole super-top-secret space lab, where a bunch of engineers worked on weapons of unimaginably massive destruction such as the Death Star... while being fed bull about how they'd be used for good (I recall the Death Star specifically was supposed to be used for mining. I totally don't remember what the Sun Crusher was supposedly going to be used for, but it was something hilariously *unlike* "destroy whole inhabited solar systems", I'm sure.)
Moderately off-topic, but I always thought that was one of the most brilliant pieces of Expanded Universe writing.
How about weapons of mass destruction?.
As others have pointed out, the items are merely 'tools' and the application the tool is where the morality lies.
Could you imagine a world where we routinely use nuclear weapons to relieve stresses in the Earth's crust and prevent large earthquakes and their devastating effects? What about stopping large oil spills quickly?
Using a nuclear weapon on an oil spill.
Unfortunately, many helpful adaptations of large scale explosives are not being utilized due to the political implications, but even a 'weapon of mass destruction' can be a useful tool to save lives (and money).
Strictly speaking coders are not engineers. We use that term colloquially but I definitely got the impression that the article was speaking primarily of PEs.
Strictly speaking coders are not engineers.
They may or may not have an engineering degree/license but what coders are doing is most assuredly engineering. I'm an industrial engineer by training but I also do work almost daily that could be described as electrical engineering and sometimes mechanical engineering. Just because you don't have a document hanging on the wall saying you are an engineer doesn't mean you aren't one in real life.
We use that term colloquially but I definitely got the impression that the article was speaking primarily of PEs.
There are relatively few PEs compared to the number of engineers out there. Having a PE license doesn't mean you are better at engineering than someone who doesn't have one. The the sort of engineering I do it would have been a complete waste of my time for me to go get a PE license. It simply isn't necessary for many engineers. A PE is only required in certain circumstances and primarily for liability and statutory reasons. (Though it must be said that people with PE licenses tend to be good engineers in my experience)
Choosing to shelve a project for an ethical principle, for the reason I stated in my last sentence, makes sense only if you think you are uniquely clever and no one else will think of it.
That strikes me as profoundly arrogant. Perhaps a few people in the world at any time are justified in so thinking, but there are plenty of historical examples that suggest even really clever things can be conceived of independently.
Chemical/biological weapons for example.
The laws of physics shall not have a LICENSE.TXT file.
I was brought in to a government contractor's project as consultant during the Vietnam War. They were having severe problems with building their software system, and expected me to help them identify the root causes. For two weeks, they hemmed-and-hawed, trying to keep from telling me the true purpose of the system. Finally, when it was clear they couldn't understand the root problems themselves, they briefed me on what the system was ACTUALLY intended to accomplish. They did this on a Friday.
I was appalled that American citizens could dream up such an incredibly horrible intention: I can't say more, but the goal (in part) was to efficiently kill innocent civilians.
My choice was clear: I packed up, went to the airport, and bought a ticket home. On Monday, I was back at my regular desk. There was simply no way my conscience would allow me to optimize the schedule and effectiveness of such a project. There was never any repercussion, from anyone. I understand the contract was cancelled for non-performance several months later.
We who understand technology need to make value judgements: Do YOU want to write code that disadvantages fellow citizens? Do YOU want to create systems that transfer wealth from middle-class to rich folks? Do YOU want to write code that has secrets that could harm someone in the future buried inside? Do YOU want to make money by cheating ordinary citizens (think High-frequency "trading")? Do YOU want to see more systems, like NSA's, that violate the constitution, the law, and common decency?
I didn't, and I don't. Stand up for what you believe.
Short of WMD the issue is not as simple as the author suggests.
Really? Explain to me what purpose an M1 tank or an F22 fighter has besides killing people? What humanitarian purpose do land mines serve? Assault rifles? (target shooting? don't make me laugh) Hand grenades? Let's not pretend that the engineers working on these products have no idea what they will be used for. Plausible deniability does not apply to a lot of weapons.
There are many technologies where the line between ethical and not-so-much is fuzzy but you hardly have to go to WMDs to get there.
Does a teller at JP Morgan think about the ethical implications of their work?
Does a cashier at WalMart think about the ethical implications of their work?
In both examples a person that is in no way responsible for the overall direction of the company is facilitating the daily operations of the company that will commit unethical acts.
We cant just ask Engineers to sacrifice their careers because some whiny Journalist/Engineer is having a moral crisis. Our entire society is unethical. We buy clothes that were made by people working for less than $100 a month, living in concrete rooms smaller than most jail cells, who were forced into that labor by their parents. We shop at companies who are lobbying to oppress workers rights. We use electronics made by children and people who would rather kill themselves than continue working at Foxconn.
Get off of your fucking high horse and stop acting like an Engineer is any different than a Banker or a CEO or a cashier. We are all players in the same fuckedup game.
I find it extraordinary that you believe engineers have the choice to "shelve their own projects" in any significant frequency, given that nearly all of the perform work for hire, and don't own what they create. And don't say "patents" to me - my name is on a small number of patents, and I have fuck-all say in how those things are used; I didn't even get my dollar for the patent rights!
Some of us choose not to design weapons. It isn't theoretical.
Actually it is in a way. This strategy only works because you are protected by other engineers who design the weapons that protect you, whether you approve of those weapons or not.
There is nothing wrong with your moral choice but lets face facts. Pacifists can only exits in absolute isolation or where they are protected by friendly non-pacifists. In the real world there will be unfriendly non-pacifists who will subjugate, enslave or kill you. Regrettably this is the way some humans are wired.
In the book "Guns, germs, and steel" a warlike group of pacific islanders are mentioned. A subgroup colonizes a new island, loses contact with the original group and in isolation becomes pacifist. When contact is reestablished the subgroup is enslaved. This was done to blood relatives separated by only a small amount of time (in generational terms) with a common culture, language and religion.
I think George Orwell summed it up thusly: People sleep peaceably in their beds at night only because rough men stand ready to do violence on their behalf. In other words don't sit around in a nice 1st world country and pretend it got that way by accident.
So a physician or nurse should just go ahead and help the CIA torture some poor goat herder in Gitmo, because they're not 'uniquely clever' and someone else could do it? Sorry, doesn't fly.
If I don't feel something is ethical I won't do it. I don't care if the guy at the next desk over can do it or not, that has no bearing on my decision at all. **I** won't do it because I feel it's wrong.
"Think about how stupid the average person is. Now, realise that half of them are dumber than that." - George Carlin
As an engineering consultant, I do in fact have a copyright on the design documents I produce. If I really wanted to, I could sue a former client if they attempt to continue using my designs. They are licensed, not sold.
And if I shelve a project (or more appropriately, walk off a job - which I've done) then all that really happens is I'm no longer legally responsible for any shit that goes down from that point on. I can refuse to deliver finished project documents, oversee the project construction, and address issues that arise during and after installation.
And just because your name is on a patent doesn't mean you own the rights. Must be one of those situations where the stipulation of your employment at a company means anything you create belongs to them? If so, then that's what you agreed to and that's what you get.
=Smidge=
Kind of have to agree on this one. Look at the history of war. Look at how many fewer deaths we've had from war after WWII. It's much smaller than what we had before WWII. For instance, Number of deaths in Vietnam vary depending on who you ask, but the number is for sure less than 1 million. Compare that to WWII, where 50-80 million people died.WWI was around 37 million. Why such a sudden decline all of a sudden? hard to say it was all because of the bomb, but it's definitely a deterrent. Maybe we are just better at stopping conflicts before they get out of hand.
Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
The big difference with medical knowledge being used to torture versus, say, design of a weapon, is that weapons have moral use. Defense is inherently moral, and technology makes that safer and better. Technology used to kill is the same as a sword being used to kill, is the same as a rock- if the man behind the tool is working towards good, defending his nation / family / self, then the action, and the weapon is moral.
Torture, on the other hand, is always wrong. But that doesn't make scalpels evil, or handguns, or rockets.
You end up with the engineering equivalent of pharmacists refusing to sell contraceptives because they think that contraceptives are immoral.
It's amazing how everyone who says "so-and-so profession must consider the ethical implications of their work" always imagines that the person considering the ethics happens to be considering ethics that they agree with. They never think that they might consider unethical contraceptives, abortion, gay sex, miscegenation, etc.
We *want* apartment owners to say "If you use that apartment for sodomy, I am not responsible just because I rented you the apartment you used to do it in."
...who cares vere zay come down.
Zat's not my department,"
says Werner Von Braun.
Is it just my observation, or are there way too many stupid people in the world?
Vietnam is far from the biggest war after WWII - just the best known to Americans. Try the recent second Congo war, which killed over five million people.
Yet most people here haven't even heard about it.
I personally don't think that nuclear weapons have deterred much. USA, Soviet/Russia, China, United Kingdom and France all have been waging wars after WWII. You'd think that those would be the countries fighting the least post-WWII if that were the case.
I think it's more that the huge big wars don't happen all that often in the first place.
I suspect that another key problem with ethics is that many evil things are more fun to do, think about, and tell people than the boring things. If you tell people that you are an HVAC engineer they will think, "BORING!!!" yet the reality is that you make many poeples' lives more comfortable, and with good designs, save energy, and create healthy environments. But if you tell people you are the inventor of the hellfire missile or build nuclear bombs then they will go, "oooooooh"
I'm not sure there is any limit to the "coolness" of what is effectively sociopathic behavior; If you tell tech people that you are building a military robot that is designed to hunt and then jump onto the faces of the Taliban (alien style) and stuff a GPS tracker/grenade down their throat that forces them to surrender or be blown up from the inside then you will make headline news. If you develop a way to make cheap home wiring that conducts better than silver you are back to boring.
The above evil will get a few people to gasp in horror but most people will want to know more.
Now normally the defense industry goes through spasms of peace and the engineers face huge layoffs. But this time around the US is effectively doing a War on Fear which will pretty well never end. So if you can invent a tool for annihilating the boogy man then you will remain solidly employed. I you are inventing somethings solar that reduces fossil fuel use then your employment will be fitful at best.
Engineers created RADAR to make it easier to find and kill people. That work wouldn't have been done under the ethical guidelines outlined by some. I on the other hand am fine with something that makes it easier for the other guy to get killed before I do. I'm also fine with using that technology to get from place to place in peace time.
It is very much true for any kind of technical innovation, not only weapons.
An entrepreneur, who thinks of a new idea and rushes to set up a startup company, doesn't stop to think on the full implications of uses for his future product.
In most cases, I would think, engineers are very much not the kind of people who philosophize on the merits and dangers of technology... They're much to busy solving technical problems.
Playing the devil's advocate, when was the last time you got sued for malpractice for bugs in your code?
What is wrong with that concept, really?
I've no problem hearing Kaching! every time I get a blue screen.
Just wait until your code is driving a car and crashes. I predict the lawyer scum will keep afloat.
Gosh, thanks. That must be why the other ships call me Meatfucker -- GCU Grey Area (Eccentric)
And did those rough men design and build their own weapons?
There are also a lot of areas where there isn't a clear ethical line.
You work in petrochem. Are you killing the planet or providing necessary energy?
You are an interior ballistics expert working for a swedish arms manufacturer (Hi Oerjan). The weapons can be used to stomp ruthless murderers and terrorists or it can be shot into crowded buses. Do you own all of that?
You build dams. They provide power but they flood habitat. Are you doing something ethically good or bad?
You develop car engines. Cars get people around. Your engine might save on greenhouse gases. But it furthers the automotive culture. Ehtical or not?
I have written software for: Large HR system, massive multi-user gaming platform, 3G/4G network policy enforcement, massive online gaming platform where many of the games were slots, poker, etc, major federal police force dispatch and mobile computing software, point of sale systems, network management systems, RCAF Tactical Navigation Trainer, etc.
I'd say at least half of the projects had some dodgy aspect somewhere in them (the online gambling one was the most distasteful but it was legal and thus within general public acceptance although I felt a bit slimy as did many developers). You can pick ethical issues out in most projects both in terms of the final product, the marketing, or the underlying model of operations that the invention supports.
Do engineers own all of that? That's a lot to expect. If none of us worked on projects in any way dodgy ethically or morally, society would not have most of what it has today. I'd image a good 70% of technology would not exist.
-- Mal: "Well they tell you: never hit a man with a closed fist. But it is, on occasion, hilarious."
For now they have. What happens when they get used? Don't tell me you've fallen for that MAD survivor's bias nonsense. The only reason the Cold War didn't destroy human civilization is luck. More than once it came down to the right man in the right place making the right call. We won't always be so lucky.
And remember: we need to get lucky EVERY time. If we get unlucky just once, it's all over. Forever. All the easily accessible petroleum is gone. If we get unlucky, there will be no rebuilding of civilization. Our species will die.