Slashdot Mirror
Will Unpredictable 'Franken-Algorithms' Have Deadly Consequences and Make Programmers Obsolete? (theguardian.com)
Zorro (Slashdot reader #15,797) summarizes a new article in the Guardian:
The death of a woman hit by a self-driving car highlights an unfolding technological crisis, as code piled on code creates "a universe no one fully understands."
"In some ways we've lost agency. When programs pass into code and code passes into algorithms and then algorithms start to create new algorithms, it gets farther and farther from human agency. Software is released into a code universe which no one can fully understand."
The author dubs these man-made monsters "franken-algos," since "After a time in the wild, we no longer know what they are: they have the potential to become erratic." Self-learning algorithms are already part of the "new all-machine phase" of Wall Street trading, leading to what science historian George Dyson believes are rules "where nobody knows what the rules are: the algorithms create their own rules -- you let them evolve the same way nature evolves organisms."
Where does it end? There's already a robotic sharpshooter policing the demilitarized zone between North and South Korea, and "swarms of coordinated, weaponized drones" already being developed by three different countries. The article suggests re-thinking our legal system to assign blame for any badly malfunctioning algorithms, noting that the Association for Computing Machinery recently updated its code of ethics "along the lines of medicine's Hippocratic oath, to instruct computing professionals to do no harm and consider the wider impacts of their work.... Solutions exist or can be found for most of the problems described here, but not without incentivizing big tech to place the health of society on a par with their bottom lines.
"More serious in the long term is growing conjecture that current programming methods are no longer fit for purpose given the size, complexity and interdependency of the algorithmic systems we increasingly rely on." Toby Walsh, a professor of artificial intelligence at the University of New South Wales, even says "We will eventually give up writing algorithms altogether... "because the machines will be able to do it far better than we ever could. Software engineering is in that sense perhaps a dying profession."
"In some ways we've lost agency. When programs pass into code and code passes into algorithms and then algorithms start to create new algorithms, it gets farther and farther from human agency. Software is released into a code universe which no one can fully understand."
The author dubs these man-made monsters "franken-algos," since "After a time in the wild, we no longer know what they are: they have the potential to become erratic." Self-learning algorithms are already part of the "new all-machine phase" of Wall Street trading, leading to what science historian George Dyson believes are rules "where nobody knows what the rules are: the algorithms create their own rules -- you let them evolve the same way nature evolves organisms."
Where does it end? There's already a robotic sharpshooter policing the demilitarized zone between North and South Korea, and "swarms of coordinated, weaponized drones" already being developed by three different countries. The article suggests re-thinking our legal system to assign blame for any badly malfunctioning algorithms, noting that the Association for Computing Machinery recently updated its code of ethics "along the lines of medicine's Hippocratic oath, to instruct computing professionals to do no harm and consider the wider impacts of their work.... Solutions exist or can be found for most of the problems described here, but not without incentivizing big tech to place the health of society on a par with their bottom lines.
"More serious in the long term is growing conjecture that current programming methods are no longer fit for purpose given the size, complexity and interdependency of the algorithmic systems we increasingly rely on." Toby Walsh, a professor of artificial intelligence at the University of New South Wales, even says "We will eventually give up writing algorithms altogether... "because the machines will be able to do it far better than we ever could. Software engineering is in that sense perhaps a dying profession."
What do you call the person who gives the franken-algorithm a specific list of instructions to accomplish your goal? A programmer.
Might want to look up the facts of this case. Uber had disengaged the "stop if something's in front of you" aspects of the self-driving car's code, and the woman who was responsible for taking over in such a situation was watching a reality show on her phone at the time of the incident. Nobody even tried to stop this car.
There's a real threat these days of big companies and naive developers pushing machine learning as a panacea. Really it's more like a potent mutagen escaped from lab containment into your other work. People trying to obsolete expertise with poorly thought out and even more poorly tested applications of machine learning are definitely going to continue to cause deadly consequences in and around self-driving vehicles.
Someone has to program the initial version.
who better than Java programmers?
forget all those slow, bug-laden, inefficient, crappy languages like C/C++ or ASSembly. Java is where it's at.
Java is fast, and each iteration of a function or program makes it faster!
Forget stupid stuff like IEEE-754 compliance, ditch the hard stuff and focus on performance!
Java is where it's at.
Will Unpredictable 'Franken-Algorithms' Have Deadly Consequences
Probably yes.
and Make Programmers Obsolete?
Almost certainly not.
We are creating algorithms where the result can not be explained in human terms. Nobody can truly understand why AlphaGo thinks a move is good, it's a neural network of weights we don't understand. It's about as useless as trying to get a chess grandmaster to articulate why a particular move is good, it's subtleties you can't record and put in a rule book. Which is fine for AlphaGo since the worst it'll do is lose a game. If it's Watson totally misdiagnosing your cancer or Waymo's car T-boning a school bus it matters a lot.
That is why I think developers will always be busy implementing guard rails. Like if you're trying to minimize humanity's environmental impact then the divide by zero solution is obviously superior. It's not a practically feasible solution in the real world though.
Live today, because you never know what tomorrow brings
So-called 'AI' has been so over-hyped by development companies and their marketers (because they need to show ROI or get their heads chopped off), the news media doesn't have a clue how anything actually works and they're amplifying the hype, then entertainment media (TV, movies, even books) present these fantasy images of 'AI' technology that doesn't exist (and might never exist), and shockingly enough, people believe what they see hear and read. The long-term result of this, left unchecked, will be people actually believing that these 'algorithms' masquerading as Artificial Intelligence are capable of far more than they actually are, resulting in financial disasters, property damage, and loss of human life. Meanwhile the programmers that create these half-assed machines can't even tell you what's going on 'under the hood' when the thing's running, and can't really explain why it does what it does when it screws up. I for one will be glad when the current crop of so-called 'AI' they keep trotting out is shit-canned.