Slashdot Mirror
Will Unpredictable 'Franken-Algorithms' Have Deadly Consequences and Make Programmers Obsolete? (theguardian.com)
Zorro (Slashdot reader #15,797) summarizes a new article in the Guardian:
The death of a woman hit by a self-driving car highlights an unfolding technological crisis, as code piled on code creates "a universe no one fully understands."
"In some ways we've lost agency. When programs pass into code and code passes into algorithms and then algorithms start to create new algorithms, it gets farther and farther from human agency. Software is released into a code universe which no one can fully understand."
The author dubs these man-made monsters "franken-algos," since "After a time in the wild, we no longer know what they are: they have the potential to become erratic." Self-learning algorithms are already part of the "new all-machine phase" of Wall Street trading, leading to what science historian George Dyson believes are rules "where nobody knows what the rules are: the algorithms create their own rules -- you let them evolve the same way nature evolves organisms."
Where does it end? There's already a robotic sharpshooter policing the demilitarized zone between North and South Korea, and "swarms of coordinated, weaponized drones" already being developed by three different countries. The article suggests re-thinking our legal system to assign blame for any badly malfunctioning algorithms, noting that the Association for Computing Machinery recently updated its code of ethics "along the lines of medicine's Hippocratic oath, to instruct computing professionals to do no harm and consider the wider impacts of their work.... Solutions exist or can be found for most of the problems described here, but not without incentivizing big tech to place the health of society on a par with their bottom lines.
"More serious in the long term is growing conjecture that current programming methods are no longer fit for purpose given the size, complexity and interdependency of the algorithmic systems we increasingly rely on." Toby Walsh, a professor of artificial intelligence at the University of New South Wales, even says "We will eventually give up writing algorithms altogether... "because the machines will be able to do it far better than we ever could. Software engineering is in that sense perhaps a dying profession."
"In some ways we've lost agency. When programs pass into code and code passes into algorithms and then algorithms start to create new algorithms, it gets farther and farther from human agency. Software is released into a code universe which no one can fully understand."
The author dubs these man-made monsters "franken-algos," since "After a time in the wild, we no longer know what they are: they have the potential to become erratic." Self-learning algorithms are already part of the "new all-machine phase" of Wall Street trading, leading to what science historian George Dyson believes are rules "where nobody knows what the rules are: the algorithms create their own rules -- you let them evolve the same way nature evolves organisms."
Where does it end? There's already a robotic sharpshooter policing the demilitarized zone between North and South Korea, and "swarms of coordinated, weaponized drones" already being developed by three different countries. The article suggests re-thinking our legal system to assign blame for any badly malfunctioning algorithms, noting that the Association for Computing Machinery recently updated its code of ethics "along the lines of medicine's Hippocratic oath, to instruct computing professionals to do no harm and consider the wider impacts of their work.... Solutions exist or can be found for most of the problems described here, but not without incentivizing big tech to place the health of society on a par with their bottom lines.
"More serious in the long term is growing conjecture that current programming methods are no longer fit for purpose given the size, complexity and interdependency of the algorithmic systems we increasingly rely on." Toby Walsh, a professor of artificial intelligence at the University of New South Wales, even says "We will eventually give up writing algorithms altogether... "because the machines will be able to do it far better than we ever could. Software engineering is in that sense perhaps a dying profession."
Someone's got to fix it when it goes wrong.
Typical click bait.
What do you call the person who gives the franken-algorithm a specific list of instructions to accomplish your goal? A programmer.
Might want to look up the facts of this case. Uber had disengaged the "stop if something's in front of you" aspects of the self-driving car's code, and the woman who was responsible for taking over in such a situation was watching a reality show on her phone at the time of the incident. Nobody even tried to stop this car.
There's a real threat these days of big companies and naive developers pushing machine learning as a panacea. Really it's more like a potent mutagen escaped from lab containment into your other work. People trying to obsolete expertise with poorly thought out and even more poorly tested applications of machine learning are definitely going to continue to cause deadly consequences in and around self-driving vehicles.
Someone has to program the initial version.
who better than Java programmers?
forget all those slow, bug-laden, inefficient, crappy languages like C/C++ or ASSembly. Java is where it's at.
Java is fast, and each iteration of a function or program makes it faster!
Forget stupid stuff like IEEE-754 compliance, ditch the hard stuff and focus on performance!
Java is where it's at.
Will Unpredictable 'Franken-Algorithms' Have Deadly Consequences
Probably yes.
and Make Programmers Obsolete?
Almost certainly not.
We are creating algorithms where the result can not be explained in human terms. Nobody can truly understand why AlphaGo thinks a move is good, it's a neural network of weights we don't understand. It's about as useless as trying to get a chess grandmaster to articulate why a particular move is good, it's subtleties you can't record and put in a rule book. Which is fine for AlphaGo since the worst it'll do is lose a game. If it's Watson totally misdiagnosing your cancer or Waymo's car T-boning a school bus it matters a lot.
That is why I think developers will always be busy implementing guard rails. Like if you're trying to minimize humanity's environmental impact then the divide by zero solution is obviously superior. It's not a practically feasible solution in the real world though.
Live today, because you never know what tomorrow brings
This seems right that complexity like this is likely unavoidable. Evolution has a fairly reliable way to deal with this. Complexity is allowed to grow unchecked but every generation of a species is required to be able to survive and reproduce or it goes extinct. That seems to be pretty much what is happening with software.
There seems to be a dream that there is a simple theory of everything that humans can understand and that will allow us to do what we want in our complex world and still understand it. But it seems not to be founded in reality. The biological human mind can't and won't understand many of the complex systems we rely on to survive. Software complexity adds another entry to an already long list of complex systems we rely on but don't understand including ecosystems, brain function, microbiome, etc.
So-called 'AI' has been so over-hyped by development companies and their marketers (because they need to show ROI or get their heads chopped off), the news media doesn't have a clue how anything actually works and they're amplifying the hype, then entertainment media (TV, movies, even books) present these fantasy images of 'AI' technology that doesn't exist (and might never exist), and shockingly enough, people believe what they see hear and read. The long-term result of this, left unchecked, will be people actually believing that these 'algorithms' masquerading as Artificial Intelligence are capable of far more than they actually are, resulting in financial disasters, property damage, and loss of human life. Meanwhile the programmers that create these half-assed machines can't even tell you what's going on 'under the hood' when the thing's running, and can't really explain why it does what it does when it screws up. I for one will be glad when the current crop of so-called 'AI' they keep trotting out is shit-canned.
We will eventually give up writing algorithms altogether...
Automated code generation. Automated test generation. Automated test coverage. Been there, done that. Twenty years ago. The people who say 'eventually' are the ones with a vested interest in selling meat sack coding labor to customers.
You think it's unpredictable and not to be trusted? Better not fly in a modern airplane.
Have gnu, will travel.
This is really just a continuation of the "software crisis", the discovery of human error when programming machines. Human error occurs on all levels of the software process, there is no "silver bullet" for fixing conflicting requirements.
We should not "incentivizing" big tech to pass the risks on to the public. QA is the boring stuff that involves regulation, redundancy and statistical modeling, of course the fancy Internet companies want to pass the responsibility for QA to end users with eternal Beta versions and EULAs filled with legalese.
Ridiculous statements delivered by an evidently programming-ignorant person aside, I have an anecdote which might help sensible people without a too deep understanding about all this to get the real implications of ideas on these lines (= absolute impossibility).
Although I have lots of experience in automated data understanding, numerical modelling, even in coming up with relatively complex algorithms taking care of a wide range of situations, my expertise on the image side of things is quite limited. More specifically, I haven't worked too much with the approaches which are more widely used to deal with these problems: neural networks. I mean that I haven't gone too depth into all these methodologies like properly understanding how (the complex versions of) their algorithms work or spending a relevant amount of time on tuning them up/coming up with good models. In fact, I have always tried to avoid this kind of trendy, theoretically-easy-to-use approaches which usually require you to spend a relevant amount of effort to understand everything properly (basically, new names and concepts to perform pretty much the same kind of calculations that many other methodologies can do) and rarely are comprehensive and adaptable enough, not more than older and more solid methodologies together with a good deal of personal experience.
Despite the previous paragraph, I am a very reasonable person always ready to update my assumptions when required. It is quite clear that all the work done in NN during the last years have allowed these systems to deliver a quite good performance under conditions like image analysis. So, I have been spending some time during the last weeks to gain a better insight into this sub-world, from both the algorithm development and parameter tuning sides. One of the first things I observed was that efficiency wasn't precisely a main concern for some of the most popular software packages; this is a relatively common situation nowadays, but here might become particularly problematic by bearing in mind the fact that some of these simulations might grow really complex/slow. Precisely improving efficiency is one of my strongest suits and that's why I decided to focus this first deep contact on that aspect.
It is still work in progress, so I will not talk too much about the specific details. The basic idea is that it is a popular, open-source package dealing with NN. It relies on well-documented theories and its algorithm, although pretty complex, is reasonably well commented too. All this together with my extensive experience on (efficiency) algorithm improvement (not too much on that specific programming language, although this issue isn't too relevant on account of my expertise and the myriad of available resources) seems to indicate that this should be a relatively easy endeavor. Logically, I had done a pre-analysis concluding that a relevant performance improvement was possible by even locating the parts to be modified. But the reality has been way different than expected...
Even after perfectly understanding the underlying theory, the main parts of the algorithms to be modified and having done a relevant amount of debugging/tests, the final solution isn't still quite there. The reason? Too complex, highly-abstracted code and (also complex) external dependencies. Similarly to what happens with many modern pieces of software, it was built to be used under certain conditions and together with certain resources; theoretically scalable and easily modifiable, but practically linked to a set of specific conditions. The whole approach didn't take efficiency into consideration since the start and, consequently, any relevant improvement on this front implies a big effort and essential modifications. Or, in other words, optimising that code by following its defining guidelines might be relatively easy, even automated in some ways; any other change can be very complex. Applying a set of more or less defined instructions is easy/automatisable, but creating those instructions or performing re
Custom Solvers 2.0 = Alvaro Carballo Garcia = varocarbas.