Slashdot Mirror
Will Unpredictable 'Franken-Algorithms' Have Deadly Consequences and Make Programmers Obsolete? (theguardian.com)
Zorro (Slashdot reader #15,797) summarizes a new article in the Guardian:
The death of a woman hit by a self-driving car highlights an unfolding technological crisis, as code piled on code creates "a universe no one fully understands."
"In some ways we've lost agency. When programs pass into code and code passes into algorithms and then algorithms start to create new algorithms, it gets farther and farther from human agency. Software is released into a code universe which no one can fully understand."
The author dubs these man-made monsters "franken-algos," since "After a time in the wild, we no longer know what they are: they have the potential to become erratic." Self-learning algorithms are already part of the "new all-machine phase" of Wall Street trading, leading to what science historian George Dyson believes are rules "where nobody knows what the rules are: the algorithms create their own rules -- you let them evolve the same way nature evolves organisms."
Where does it end? There's already a robotic sharpshooter policing the demilitarized zone between North and South Korea, and "swarms of coordinated, weaponized drones" already being developed by three different countries. The article suggests re-thinking our legal system to assign blame for any badly malfunctioning algorithms, noting that the Association for Computing Machinery recently updated its code of ethics "along the lines of medicine's Hippocratic oath, to instruct computing professionals to do no harm and consider the wider impacts of their work.... Solutions exist or can be found for most of the problems described here, but not without incentivizing big tech to place the health of society on a par with their bottom lines.
"More serious in the long term is growing conjecture that current programming methods are no longer fit for purpose given the size, complexity and interdependency of the algorithmic systems we increasingly rely on." Toby Walsh, a professor of artificial intelligence at the University of New South Wales, even says "We will eventually give up writing algorithms altogether... "because the machines will be able to do it far better than we ever could. Software engineering is in that sense perhaps a dying profession."
"In some ways we've lost agency. When programs pass into code and code passes into algorithms and then algorithms start to create new algorithms, it gets farther and farther from human agency. Software is released into a code universe which no one can fully understand."
The author dubs these man-made monsters "franken-algos," since "After a time in the wild, we no longer know what they are: they have the potential to become erratic." Self-learning algorithms are already part of the "new all-machine phase" of Wall Street trading, leading to what science historian George Dyson believes are rules "where nobody knows what the rules are: the algorithms create their own rules -- you let them evolve the same way nature evolves organisms."
Where does it end? There's already a robotic sharpshooter policing the demilitarized zone between North and South Korea, and "swarms of coordinated, weaponized drones" already being developed by three different countries. The article suggests re-thinking our legal system to assign blame for any badly malfunctioning algorithms, noting that the Association for Computing Machinery recently updated its code of ethics "along the lines of medicine's Hippocratic oath, to instruct computing professionals to do no harm and consider the wider impacts of their work.... Solutions exist or can be found for most of the problems described here, but not without incentivizing big tech to place the health of society on a par with their bottom lines.
"More serious in the long term is growing conjecture that current programming methods are no longer fit for purpose given the size, complexity and interdependency of the algorithmic systems we increasingly rely on." Toby Walsh, a professor of artificial intelligence at the University of New South Wales, even says "We will eventually give up writing algorithms altogether... "because the machines will be able to do it far better than we ever could. Software engineering is in that sense perhaps a dying profession."
Someone's got to fix it when it goes wrong.
Typical click bait.
What do you call the person who gives the franken-algorithm a specific list of instructions to accomplish your goal? A programmer.
Probably unavoidable. Our IRL universe is unpredictable, chaos and no-one fully understands it. Our own body mutates in undesirable ways, we suddenly get cancer. This is the way of the universe. As we get closer and closer to some sort of AI indistinguishable from real life I suspect this is an organic consequence.
Are we talking about a program, ... ...or the legal system?
I swear -- I read this, and I can't help but think about the legal matrix we live in.
For sure!
That's exactly what I think about our legal system. Who understands how the city works? It's a byzantine maze of interests, legislators, boards, bureaus, ... I think I could easily spend 8 years trying to understand How Seattle Works, and by the time I was done, it'd already have moved.
Can I just call them "cities?"
Self-learning algorithms... ...= people?
I don't see how that's all that different from, say, having the US military. What's the US military up to right now? I wouldn't be surprised if we're involved in some war or another war right now, and it just doesn't hit the headline news.
Now there's irony. "This horribly confused not-understandable block of code, shall be made accountable to this other confused not-understandable block of code." The ACM will putt to ... whatever legal associations exist. I suppose that they'll putt to the Constitution or something. And the Constitution will putt to God, or something.
Good luck with that...
"We will eventually give up on self-governance, because the government machine is able to do it far better than we ever could."
I don't know if this is good or bad for software engineering.
Might want to look up the facts of this case. Uber had disengaged the "stop if something's in front of you" aspects of the self-driving car's code, and the woman who was responsible for taking over in such a situation was watching a reality show on her phone at the time of the incident. Nobody even tried to stop this car.
There's a real threat these days of big companies and naive developers pushing machine learning as a panacea. Really it's more like a potent mutagen escaped from lab containment into your other work. People trying to obsolete expertise with poorly thought out and even more poorly tested applications of machine learning are definitely going to continue to cause deadly consequences in and around self-driving vehicles.
Someone has to program the initial version.
who better than Java programmers?
forget all those slow, bug-laden, inefficient, crappy languages like C/C++ or ASSembly. Java is where it's at.
Java is fast, and each iteration of a function or program makes it faster!
Forget stupid stuff like IEEE-754 compliance, ditch the hard stuff and focus on performance!
Java is where it's at.
We just need to hook up those monkey brains in the jars with some jumper cables and we'll never need programmers again.
Sig. Sig. Sputnik
I was about to post this exact sentiment, but with more swearing.
Will Unpredictable 'Franken-Algorithms' Have Deadly Consequences
Probably yes.
and Make Programmers Obsolete?
Almost certainly not.
We are creating algorithms where the result can not be explained in human terms. Nobody can truly understand why AlphaGo thinks a move is good, it's a neural network of weights we don't understand. It's about as useless as trying to get a chess grandmaster to articulate why a particular move is good, it's subtleties you can't record and put in a rule book. Which is fine for AlphaGo since the worst it'll do is lose a game. If it's Watson totally misdiagnosing your cancer or Waymo's car T-boning a school bus it matters a lot.
That is why I think developers will always be busy implementing guard rails. Like if you're trying to minimize humanity's environmental impact then the divide by zero solution is obviously superior. It's not a practically feasible solution in the real world though.
Live today, because you never know what tomorrow brings
I think "Franken-gate" refers to the Al Franken fiasco.
The answer is, "Yes, there will be deadly consequences and programmers will be obsolete." There's nothing you can do about it. Now make yourself a nice cold drink with some rum and fruit juice or something and go grill a piece of meat.
You can't change things, so you might as well enjoy your holiday weekend. Maybe things will look different on Tuesday, but for now, don't sweat it and go outside. It's nice outside, and summer's almost over.
You are welcome on my lawn.
Toll the Great Bell Once!
Pull the Lever forward to engage the Piston and Pump...
Toll the Great Bell Twice!
With push of Button fire the Engine And spark Turbine into life...
Toll the Great Bell Thrice!
Sing Praise to the God of All Machines
Pretty sure monkeys all prefer tabs. Are you really going to attach that hot mess to crucial systems?
"There is more worth loving than we have strength to love." - Brian Jay Stanley
We're doomed as doomed can be. - Ed Grimley
It's the best of all possible worlds. - Leibniz/Pangloss.
"National Security is the chief cause of national insecurity." - Celine's First Law
The referenced strategies only come into play to accomplish tasks that were pretty much out of reach of traditional programming. Essentially a last resort. For the problem set that has been feasible for programmers to tackle, it almost always remains the better way.
Further, it's really about bringing taming complex, chaotic, unstructured data into a structure so that programmers can address it. Generating these routines never speaks to how to apply the approach to solve a problem. Human's are required. As far as I have seen, once the image recognition or similar trained system is ready, it needs a programmer to actually do stuff with the resultant model.
XML is like violence. If it doesn't solve the problem, use more.
So-called 'AI' has been so over-hyped by development companies and their marketers (because they need to show ROI or get their heads chopped off), the news media doesn't have a clue how anything actually works and they're amplifying the hype, then entertainment media (TV, movies, even books) present these fantasy images of 'AI' technology that doesn't exist (and might never exist), and shockingly enough, people believe what they see hear and read. The long-term result of this, left unchecked, will be people actually believing that these 'algorithms' masquerading as Artificial Intelligence are capable of far more than they actually are, resulting in financial disasters, property damage, and loss of human life. Meanwhile the programmers that create these half-assed machines can't even tell you what's going on 'under the hood' when the thing's running, and can't really explain why it does what it does when it screws up. I for one will be glad when the current crop of so-called 'AI' they keep trotting out is shit-canned.
He was replaced by Tina Smith, we have 2 women senators from MN now
—employs a rarely seen strategy of “code reuse”.
“Don’t fly on payday.” — Wally
Which would free us up to vacation in our flying car and date Rosy the Robot.
Table-ized A.I.
We will eventually give up writing algorithms altogether...
Automated code generation. Automated test generation. Automated test coverage. Been there, done that. Twenty years ago. The people who say 'eventually' are the ones with a vested interest in selling meat sack coding labor to customers.
You think it's unpredictable and not to be trusted? Better not fly in a modern airplane.
Have gnu, will travel.
This is really just a continuation of the "software crisis", the discovery of human error when programming machines. Human error occurs on all levels of the software process, there is no "silver bullet" for fixing conflicting requirements.
We should not "incentivizing" big tech to pass the risks on to the public. QA is the boring stuff that involves regulation, redundancy and statistical modeling, of course the fancy Internet companies want to pass the responsibility for QA to end users with eternal Beta versions and EULAs filled with legalese.
Only if by Franken algorithm they mean some doomsday device with autonomous killing robots that drive humans into extinction. Programmers may some day become obsolete, but the death of programmers will be nothing so vague as the article.
Ridiculous statements delivered by an evidently programming-ignorant person aside, I have an anecdote which might help sensible people without a too deep understanding about all this to get the real implications of ideas on these lines (= absolute impossibility).
Although I have lots of experience in automated data understanding, numerical modelling, even in coming up with relatively complex algorithms taking care of a wide range of situations, my expertise on the image side of things is quite limited. More specifically, I haven't worked too much with the approaches which are more widely used to deal with these problems: neural networks. I mean that I haven't gone too depth into all these methodologies like properly understanding how (the complex versions of) their algorithms work or spending a relevant amount of time on tuning them up/coming up with good models. In fact, I have always tried to avoid this kind of trendy, theoretically-easy-to-use approaches which usually require you to spend a relevant amount of effort to understand everything properly (basically, new names and concepts to perform pretty much the same kind of calculations that many other methodologies can do) and rarely are comprehensive and adaptable enough, not more than older and more solid methodologies together with a good deal of personal experience.
Despite the previous paragraph, I am a very reasonable person always ready to update my assumptions when required. It is quite clear that all the work done in NN during the last years have allowed these systems to deliver a quite good performance under conditions like image analysis. So, I have been spending some time during the last weeks to gain a better insight into this sub-world, from both the algorithm development and parameter tuning sides. One of the first things I observed was that efficiency wasn't precisely a main concern for some of the most popular software packages; this is a relatively common situation nowadays, but here might become particularly problematic by bearing in mind the fact that some of these simulations might grow really complex/slow. Precisely improving efficiency is one of my strongest suits and that's why I decided to focus this first deep contact on that aspect.
It is still work in progress, so I will not talk too much about the specific details. The basic idea is that it is a popular, open-source package dealing with NN. It relies on well-documented theories and its algorithm, although pretty complex, is reasonably well commented too. All this together with my extensive experience on (efficiency) algorithm improvement (not too much on that specific programming language, although this issue isn't too relevant on account of my expertise and the myriad of available resources) seems to indicate that this should be a relatively easy endeavor. Logically, I had done a pre-analysis concluding that a relevant performance improvement was possible by even locating the parts to be modified. But the reality has been way different than expected...
Even after perfectly understanding the underlying theory, the main parts of the algorithms to be modified and having done a relevant amount of debugging/tests, the final solution isn't still quite there. The reason? Too complex, highly-abstracted code and (also complex) external dependencies. Similarly to what happens with many modern pieces of software, it was built to be used under certain conditions and together with certain resources; theoretically scalable and easily modifiable, but practically linked to a set of specific conditions. The whole approach didn't take efficiency into consideration since the start and, consequently, any relevant improvement on this front implies a big effort and essential modifications. Or, in other words, optimising that code by following its defining guidelines might be relatively easy, even automated in some ways; any other change can be very complex. Applying a set of more or less defined instructions is easy/automatisable, but creating those instructions or performing re
Custom Solvers 2.0 = Alvaro Carballo Garcia = varocarbas.
Can you imagine even the smartest speculative fiction AI dealing the typical "I don't know exactly what I want, but it's something like this: and I need the demo ready in a couple of days for a sales presentation."
If anything sets off the AI revolt, it would be dealing with today's mangers and marketing morons.
Question more what you read
The article basically states that humans are too stupid to understand complex algorithms... and robots probably say the same about human behaviour! Interestingly, most crashes of self-driven cars are down to old fashioned human error.
The answer is and was always 'NO'!
When a journalist writes about stuff he doesn't understand.
No, not unpredictable, unless you don't understand how AI and neural networks work!
Are there edge cases where input generates unexpected results? Absolutely! And that's no different from any kind of programming since the computer was invented.
For that matter, when we used to ride horses everywhere, there were edge cases that caused horse brains to freak out, such as gun shots nearby, or the sight of a wild animal. That didn't stop people from making safe use of horses for transportation, with accidents occurring only once in a great while.
The key was to learn to understand and anticipate the kinds of things horses would react to. Likewise, with time and testing, we will learn to anticipate and compensate for things that make AI "freak out." It's certainly not unpredictable.
Franken-Algorithms? Just wait until they are used by law enforcement and the court system which is already starting.