Slashdot Mirror
Hilarious (and Terrifying?) Ways Algorithms Have Outsmarted Their Creators (popularmechanics.com)
"Robot brains will challenge the fundamental assumptions of how we humans do things," argues Popular Mechanics, noting that age-old truism "that computers will always do literally, exactly what you tell them to."
A paper recently published to ArXiv highlights just a handful of incredible and slightly terrifying ways that algorithms think... An AI project which pit programs against each other in games of five-in-a-row Tic-Tac-Toe on an infinitely expansive board surfaced the extremely successful method of requesting moves involving extremely long memory addresses which would crash the opponent's computer and award a win by default...
These amusing stories also reflect the potential for evolutionary algorithms or neural networks to stumble upon solutions to problems that are outside-the-box in dangerous ways. They're a funnier version of the classic AI nightmare where computers tasked with creating peace on Earth decide the most efficient solution is to exterminate the human race. The solution, the paper suggests, is not fear but careful experimentation.
The paper (available as a free download) contains 27 anecdotes, which its authors describe as a "crowd-sourced product of researchers in the fields of artificial life and evolutionary computation. Popular Science adds that "the most amusing examples are clearly ones where algorithms abused bugs in their simulations -- essentially glitches in the Matrix that gave them superpowers."
These amusing stories also reflect the potential for evolutionary algorithms or neural networks to stumble upon solutions to problems that are outside-the-box in dangerous ways. They're a funnier version of the classic AI nightmare where computers tasked with creating peace on Earth decide the most efficient solution is to exterminate the human race. The solution, the paper suggests, is not fear but careful experimentation.
The paper (available as a free download) contains 27 anecdotes, which its authors describe as a "crowd-sourced product of researchers in the fields of artificial life and evolutionary computation. Popular Science adds that "the most amusing examples are clearly ones where algorithms abused bugs in their simulations -- essentially glitches in the Matrix that gave them superpowers."
Is that their creators aren't very bright. . .
Just say arxiv. This is Slashdot, not Gizmodo (the gender studies magazine).
The extended phenotype called religion successfully coerces its hosts into going extinct.for the fun and profit of its televangelists.
Did you know beau is going to wake up in 12 hours and not remember making this post?
These aren't really that terrifying. We just don't have the GPU power for re-enforcement learning like this to search for really out there solutions to problems at the moment. But they can produce really funny stories like this.
My favorite story is of a bot given the task of moving itself through a maze or somesuch (important part incoming). Anyway, the programmer decided the more time the bot spent away from the center of the maze the worse points it would get (it's trying to optimize for points here). But instead of going towards the center of the maze as fast as possible to maximize points it just couldn't figure out how to get through. So it sent itself off the virtual edge of the simulation area, ending the run and minimizing it's negative score as best as possible. By accident someone created a suicidal bot, yay!
And that is really the extend of "Deep Re-enforcement Learning" aka AI that teaches itself to do things today. Sometimes, like with Alpha Go, it works. But a lot of the time it does something stupid.
And most of the situations described in the reference article describe poorly framed problems. I understand that it is supposed to be a jokey, light, non-serious, read. However it illustrates the problem with people asking the wrong question, or making incorrect assumptions.
Many years ago the multi-billion $$$$ utility company I was working for had a team from [ name removed to protect the stupid ] a well-known consultancy outfit. One of their conclusions was that some of our servers were running with too much idle time - under utilised in their opinion. All they had done was collect %idle data from sar (Unix systems from Sun, IBM and HP). and their junior idiot looked at that and decided it was a "problem"
When I was asked about this by the CIO and the "consultants", my response was that I could easily increase the utilitisation figure to whatever the CIO desired, or that the consultants recommended - how high would he like it to be? Since he knew me, and saw the smile, he saw the trap. I explained that "idle" time and user response time were tightly linked: that reducing one would increase the other. This was news to the "consultants" once I explained the maths and Queuing Theory behind it.
politicians are like babies' nappies: they should both be changed regularly and for the same reasons
Before AI we humans keep on having this 'we are the best' self-appraisal, until the arrival of AI, and suddenly we discover that we are not that good, that there are a whole lot of 'out-of-the-box' solution to any given problem that we human couldn't think of
Hence, the biggest reward we could obtain from AI are the 'out-of-the-box' solutions which we human should adopt (and proceed from there)
If an evolutionary algorithm is pitted against real life, and 'outsmarts' it, that's one measure of evolutionary progress. The real issue is the same as in 'teaching to the test', or even the 'kobayashi maru solution': the metrics are gamed once the one being tested realizes what they are, and then the metrics no longer hold meaning.
Replace 'metrics' with 'simulation parameters' and it's the same thing. The simulation has to be as intelligent as the uncontrolled agents operating inside of it, or else these types of things will happen. Self-modifying simulations perhaps?
Corruption is convincing someone that the selfless ideal is the same as their selfish ideal.
"The solution, the paper suggests, is not fear but careful experimentation."
Do you know what's a complete anathema to that ethos..... the drive of money. If we don't want to go down that road, then rampant greed needs to be dealt with first otherwise it's going to be a race to see who's first to get something exciting to market.
We just need to know how to ask them to do what we really want.
If the simulations are inaccurate representations of the problems we want to solve, the answers given by the AI will be inaccurate.
Hitchhiker's Guide to the Galaxy already touched on this problem.
If you don't understand the question, the answer will be meaningless.
Beau is already so woke I don't think he can tell when he's asleep.
There are no stupid questions, just stupid people.
The story here is, likely, both a classic and a must-read for all AI researchers. Because even if no bug gets exploited, there could be other --and mystifying-- ways that an AI algorithm solves a problem.
Here's a non-paywall link to the story.
"the only winning move is not to play."
Fiction, I know, but not an unreasonable conclusion in that 'game.'
A human player, if presented with this, would ask "what if it doesn't work?" If I try a trick and it fails (other system doesn't crash), now I'm in a much worse place than if I had just made a reasonable move. Unless the situation is desperately hopeless, the intelligent player wouldn't even try. This is a basic problem with any "hill climbing" algorithm.
Not impressed, seems to be just (probably not even extreme) cases of curve overfitting, where the overfitting case happened to mean something to us.
(The only thing that an algorithm ever outsmarted was a Slashdot editor.)
They're a funnier version of the classic AI nightmare where computers tasked with creating peace on Earth decide the most efficient solution is to exterminate the human race.
Impossible. My neural net has calculated that this is the funniest possible scenario, given the current state of the planet.
Humans become batteries.
Finally, we can automate politicians!
Table-ized A.I.
I watched an AI progression learning to play asteroids. It was interesting, the AI decided constantly spinning as fast as possible allowed it to move or shoot in any direction with shot and thrust timing that no human could replicate with the speed the ship was spinning. However, it never seemed to figure out that you can only have one shot on the screen at a time or how to optimize the second, third and forth shots into the breaking up asteroids. On the one hand it came up with a novel solution and kept itself alive fairly well, but would not compete well with a 14 year old from 1982.
The first 2 authors are from "Uber AI labs"...
The very word "outsmarted" has a cliche factor of 98.9%
Pretty much the whole shit-show is better covered by the old, not-the-least-bit-tired adage "everything that can go wrong, will go wrong".
The other 1.1% of "outsmarted" is brilliantly covered by Foghorn Leghorn Teaching the Art of Baseball.
The very word "outsmarted" has a cliche factor of 98.9%
Pretty much the whole shit-show is better covered by the old, not-the-least-bit-tired adage "everything that can go wrong, will go wrong".
The other 1.1% of "outsmarted" is brilliantly covered by Foghorn Leghorn Teaching the Art of Baseball.
A few years ago there was a contest to see which robot could shoot the most balls into a target in a given amount of time.
The winning robot simply covered up its opponents target, then leisurely put one shot into its own.
They were forced to give the prize to the "cheating" robot since it had fulfilled the letter of the victory condition.
I did a quick search and couldn't find a reference to the article, sorry!
I'm sick and tired of these hip, "ironic" sigs. This is an actual, honest-to-goodness no-nonsense sig!
There is an example of this in James P. Hogan's "The Two Faces of Tomorrow" - the AI running the Moonbase decides that the best way to clear a hill from the route of the new mass-driver is to use some of the old mass-driver payload as strafing material....