Slashdot Mirror
IBM Researchers Teach Pac-Man To Do No Harm (fastcompany.com)
harrymcc writes: The better AI gets at teaching itself to perform tasks in ways beyond the skills of mere humans, the more likely it is that it may unwittingly behave in ways a human would consider unethical. To explore ways to prevent this from happening, IBM researchers taught AI to play Pac-Man without ever gobbling up the ghosts. And it did so without ever explicitly telling the software that this was the goal. Over at Fast Company, I wrote about this project and what IBM learned from conducting it.
The researchers built a piece of software that could balance the AI's ratio of self-devised, aggressive game play to human-influenced ghost avoidance, and tried different settings to see how they affected its overall approach to the game. By doing so, they found a tipping point -- the setting at which Pac-Man went from seriously chowing down on ghosts to largely avoiding them.
The researchers built a piece of software that could balance the AI's ratio of self-devised, aggressive game play to human-influenced ghost avoidance, and tried different settings to see how they affected its overall approach to the game. By doing so, they found a tipping point -- the setting at which Pac-Man went from seriously chowing down on ghosts to largely avoiding them.
This is interesting, but I'm not sure how useful it is. You might be able to create an AI that has some desirable characteristics based on human morality, but as soon as you make it compete against other AIs that don't possess those characteristics, it will either adapt to possess them itself in order to remain competitive or it will perish if it's been crippled in a way to prevent it from adjusting. A pacifistic Pac-Man AI might be novel, but if it was made to compete, it wouldn't do as well.
"Pacifist" runs of games are popular with the speedrunning community. Depending on the game it might be an achievement to simply finish it without harming anything, or it may just be another category to get the fastest time in.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
That's the point. There are plenty of real world cases where we could be more efficient if we simply disregarded moral and ethical concerns. One of the concerns with machine learning is that they may find an optimal solution that violates ethical considerations. The problem is even larger when you consider an AI finding locally optimal solutions disregarding externalities.
For a classic example, Ford once determined that paying off expected damages in wrongful death suits would be slightly cheaper than refitting existing Pintos to not explode.
The Pac-Man simulation is a very simplified version of a case where, due to ethical considerations it is necessary to avoid the locally optimal solution.
> The Pac-Man simulation is a very simplified version of a case where, due to ethical considerations it is necessary to avoid the locally optimal solution.
The Pac Man doesn't avoid any optimal solution. It simply defines optimum as not to include not touching ghosts - ghosts are bad. In the classic version of the game, touching a ghost is bad. Unless you've eaten a Power Pellet in the last few seconds. They trained the AIto NOT learn the "unless you've eaten a pellet". It just does "touching ghosts is bad".
There's nothing moral, or even interesting, about "in Pac-Man, touching ghosts is bad". Essentially, just one too stupid to know that Power Pellets do anything.