Slashdot Mirror
IBM Researchers Teach Pac-Man To Do No Harm (fastcompany.com)
harrymcc writes: The better AI gets at teaching itself to perform tasks in ways beyond the skills of mere humans, the more likely it is that it may unwittingly behave in ways a human would consider unethical. To explore ways to prevent this from happening, IBM researchers taught AI to play Pac-Man without ever gobbling up the ghosts. And it did so without ever explicitly telling the software that this was the goal. Over at Fast Company, I wrote about this project and what IBM learned from conducting it.
The researchers built a piece of software that could balance the AI's ratio of self-devised, aggressive game play to human-influenced ghost avoidance, and tried different settings to see how they affected its overall approach to the game. By doing so, they found a tipping point -- the setting at which Pac-Man went from seriously chowing down on ghosts to largely avoiding them.
The researchers built a piece of software that could balance the AI's ratio of self-devised, aggressive game play to human-influenced ghost avoidance, and tried different settings to see how they affected its overall approach to the game. By doing so, they found a tipping point -- the setting at which Pac-Man went from seriously chowing down on ghosts to largely avoiding them.
That's the Conservative Pac Man you're describing. The Liberal Pac Man takes care not to eat the ghosts, and uses whatever points he get from eating pellets in a sustainable way to pay for ghost shelters and outreach to better understand them and protect them.
When the AI begins watching humans play Pac-Man, doing harm to the ghosts, it will consider humans a threat to ghosts and thus eliminate the humans to satisfy its directives. Is IBM's median employee age too young to have seen Robocop?
Title: "IBM Researchers Teach Pac-Man To Do No Harm"
Blurb: "they found a tipping point -- the setting at which Pac-Man went from seriously chowing down on ghosts to largely avoiding them."
So companies will presumably use a similar method to design AIs that will maximize corporate profit with only a _small_ amount of acceptable human murdering in the process?
This Space Intentionally Left Blank
That's the point. There are plenty of real world cases where we could be more efficient if we simply disregarded moral and ethical concerns. One of the concerns with machine learning is that they may find an optimal solution that violates ethical considerations. The problem is even larger when you consider an AI finding locally optimal solutions disregarding externalities.
For a classic example, Ford once determined that paying off expected damages in wrongful death suits would be slightly cheaper than refitting existing Pintos to not explode.
The Pac-Man simulation is a very simplified version of a case where, due to ethical considerations it is necessary to avoid the locally optimal solution.
It's having an algorithm grind out a solution to playing the game which meets an additional constraints, which they tweak.
If they'd actually taught the AI ethics, the AI would construct the play constraints for itself starting from ethical principles. At full human levels of ethical self awareness, the AI would be chasing ghosts down the hall and then -- unprompted -- stop and ask itself, "What am I doing?"
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
That's the Conservative Pac Man you're describing. The Liberal Pac Man takes care not to eat the ghosts, and uses whatever points he get from eating pellets in a sustainable way to pay for ghost shelters and outreach to better understand them and protect them.
And PAC Pac Man doesn't disclose how many points he has and uses them to get the ghosts to do his bidding.
It must have been something you assimilated. . . .