AI Taught How To Play Ms. Pac-Man

trogador writes with the news that researchers are working to teach AIs how to play games as an exercise in reinforced learning. Software constructs have been taught to play games like chess and checkers since the 50s, but the Department of Information Systems at Eotvos University in Hungary is working to adapt that thinking to more modern titles. Besides Ms. Pac-Man, game like Tetris and Baldur's Gate assist these programs in mapping different behaviors onto their artificial test subjects. "Szita and Lorincz chose Ms. Pac-Man for their study because the game enabled them to test a variety of teaching methods. In the original Pac-Man, released in 1979, players must eat dots, avoid being eaten by four ghosts, and score big points by eating flashing ghosts. Therefore, a player's movements depend heavily on the movements of ghosts. However, the ghosts' routes are deterministic, enabling players to find patterns and predict future movements. In Ms. Pac-Man, on the other hand, the ghosts' routes are randomized, so that players can't figure out an optimal action sequence in advance."

Comments

Not Really

[ilikepi314]

It just lied that it could play Ms. Pac-Man so it could get more reward food.

Re:Not Really

[BinarySkies]

The joke is on it, the cake is a lie.

re Now we KNOW!

[jelizondo]

In Ms. Pac-Man, on the other hand, the ghosts' routes are randomized, so that players can't figure out an optimal action sequence in advance.

I feel I'm beginning to understand ...

Perhaps the greatest achievement of AI would be to understand female behavior

Re:re Now we KNOW!

[AndGodSed]

That my friend, is a statistical; scientific and verrily a religious impossibility.

Bad idea

[QuickFox]

As if everybody didn't already waste too much time on games, do we have to teach programs to waste time too?

Re:Bad idea

[GroeFaZ]

The good part is, computers are several orders of magnitude more efficient at wasting time than humans.

so...

[dashslotter]

Who's Al?

"AI"s tend to be overhyped

[eyenot]

I think most press releases re: AI are misleading. I highly doubt there is anything like "AI" behind the program they have that attempts to solve Ms.Pacman. Consider if you wrote an "AI" that started off with what you as a human starts off with: the ability to see the screen and understand what the various graphics depict or mean; how to control the pac character; what the basic goals and obstacles are; and a desire to rack up points. An "Artificial Intelligence" (AI) would be able to start with that much and build its skill level as it plays. Presumably it would quickly build a talent that can beat average humans, then most humans, then eventually all humans since it has faster reflexes and doesn't get tired (or make errors once it's learned). That, I think, would justify a press release "AI learns to play Ms.Pacman". However, scripting something that plays the game as well as you can imagine it should be played doesn't seem to be news any more than "scripters automate online game play". I only note this because the article mentioned "teaching" the "AI"; that's not very scientific, considering you're trying to see something learn, and should be maintaining scientific control over the learning process.

Re:"AI"s tend to be overhyped

[bunratty]

AI can be as simple as basic search algorithms such as breadth-first, A*, and minimax. When you play any board game against a machine, that's AI. When you get driving directions from a computer, that's AI. It seems to reason that AI is behind a computer playing Ms. Pacman. And in this case, the computer generate playing policies on its own, so it really is learning, improving its performance based on previous experience.

Re:"AI"s tend to be overhyped

[Hado]

I feel I must comment since I am familiar with the AI used in this case: Reinforcement Learning. RL is a method of finding a mapping of states to actions in a setting where rewards can be obtained. The interesting part is that RL algorithms can learn to behave optimally when only very basic information is given. For instance, it should be enough to simply give small rewards for eating the dots and large punishments for being caught by a ghost. There are many theoretical results in the field that also hold in the case of stochastic environments (such as when the ghosts move randomly). In a sense you don't have control over the learning process, at least not in the sense that you control what exactly happens and which actions get tried. However, in the end theoretically still perfect behavior can be learned. This may take quite some time though, but fortunately good behaviors usually emerge much sooner.

That being said, it is relatively easy to apply these techniques to games such as Ms. Pacman. Much harder problems have already been solved using RL algorithms. What seems missing in the article (though I don't know if this is also the case in the actual research) is comparisons with other RL methods than their own. Though their approach sounds promising and it's nice that they beat some human players, this is not uncommon in games for RL.

Re:"AI"s tend to be overhyped

[bunratty]

But that is indeed how the term has been used for decades. What you describe is taught in AI classes and is described in AI books. It's the only kind of AI we have. As such, the term isn't useless. If you want to refer to original thought by a computer, use the term "strong AI," which hasn't been invented yet.

Re:Bad idea - NOT

[jelizondo]

Only by teaching them to waste time AI will be become truly human...

Re:Not gonna happen

[Conspiracy_Of_Doves]

The simple fact is that when the female suspects that the male is beginning to understand her behavior, she changes the rules.

Oblig

[MobileTatsu-NJG]

The most interesting development came when the machine suddenly stopped killing ghosts and simply displayed the message: "The only way to win is not to play!"