Artificial Intelligence Bests Humans At Classic Arcade Games

sciencehabit writes The dream of an artificially intelligent computer that can study a problem and gain expertise all on its own is now reality. A system debuted today by a team of Google researchers is not clever enough to perform surgery or drive a car safely, but it did master several dozen classic arcade games, including Space Invaders and Breakout. In many cases, it surpassed the best human players without ever observing how they play.

Comments

Breaking news!

[penguinoid]

Someone made a computer that's really good at reaction time, and at calculating trajectories.

Re:Breaking news!

[ShanghaiBill]

Someone made a computer that's really good at reaction time

It was done awhile ago. By IBM. Watch Watson play Jeopardy, and it is pretty obvious it won mainly because it was much faster at triggering the button. Watson wasn't better at answering the questions, it just got more chances.

Re:Breaking news!

[Dutch Gun]

Seriously, is there any doubt that a computer can easily defeat a human at a computer game that involves 95% pure reflexes and 5% strategy?

The article shows a picture of Breakout, and tends to focus on the wrong things entirely... especially the title, trumping that "computers can beat humans". It's fairly impressive that computers can learn the rules of a simple videogame on their own and perform well, but beating humans is not exactly an apples to apples comparison, because while we can formulate strategies to maximize points, we're also prone to making simple mistakes due to our much poorer reflexes and coordination. So AI has a massive advantage with precision reflexes and calculations that it can make much faster than humans.

Some of my previous jobs involved programmed AI game opponents for action games. As anyone who's faced an aim-bot knows, there's no real challenge for computers to perform many of the tasks humans find difficult, like putting a bullet through a moving target's forehead. I actually had do a lot of extra work to programmatically replicate the difficulties humans face when aiming at a moving target. However, collecting and processing global environmental knowledge and formulating complex strategies based on that knowledge is extremely difficult. That's why we typically build a lot of invisible hints into the environment itself for the benefit of AI, such as pathfinding-specific structures, or dynamic flags that signal potential rewards or danger. Even today, in many strategy games that involve complex ruleset (meaning brute force calculations can't work as well), the computer opponents inevitably have to cheat in order to compete with even modestly skilled players.

Early videogames have very few of these sorts of challenges because of their largely static environments and the basic nature of the games. For the most part, you just need to formulate a few simple rules for an optimal victory condition, and when combined with a computer's incredible performance, you can easily trounce the best human players, simply because a computer never gets distracted, tired, or makes silly mistakes in judgement.

Again, I'm not dissing the work the researchers did, which I found to be impressive, but the article and summary seem to be missing the point entirely by comparing them to human scores. It's fairly obvious that once a computer learns how to play with an optimal strategy, it's an absolute given that they'll score better than humans ever could.

Re:Breaking news!

[Anonymous Coward]

By telling the lie that you think this happened makes you a liar. Believing a liar makes you a liar. Are you smart enough to understand that? This did not happen. There is no proof. There is no video. The only evidence is a vague statement from a Republican. We all know their kind lies. Just look at Rmoney's claim that he is human. No. He is a man-like object. That is why he is so hateful. Again, you are a liar since you believe a liar.

Re:Breaking news!

[deesine]

I figured out an endless pattern to Atari 2600 Space Invaders and PacMan, high score stuff. Was thrilled and disappointed to read about my solution in some Atari mag several years after my discovery.

I figured I had beat the computer and was disappointed when War Games came out.

--

Re:Breaking news!

[ljw1004]

The article shows a picture of Breakout, and tends to focus on the wrong things entirely... especially the title, trumping that "computers can beat humans". It's fairly impressive that computers can learn the rules of a simple videogame on their own and perform well, but beating humans is not exactly an apples to apples comparison, because while we can formulate strategies to maximize points, we're also prone to making simple mistakes due to our much poorer reflexes and coordination.

Exactly. The article talks about the "advanced strategy" of tunneling a hole through to bounce the ball of the back wall. But that's only a useful strategy to make up for someone who doesn't have the reflexes to bounce the ball with their paddle, or can't be bothered. If the program had good reflexes and didn't get bored, then tunneling in breakout isn't any advantage.

Re:Breaking news!

[mrmeval]

Also the game has finite limits which over time the Artificial Stupid can memorize completely enough to anticipate it ahead of time.

Re:Breaking news!

[Lennie]

I actually didn't see this story as news, I had seen a video of there work last year from before they were bought by Google.

That same video was linked from the article:
https://www.youtube.com/watch?...

What makes this more interresting is, they didn't tell the AI how to play the game, they let the AI learn to play the game on it's own.

I think one of the things this what makes this also interresting is how few times the AI needed to learn the game and then also be good at it.

Re:Breaking news!

[Ralph Wiggam]

At high levels, Jeopardy is all about who presses the button first. Watch a Tournament of Champions. All 3 people buzz in for pretty much every question.

The trick is who can parse what the "answer" is really asking, recall the fact required, and then buzz in before the other players can do all 3 things. If Watson can do those things faster than a person, it won fair and square. Just being able to parse the "answer" was an incredibly impressive achievement.

Re:Breaking news!

[PRMan]

It cheated. It was fed text instead of having to take the time (like the people did) to determine the meaning from the audible clue given.

Re:Breaking news!

[PRMan]

Pfft. It didn't take me 600 games of Breakout to learn to put the ball above the bricks...

Re:Breaking news!

[rtb61]

Hate to break it to you but that was done, years and years ago, it is really easy for computers to do that stuff. The really hard bit it to analyse the environment and from that analysis create an internal virtual environment that you can base your calculations and optimum decisions on. It really is difficult for computers to analyse the visual environment, understand what is within that view and how the various elements will behave as changes occur. So virtual computer robotics is pretty easy because you skip that whole extraordinarily difficult area of actual real time environmental analysis. Fast track robotics development means you would skip that area whilst developing appropriate robotic responses in virtual environments, you could cunningly fund this by creating animation content using those robots.

Re:Breaking news!

[Comboman]

The human players get the clue in text format also (printed on the monitor wall). Alex Trebek reading the clue aloud is strictly for the benefit of the mouth-breathers watching at home.

Re:Breaking news!

[ShanghaiBill]

... and then buzz in before the other players can do all 3 things.

This is not correct. It is NOT who buzzes first. Alex reads the question and then a light comes on. If you push the button before the light comes on, then you are locked out for a quarter of a second. So the trick is to push the button the instant the light flashes on. It is pure response time. Of course a computer is going to be better at that. That is 99% of the reason Watson won.

Re:Breaking news!

[Maxo-Texas]

That depends on the incentives the AI has.

In this case, it appears it has incentives to gain the highest possible score as quickly as possible.

In this case, tunneling and bouncing off the top wall better matches those goals.

I read about his before and the computer starts out not knowing where the score is-- it has to learn which area is score and then do random things with the game until something succeeds at causing the score area to go up... and then optimize for high score and high speed.

That sure sounds like learning to me.

Re:Breaking news!

[pitchpipe]

Seriously, is there any doubt that a computer can easily defeat a human at ___________ that involves _____________?

Of course not. Whenever a computer defeats a human easily, of course it isn't true AI. Computers were better at that all along. Leave that to computers so that humans can do the truly human work.

Re:Breaking news!

[narcc]

The bit about the score is key here. It's essentially no different than any other learning algorithm as it does not discover on its own that the goal is to achieve a high score. The computer vision part is neat, but nothing new, and ultimately does nothing to differentiate this from the zillion other similar projects as it is only used to find the score! Countless hobbyists and researchers have made ANN's and Genetic algorithms which produce similar results, both the computer vision part and the game-playing part.

The program does not, in any way, "study a problem and gain expertise all on its own". It's pure click-bait. I'm ashamed to have fallen for that particular trap.

Re:Breaking news!

[narcc]

Whenever a computer defeats a human easily, of course it isn't true AI.

You're confused. I'm not sure how, exactly, but you might want to google "hard problem" and "strong AI" to net (ha!) yourself a better grounding.

Re:Breaking news!

[itzly]

As anyone who's faced an aim-bot knows, there's no real challenge for computers to perform many of the tasks humans find difficult, like putting a bullet through a moving target's forehead.

It always makes me wonder why fights in futuristic movies are always done by people aiming by hand.

Re:Breaking news!

[AmiMoJo]

The key achievement here is that the AI was able to learn the game on its own in a relatively short time. Imagine if you had an industrial robot that could learn how to do tasks on its own and then modify its behaviour if the situation changed, and generally cope with a variety of situations.

Also, they called it DQN which means "dumbass" in Japanese, so bonus points for that.

Re:Breaking news!

[telchine]

Some of my previous jobs involved programmed AI game opponents for action games. As anyone who's faced an aim-bot knows, there's no real challenge for computers to perform many of the tasks humans find difficult, like putting a bullet through a moving target's forehead.

Then why did Steven Polge resort to making the ReaperBot cheat?

Re:Breaking news!

[Bonzoli]

Yes, it has a great language/text codex, and the training process for watson is a scheduled batch job oddly enough. But once its trained its good at finding answers. I was disappointed they didn't have a good voice to text system running for the game.
I've seen it in the lab at ibm and asked a lot of questions. Its a language codex which is quite good, sitting infront of a ranking database of information.
The demo they had was medical journal based, and seemed quite useful for doctors that are looking for answers which are current.

Re:Breaking news!

[gl4ss]

a looong while ago.

what do you think the military used the computers first for? I thought that was the joke of the comment.

the thing is though giving it an arbitrary video game. making a robot to play just breakout is very easy, but that it plays breakout after it sees it is not that easy (though breakout is very easy on that scale, paddle follows ball and you get points, not much trial and error involved before you have winning combination(also wasn'tthat done a year or three ago already??)

Re:Breaking news!

[thedonger]

I figured out an endless pattern to Atari 2600 Space Invaders and PacMan, high score stuff. Was thrilled and disappointed to read about my solution in some Atari mag several years after my discovery.

I figured I had beat the computer and was disappointed when I wasn't asked to help defeat Xur and the Ko-Dan Armada.

Fixed that for you.

Lower the bar further.

[HornWumpus]

AI is now trivial pattern matching.

Re:Lower the bar further.

[AK Marc]

For non trivial patterns, that is Intelligence. So they have trivial now, next they just need to improve that. A human does nothing but patten matching.

Re:Lower the bar further.

[jellomizer]

The classic arcade games are more of an issue of attention, than skill or thinking ahead.
Once the AI figured out how the play the game, I bet it can focus and pay attention to the detail, more than a human can.

Re:Lower the bar further.

[Anonymous Coward]

Not true, we also do input and output. Gabage in, garbage out.

Re:Lower the bar further.

[Austerity Empowers]

Don't even need that. I made pong for the TI-82 instead of paying attention in calculus class in high school, my "AI" could not be beat. Because it's really easy to do things when you can precisely calculate vectors and positions... It's actually harder to have something that makes human-like mistakes.

I don't think actual breakout or space invaders would be significantly harder.

Re:Lower the bar further.

[s.petry]

And Pay walled to boot!

Re:Lower the bar further.

[AK Marc]

Outputs caused by (or matched from) inputs.

Re:Lower the bar further.

[AK Marc]

What problem does it solve? I have a "robot" at home that makes things colder. It's called a refrigerator. Improving it won't solve anything else. It does no pattern matching. Neither does your robot.

Re: Lower the bar further.

[AK Marc]

Only if we put the heat exchanger on the Moon.

Re:Lower the bar further.

[HornWumpus]

For example: I have pattern matched this thread^Warticle^Wweb site and decided it was a repetitive waste of time.

Re: Lower the bar further.

[narcc]

You've missed the joke.

Nooooooo

[aaaaaaargh!]

I want my symbolic AI back...

Strategy games?

[gurps_npc]

No, merely reaction time.

In other words, this is an example of good image recognition software, that's it.

Show me a game that beat a human on a strategy based game, then you have something.

Re:Strategy games?

[fisted]

how about chess?

Re:Strategy games?

[gurps_npc]

Humans play chess as a strategy game. Computers play chess as a math problem to solve by looking ahead x turns.

"Go" on the other-hand must still be played as a strategy game by computers because looking ahead is not that helpful.

Re:Strategy games?

[Nethemas the Great]

Actually it does develop strategy. In BreakOut it doesn't merely bounce the ball with good reaction time. It learned to tunnel up to the back side of the wall. RTFA

Re:Strategy games?

[Pieroxy]

A pic^H^H^H link or it didn't happen.

Re:Strategy games?

[Trogre]

To quote Queeg:
Chess

Re:Strategy games?

[msobkow]

It's not even a good example of image recognition, because the images to be processed don't have to be "understood" to be used. On top of that, the graphics of the games in question were very simple and primitive compared to what image recognition software deals with.

Add to that the repetitive nature of old video games that were based on 99% reaction time and 1% strategy, and you can just flat out colour me "unimpressed" with this "research".

Back in University, my AI project was a game player (a simple strategy game whose name I forget.) As it turned out, the entire game mapped down to a pre-determined set of decisions, so after playing only a dozen games, the "AI" would win every time, and that was just with a simple weighted-algorithm system of play. Some problems are just eminently suited to "AI" is all that I ended up learning from that project, but it was a useful lesson on the difference between optimizing a decision tree and actual "intelligence".

Until someone comes up with a system that can deal with bad and erroneous inputs as well as humans, I will continue to be unimpressed. Yet at the same time, I don't consider it necessary for a computer to be able to think and understand per se to be considered an "intelligence." It just needs to be able to make decisions and choose between alternatives faster than it's human counterparts in order to be useful, and to reduce the number of errors compared to it's human counterparts.

I have little faith in "neural networks." They place too much emphasis on emulating simple biological components and not enough on the "art" of understanding. Neural networks basically take the approach that "if it's big enough, we'll maybe get lucky and it will start to think." That's not "solving a problem." That's "playing the lottery."

Re:Strategy games?

[Tablizer]

Go atse?

Re:Strategy games?

[Tablizer]

Maybe humans memorize patterns of play (responses) that win then. If the computer cannot "look" far ahead in Go, than neither can people, most likely. What exactly is "strategy" that is different from predicting ahead and/or learning successful patterns?

Let it try at 80s/90s games

[Twinbee]

I like to see them play more advanced games such as Rainbow Islands, The New Zealand Story, Strider, or Zelda 3.

Re:Let it try at 80s/90s games

[Anonymous Coward]

Or say, Angry Birds or Fruit Ninja.

Re:Let it try at 80s/90s games

[Pieroxy]

The real question being: did the AI enjoy it as much as a human?

Re:Let it try at 80s/90s games

[Twinbee]

Not in my opinion because I don't think AI will ever experience qualia or anything for that matter.

Re:Let it try at 80s/90s games

[vivian]

I'd like to see a robot play fruit ninja for real. Sure, it'd be a little scary having a robot waving a razor sharp katana around, in your kitchen, but think of the time you could save making fruit salad!

Re:Let it try at 80s/90s games

[gman003]

I want to see a computer play Mornington Crescent.

Re:Let it try at 80s/90s games

[PRMan]

Yeah, I've solved the first 3 of those. Have at it AI!

Re:Let it try at 80s/90s games

[weilawei]

import random
import urllib.request

player = random.choice(['You', 'I'])
station = str()

tube_stations = urllib.request.urlopen(r'https://raw.githubusercontent.com/rk295/tube-postcodes/master/tube-postcodes.txt').readlines()
tube_stations = [i.decode('utf-8').split(',')[0] for i in tube_stations]

while True:
        if 'You' == player:
                station = input('Enter a station: ')
        else:
                station = random.choice(tube_stations)

        if 'Mornington Crescent' == station:
                print("%s won!" % player)
                break
        elif station in tube_stations:
                print("%s chose %s. Wrong!" % (player, station))
                player = 'I' if ('You' == player) else 'You'
        else:
                print('Try a real station!')

Ask and ye shall receive?

I'm one step ahead

[Tablizer]

Well, I outsourced my Donkey Kong playing before bots took it over, so there!

meh

[Anonymous Coward]

This just in: Even in simplistic AAA games with bots, the AIs are better than human players, we have to dumb them down to keep the game fun.

Training a neural net to play tetris is AI:101. Teaching it to play mario, has been done to death by AI students. Let me know when the AI complains about the ending of Mass Effect 3. Then I'll care.

Re:meh

[the_skywise]

> Let me know when the AI complains about the ending of Mass Effect 3.

BUZZ CLICK WHIRRR... This Game Sucks. Click.

(Side note - I wonder how long before the AI evolves (degenerates?) into comic book guy nerd speak... Are all of our nerdisms really just natural progressions of logic?
BUZZ CLICK WHIRR... Worst Game EVER. Exclamation point. Exclamation point. Exclamation point. Click. )

Re:meh

[Stormy Dragon]

Wasn't the AI complaining about the ending of Mass Effect 3 pretty much the plot of Mass Effect 3?

Re:meh

[vux984]

This just in: Even in simplistic AAA games with bots, the AIs are better than human players, we have to dumb them down to keep the game fun.

First the prime challenge in the games you are talking about is lining up a crosshair with a pixel with a mouse and selecting fire.

If AI's had to do that they might have some difficulty. In practice the so-called AI bots already know where you are, and could keep their weapon lined up on your noggin through half the map without the need for line of sight. Tthey also get to target and fire at me without having to diddle around with a mouse or looking at the screen to see where I am.

Get a bot to actually play such a game with the same UI and world view I have (keyboard and mouse and what they can see on screen and hear on the speakers) and they tend to be quite abysmal.

Second, switch over to RTS games... and there the only way to give the AI any challenge is to stack the deck in its favor... whether its StarCraft or Supreme Commander or Wargame: Red Dragon. Or in a 4X game like Masters of Orion etc... we've yet to see an AI even really challenge a human being without giving it scripts to follow and extra resources to use.

Re:meh

[AK Marc]

Second, switch over to RTS games... and there the only way to give the AI any challenge is to stack the deck in its favor... whether its StarCraft or Supreme Commander or Wargame: Red Dragon. Or in a 4X game like Masters of Orion etc... we've yet to see an AI even really challenge a human being without giving it scripts to follow and extra resources to use.

Lots of extra resources, and a broken fog of war. And the easy/medium/hard on some games changes the damage/health of units.

Re:meh

[mjwx]

This just in: Even in simplistic AAA games with bots, the AIs are better than human players, we have to dumb them down to keep the game fun.

Thats because 99% of AAA games are twitch games. No strategy involved, just reflexes. Games like COD, Halo, et al. really limit what the player can do and it really is the fastest mouse wins. So they have to limit the reaction time of the AI to what a human is capable of.

Now if you look at strategy games like Civ IV, you had to give the AI unfair advantages to put it on equal footing with human players. But turn based strategy games have a lot of fuzzy logic. So really an AI mastering COD is not news, we can have been able to build bots with superhuman reaction time since the 90's. Wake me when they can master a city building sim as well as a good human player.

Re:meh

[Anonymous Coward]

Seriously, you don't sound like you know what you're talking about.

Nor do you. Your AI doesn't have the same control interface as human players. An AI would never miss a stationary target unless you programmed it to or are really bad at programming. Humans do miss them. Humans have to deal with moving a mouse an exact distance. AIs don't. They simply move the exact distance required. No need to deal with mouse physics, no need to deal with running out of desk space, no need to estimate the amount of movement, etc... They don't need think which key does what then press the key, they just call the proper function. The UIs are completely different even if both have the same available actions.

If you're giving an AI millions of training iterations, you should be giving the human a similar level of training. The AIs trained on top players only learn what they've seen. They don't handle other things very well. Look at just about every released RTS game. When you play a round long enough, the AI mostly stops everything or ends up in a 96% predictable repeating pattern.

Almost no AIs work well in teams. Humans do far, far better.

I'm a RTS AI researcher. It sounds like you do something similar, so you should know that much "advanced academic AI" is also poor. Most papers restrict their game environments so far that their results are almost meaningless. Sure they can calculate optimal attack orders for a 6x6 battle, but there's not enough CPU or memory left to do anything else.

Your AI sounds like a normal case-based, reinforcement leaning AI that trains against itself. What game/paper is it?

Re:meh

[Cytotoxic]

When Id released Quake C, this "you have to dumb down the AI" idea became very apparent. Someone wrote a replacement AI for the enemies that allowed them to learn and communicate. They had to follow the rules and physics of the game - so if they were within earshot they could communicate your position to each other, otherwise they couldn't.

The first couple of interactions would be pretty easy kills. Then one enemy would see that you were better armed and run away. That would be the last enemy you would see for a while. Then, when you were in a vulnerable position, the entire population of the level would ambush you in a coordinated attack. Game over. They were way, way, way too smart to be beaten. It was pretty fun to explore their learning capabilities and watch how they would win. But it didn't make for engaging gameplay, unless you are a complete masochist.

The same AI was applied to deathmatch player bots. They had no prior knowledge of the level, or strategies for playing the game. The first few kills were very easy as they figured out what to do. But as they learned your tendencies, they would very quickly evolve into a circle-strafe master. They also learned the map layout pretty quickly, including drop sites and periods for weapons and health. They would then time their circle-strafe to always be on the spawn site immediately as the health or ammo spawned. They would invariably win against even the best human players by monopolizing all of the supplies and winning a war of attrition. Very impressive to watch.

This "AI" program was very rudimentary, and it was already much too difficult for human players, despite being limited to the same "in game" knowledge and input capabilities as the human players. It makes perfect sense that the challenge and complexity of programming the AI for games mostly revolves around "dumbing down" the AI in a way that makes the enemies challenging and interesting, but also the right amount of "beatable".

Re:meh

[vux984]

The same AI was applied to deathmatch player bots. They had no prior knowledge of the level, or strategies for playing the game. The first few kills were very easy as they figured out what to do. But as they learned your tendencies, they would very quickly evolve into a circle-strafe master. They also learned the map layout pretty quickly, including drop sites and periods for weapons and health. They would then time their circle-strafe to always be on the spawn site immediately as the health or ammo spawned. They would invariably win against even the best human players by monopolizing all of the supplies and winning a war of attrition. Very impressive to watch.

Yes, and it gives another example of the issue. The AI has access to a perfect clock, and perfect spatial awareness, and has perfect control over its movement, sees everything in its field of view no matter how briefly it sees it...

It can run and jump a maze of catwalks over lava with perfect reliability - BACKWARDS. It can do all that and land on a specific set of coordinates where it knows supplies are going to respawn the millisecond it respawns, because it can time to the millisecond when it last picked it up.

It's cool, and it is devilishly hard to compete against, but its not good "AI". Its not out thinking its opponents, its merely taking full advantage of its machine attributes.

For example, In quake I might know the location and approximate respawn time of the quad damage. I would certainly plan my route to be in the area when I was expecting it to respawn... or perhaps plan to have a sight line to it to pick off someone else making a run at it. I have a rough idea how long it takes me to get from where I am to the Quad damage spawn point from various places on the map.

A better player than me, will have smaller error bars on all of those things. A top player smaller still. But the "AI" it has perfection. It's not coming up with a BETTER strategy than me, its simply perfect at executing the same strategy because it can run the level backwards, and end up precisely where it wants to end up precisely when it needs to.

Making the AI human-like; imparts the same reaction times, the same imprecise lossy recollection of the map (I can run some maps backwards, but its usually at least a little bit of steer-by-wall... so if its catwalks over lava or freeform jump pads on a quake 3 open level I can't do that without looking. I can maintain approximate respawn cycles for a number of spawn points, and calculate approximate lenghts of time to get from point A to B... but not for every spawn point in the level, and not to within a millisecond,

Restricting the AI to human ability is not "dumbing it down" that's just putting us both on equal footing. Go ahead and make the AI as good as it can possibly be under all those constraints...if its truly a better than human AI it will come up with better strategies given those constraints.

Because I can come up with the strategy of run and jump backwards, never miss with the rail gun, and pick up the quad damage the instant it spawns; I just can't execute on that strategy with machine precision because I'm not a machine.

Re:meh

[Cytotoxic]

That echos my point, somewhat. It is pretty easy to design an AI for a lot of video games that can beat a human (without cheating). The AI code from Quake was only a couple of pages. Whether you use the moniker "constraints" or call it "dumbing down", it would take a lot more code to give the AI more human-like abilities. Probably several times as much code.

The same goes for the enemies. The code that gave them a degree of autonomy and communication was pretty small, but it made them unstoppable because they would just run away and warn all the other enemies and come attack en mass and from cover. It wasn't sophisticated AI at all. Making an AI that was competitive with humans required a lot more coding, restricting the capabilities of the enemies. It is odd that the hard part about making a game AI would be making an AI that isn't too competetive, but that's where we are.

As to your other point, making a machine think like a human strategically is probably much more difficult. But here's the philosophical question: does the motivation behind someone's actions really matter, or is what they actually do the only thing that actually counts? If the computer is unbeatable because it is just that much faster than you does it really make for a difference in gameplay as compared to a computer that is unbeatable because it truly understands better than you and always initiates a better strategy? Sure, one might be a lot more exciting from a programming point of view, but either way you still got beat by the computer.

Re:meh

[vux984]

That echos my point, somewhat. It is pretty easy to design an AI for a lot of video games that can beat a human (without cheating).

Yes, albeit, for a slightly strained definition of "without cheating" :)

Whether you use the moniker "constraints" or call it "dumbing down",

I think the distinction is important. Dumbing it down would be deliberately sabotaging its ability to make good decisions. The constraints certainly have the same effect, but we aren't sabotaging the decision making itself, but merely restricting the information it actually has to work with access to human levels.

No perfect clock. No perfect positioning. etc.

It is odd that the hard part about making a game AI would be making an AI that isn't too competetive, but that's where we are.

Not really. It's just that we've devised a game that's difficult and interesting for humans, but is very easy and trivial for the bot; especially given that by default it has world state information we just don't have. (It knows what time it is, what its x/y/z is.

It can:

advance 4 units at 1 unit per second,
turn 28.5 degrees at 5 degrees per second
retreat 12 units at 11 units per second
turn 19 degrees a 11.4 degrees per second
advance 72 units at 15 units per second
can be arbitrarily long, is trivial for the bot to execute, doesn't require any sensory input/feedback, and can be replayed backwards.

I can't do that. No human can. We aren't that precise. We maintain position and navigate only approximately with constant correction from sensory input. And if I'm playing a game where perfect navigation is demonstrably a very valuable skill then I can't compete with a bot that is allowed to do that. (And my performance against bots drops considerably on open or lava catwalk Quake 3 maps vs closed tunnels maps due to their inhuman ability to navigate exactly where they wish to go whether they are looking where they are going or not...)

That's not "advanced AI"; that's "giving it rudimentary AI paired with superhuman ability". That's not much fun.

But here's the philosophical question: does the motivation behind someone's actions really matter, or is what they actually do the only thing that actually counts?

No it doesn't matter if they do the same thing. But they don't do the same thing. The current process of tweaking the AI's super human abilities and tossing a wrench into their decision making so its not always the optimal choice etc to make them appear more human like vs actually being human-like in response to human constraints does not result in them doing the same thing.

Instead of being truly human like. They end up acting like superhumans with narcolepsy... brilliantly efficient but occasionally they just fall asleep; and worse they usually can be relied upon to fall asleep at the wheel in a given set of circumstances that can be manufactured by the opponent (aka exploited)

For example, in RTS with harvestors a human player might trap a bunch of the AI's harvestors -- a human opponent would not be fooled once he saw what happened, and he'd build more, and suicide/free/destroy the trapped units. But the AI? The average AI will just starve, because it hasn't been programmed to recognize that scenario or how to respond when it happens. It has enough harvestors, so it won't build more... it has sent them orders to harvest...and they aren't under attack... so mission accomplished. Resources will arrive soon.... any minute now... still waiting... oh my base is under attack... hope those resources show up soon... man where are my resources... shit I lost.

All manner of path finding exploits are common in RTS games. Getting his units to stumble over themselves and get in their own way. Funnelling them into kill zones. Etc.

Don't underestimate humans

[K3rn3lPan1c]

Artificial Intelligence is no match for natural stupidity.

This guy did it in 2013...

[EmeraldBot]

This article reminds of this guy who did something similar a while back.

So ... umm...

[Opportunist]

Now I can buy an AI so it can play the computer game I bought so I'd have more spare time?

Re:Meh..

[dlingman]

That one got solved by the physics guys a while back. http://en.wikipedia.org/wiki/T...

Re:No AI is a Wizard

[narcc]

Damn, dude, you must be deaf, dumb, and blind.

Impressive accomplishment..but..

[MpVpRb]

..it's a computer figuring out how to beat a computer at a kinda simple game

The real world is a bit harder

Still..well done!

Paywall and some pdf rendering

[140Mandak262Jamuna]

Looks like the linked site is a pay wall or something. Renders the article in low res, throws a lot of pop ups. It seems to be a bad mash up of javascript running flash and pdf. Malware purveyors dream.

Wonder why the editors let such bad sites and auto playing videos to be posted.

Re:Paywall and some pdf rendering

[cfalcon]

Which one is giving you issues? With noscript / firefox I had no problem with the science mag one. The nature one is really just a link to http://www.nature.com/nature/j...

It's cool but

[cfalcon]

It's actually really cool that this happened, so it's a shame that most of the reporting on it is sort of "correction bait". The fact that it does good at these games without watching human strategies is interesting, but computers have strategies that humans lack, due to their increased reaction time (random thing happened, I can respond by doing X -> the computer is several orders of magnitude superior at this for free) and increased calculation time (the trajectory is curve such and such -> your visual cortex is great at this, but a computer can beat it at something like this). Human strategies for these games involve working around the relative difference in clock speed, so of course the computer would have no need of it.

But forgetting that garbage, it's still very cool.

Wait, not one classic arcade game in here?

[cfalcon]

It looks like it worked with Atari 2600 games, which are ports of classic arcade games. A nitpick, but about 30 seconds playing the 2600 version versus the arcade version will show you a ludicrous level of difference betwixt. I don't want to belittle the work, but calling 2600 games arcade games is like calling a motorcycle a semi truck. Words have meaning- in this case, "Atari 2600 games" or "classic games". NOT arcade games.

Re:Wait, not one classic arcade game in here?

[cfalcon]

It's a misleading headline for sure. If you check the PDF it's pretty clear about the volume of 2600 games it works with, but they are def. not arcade games.

Surprising

[nospam007]

Wow! For a generation, we knew that AIs are good at playing Global Nuclear War and now they tell us it's more like Super Mario Bros?

does it have a favorite?

[The_Rook]

which game is the AI's favorite?

Unfortunately...

[Yaztromo]

Unfortunately, the experiment came to an abrupt end when they threw "ET: The Extra Terrestrial" at the AI, whereupon after an hour of trying different tactics the AI decided that the only way to win was to send a power surge through the system, frying the only working Atari 2600 the researchers could dig up.

This still classifies the AI as coming up with the best solution to the game ever implemented.

Yaz

Re:Unfortunately...

[cfalcon]

Those early Atari 2600 ET adopters really had to stick their neck out with that game...

Re:No AI is a Wizard

[dpidcoe]

Wait.. so how would he play then? Using his sense of smell or something?

Comments are predictable...

[nathan s]

You gotta love humans. Every time an AI starts to be able to do something that was previously only our domain, it's all "Meh, wake me when..." and "Yeah, but a computer still can't..."

Funny stuff.:)

Re:Comments are predictable...

[cfalcon]

Programming a computer to ace these games was possible in the 90s, maybe the 80s. The reason this is interesting is HOW it taught itself, and how many of the games it could get good at (many of the games it could not learn). Cheerleading AI research is nice, but this isn't an example of a computer entering a new domain, this is a research example of something that can solve other problems in the field- an engineering demo of sorts.

Re:Comments are predictable...

[nathan s]

Obviously. The learning is the "something that was previously only our domain," not the playing, which is precisely what people are reacting so defensively to, and what I find funny.

Once AIs using ANN or whatever the ultimate technologies end up being can actually learn at a human level, it'll be "meh, wake me when it can appreciate a sunrise" or "Yeah, but a computer still can't fall in love!" My point is just that we move the bar in order to preserve our collective sense of being special snowflakes.

Re:Comments are predictable...

[Maxo-Texas]

Oh sure, it says it is in love but it's a computer. I know it's just a simulation of love, not the real thing.

I agree.

The thing is-- with robotics doing parallel work on human level physical reactions (like tossing things in the air and catching them- without the use of a brain), true AI may be more human like with one part being the conscious mind that says "start walking towards the door" while other parts control the actual movement, balancing, etc.

if you start reading about the brain (Brain Bugs is a good book for it), the first thing you see is that the brain is multiple independent systems. If you break them, the conscious mind does really weird things like, for example, saying "That's not my limb" (alien limb syndrome), losing the ability to form memories, crossing sensory systems (so sounds smell and odors have colors), and what's really crazy is that often- even when informed of the problem- the conscious mind of the people can't process that anything is wrong.

It looks like we have a vision system-- then an object system- and then an importance system- and then a fear system (the amygdala).

The weird thing is- for people with broken amygdala's- they know the rattle snake is important- but not that it is dangerous. In other cases, people have said "I know this is bad" logically- and then done it anyway without being able to stop themselves.
Very interesting stuff.

As of now, they have human level agility and balance with plugged in humanoid robots, vision and dexterity to pick random mixed items out of bins faster than humans. The robot population is rising at a low exponent but the exponent is increasing.

Pacman

[manu0601]

According to the paper, computer remains below human average level for Pacman. Does that means AI failed to reverse engineer Pacman's ghost behavior, including their bugs?

Re:Pacman

[cfalcon]

That one is interesting because it took humans a ludicrous number of hours to figure out emergent patterns. It was trial and error (and the AI was deliberately not given an overly large amount of time on each game). Once the ninth key pattern was solved it became execution based. You could trivially code a machine to killscreen Pacman with no AI involved. Hell, you could probably do it with a very small Perl script- because humans already know the patterns that win.

It WOULD have been interesting to give it a lot of time on Pacman and see if it worked out NEW patterns. I'm pretty sure that the world record for Pac Man will now be "how long did it take you to killscreen Pacman with the max score", and it may be possible that the AI could work out a more optimal pattern.

press your luck

[Joe_Dragon]

Well it's been 20 years and watson is still pressing his luck.

Re: No AI is a Wizard

[sunderland56]

Well, that would be the point. Things like driving a car and performing surgery need both (a) computing capability and (b) a real-world interface. It would seem that taking a pinball machine, a handful of linear motors, a camera, and some compute power would be a much, much more useful real-world learning exercise.

When playing an electronic game, the exact same input will produce the exact same result; but on a mechanical pinball game, there will always be a slight variation - which makes the whole thing far more realistic.

Let it loose on the forex market

[Neo-Rio-101]

Screw Atari 2600 games. That's small fry gaming.

Let this thing run full tilt boogie on a MetaTrader platform, and see what you get.

Re:Let it loose on the forex market

[Katatsumuri]

Well, AI has been on forex and stock market for a long while. And if you mean this specific implementation, it would probably be no match for the specialized solutions.

Reinforcement from... where?

[esaulgd7195]

I got that the AI uses reinforcement learning, but how does it know whether it is doing well or badly? Even assuming ALL these games show score as big numbers on the screen: Did the AI come pre-equipped with a "layer" that parses the pixel data to read the score? Or did it learn to read numbers all on its own?? Because if it's the latter, that's pretty darn impressive, and I don't see any indication of the former on the article or the paper.

Reminds me of a programming competition ....

[Ihlosi]

Reminds me of a competition to program a computer player for the classic Asteroids a few years ago.

The best entries, however, didn't rely on AI, but on the fact that the RNG of the arcade game isn't random. Once the Asteroids-bot figured out the internal state of the RNG, it could basically use hyperspace to make targetted jumps (and never one that lead to the destruction of the ship), shoot at asteroids that haven't appeared yet and various other tricks. It was very impressive to watch one of these bots in action.

It did well for about 50% of the tested games

[MightyDrunken]

Of the 49 games tested there were about half which it did not do as well as a human player. They rated the performance of the AI against random play which equals 0 and a fairly skilled human player at 100%. The games the DQN agent did poorly at were:

• Montezuma's revenge 0%
• Private Eye 2%
• Gravitar 5%
• Frostbite 6%
• Asteroids 7%
• Ms. Pac-Man 13%
• Bowling 14%
• Double Dunk 17%
• Seaquest 25%
• Venture 32%
• Alien 42%
• Amidar 43%
• Zaxxon 54%

• ...

It would be interesting to compare the games it did well at Vs those it did poorly at. Unfortunately I do not know my Atari games well enough to comment,

Zork

[itzly]

Now let's have an AI beat Zork.

http://thcnet.net/zork/index.p...

Breaking News: Computer pattern recognition ... ?

[fygment]

The computer mostly figures out the game design jsut like human players, and couples that with super fast reaction time.

The latter is the only reason it can beat the best human players.

The bigger story is: what games did it suck at and why?

new strategy

[slashmydots]

I will always beat the computer because I have more quarters :D

But did it enjoy playing them?

[neo-mkrey]

If not, then it is not intelligence, but just one program beating another program.

Source Code

[Graydyn Young]

How they did this is a lot more interesting than the result. They're calling it a DQN, and the implications for automation tasks are awe inspiring.

Here is a link to the source:

https://sites.google.com/a/dee...

Re:Breaking News: Computer pattern recognition ...

[Graydyn Young]

The games it sucked at are ones where progress is not tied to score. It uses a Q learner, so it relies upon having a numeric metric for success.