Slashdot Mirror
New AI Is Capable of Beating Humans At Doom (denofgeek.com)
An anonymous reader quotes a report from Den of Geek UK: Two students at Carnegie Mellon University have designed an artificial intelligence program that is capable of beating human players in a deathmatch game of 1993's Doom. Guillaume Lample and Devendra Singh Chaplot spent four months developing a program capable of playing first-person shooter games. The program made its debut at VizDoom (an AI competition that centered around the classic shooter) where it took second place despite the fact that their creation managed to beat human participants. That's not the impressive part about this program, however. No, what's really impressive is how the AI learns to play. The creator's full write-up on the program (which is available here) notes that their AI "allows developing bots that play the game using the screen buffer." What that means is that the program learns by interpreting what is happening on the screen as opposed to following a pre-set series of command instructions alone. In other words, this AI learns to play in exactly the same way a human player learns to play. This theory has been explored practically before, but Doom is arguably the most complicated game a program fueled by that concept has been able to succeed at. The AI's creators have already confirmed that they will be moving on to Quake, which will be a much more interesting test of this technologies capabilities given that Quake presents a much more complex 3D environment.
Bring on your pesky AI. What I don't break with the hammer, I will break with my urine. Bzzzt!!!
We already have computer programs beating humans, it's called aim bots!
Well I for one welcome our new 2d overlords..
That's the *only* way you could come up with to get info about the game state to the bot? Aren't there typically a billion cheaty "enemy radar" type mods for FPSes that they could've adapted for their use? Filter out info not in the player's field of vision if that's a criteria.
It's funny to see some mainstream press outlets freak about about this like it was Skynet when three is already so much anti-layer going on in games already...
Many games have creatures that hunt players, usually programmed with a somewhat limited view of the entire internal game world just as this AI has only the screen to understand where objects are.
So I guess I'm missing what is really new about this, I assume some in-game AI in a few games somewhere has already had behavior programmed by some kind of learning mechanism.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
this is a video game, after all...
A bot with the rail gun in Quake is really really hard to beat especially on that open space map with all the jump pads. No AI needed, just a decent aim and POW! You are thrown into space and turned to giblets at the boundary.
CAn a human player even stand up against a ai that has aimbot?
Does the AI even 'hear' sounds like a human does or are the audio cues directly fed into the ai, meaning that the ai has far better ability distinguishing player sounds above the background noise to know where the other player is.
I would have thought AI had already beaten us at those games long before beating us at chess or go. Note sure if I am supposed to feel good as a human or bad as a programmer now.
Aim bots.
You totally miss the point: it's trivial to write a perfect bot that hooks into the game's internals and always wins. It's difficult, and more generally applicable, to make a bot that learns to play by watching only the same info the human players get: the screen buffer.
So the title touts up the ability to beat human players but dismisses it as an aside in the blurb? WTF?
Anyway, I have to agree that being able to beat a human player isn't a big deal. In CS:S a bot can somehow headshot a player falling off a building at 120 feet but can't manage to toss a grenade through a window with any amount of success. Having a bot being able to beat a human's ass isn't hard. What's hard is to make a bot that can beat a human's ass and still be fallible enough to keep it fun. What most bots do in multiplayer games would get you banned on any server as a hack.
Can I rig Call of Duty with an AI auto-pilot plug-in, and just sit back and watch it steam-roll over all the sucker humans in the game? If I play an online game like Overwatch and get smeered over and over by an opponent(s) with perfect aim and lightning-quick moves, will I just assume someone's introduced a bot into the game and I'm wasting my time with my hopelessly inferior carbon-based reflexes? Gaming may need its own version of the Butlerian Jihad.
Take it easy, Charlie, I've got an Angle...
Do you want Skynet?! Because this is how you get Skynet!
Anons need not reply. Questions end with a question mark.
The scary part is not an AI bot beating you in a game.
The very scary part is an AI bot (with physical presence and simple vision) beating, maiming, or killing you into submission when you and [hundreds, thousands, millions] of your fellow humans rebel against an oppressive, intrusive state that uses these bot to control the "rabble", justified by "Law and Order".
- Leonard
This is a good achievement. I'll be really impressed when an AI like this can play Portal.
Until's middle finger hurts from pressing the "W" for five hours straight, I will not be impressed. (yes I did this in 1994).
It will be interesting to see how future games develop to keep them fun for humans in an AI-filled world.
Imagine your AI setup gets to the point where it truly has the same input, not needing to be directly fed the screenbuffer but can use a camera pointed at your monitor. Suddenly current anti-cheating technologies mean nothing, and enough people using these would quick ruin a game.
They've had Doombots for years.
You should continue to feel good as a human. Humanity needs that. The other posters will want to tell you that it's not that an AI beat a human, but HOW it did: By learning. I'm taking a different approach: It's not about the performance of the AI. It's all about how playing the game makes you FEEL. Sure, the AI got the most kills, but what KIND of kills, and at what cost? Did it seek out and kill the only players who were out of ammo, and low on health? Did it ever sacrifice itself to protect an asset? Deathmatch was designed for humans to play against humans. So you can choose co-op on a whim to overcome a more powerful enemy, and then betray your allies at the last second. The AI cannot experience that.
So go ahead. Feel good as a human. And you can feel good for the fact that you know that the AI can NEVER feel good.
Until a computer is playing the same way as a human - ie camera onto screen and using a mouse/keyboard - it all seems contrived.
We would have both died in the radioactive waste. I never mastered that game, and was so utterly hopeless at Quake (and the zillions of Quake clones that came after it) that I would be more useful at teaching the AI how not to play that game.
Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
There's one glaring problem with this. The bot is good enough to beat a human. Most humans don't play Doom very well. If it beat a well known good player like Ocelot, Sedlo or Johsen, then it would be impressive. It's similar to writing a chess AI that can beat a human. This was done 30 years ago, but can that AI beat a grand master? Judging by the articles, the headline is somewhat misleading.
Doom may seem simple compared to Quake, at least superficially, but Doom features the BFG 9000 which a good player can do some fairly impressive things with, that would be VERY hard to deduce from simply observing. How the BFG worked wasn't really worked out in full detail until the source code was released. The BFG9000 is probably one of the most complex FPS weapons in any mainstream game. Then there are techniques like wall running, bumping, silent BFG shots etc. Knowing about these and when they are of use, can give a player a huge edge. Can the bot discover, use and master this? Such techniques are vital on the most common deathmatch maps, map01 and 07 in Doom 2.
Doom deathmatch can also be played in altdeath mode, typically map11 or maybe map16 are used for this type of play. This introduces many new skills, and downplays other. It is a rather different experience. Does the bot handle this? Navigating the 3d space of map11 is a lot more complicated than map07, which is basically flat. Figuring out the map, teleporters, secret doors, trigger lines that activate elevators, etc is pretty complicated stuff.
Given phrases like "Their agent, he said, “was ducking most of the time and thus was hard to hit.” I suspect a good human player would outskill the bots here. From the ViZDOOM paper (https://arxiv.org/abs/1605.02097) "we test the environment by trying to learn bots for two scenarios: a basic move-and-shoot task and a more complex maze-navigation problem."
When it comes to singleplayer, I would love to see bots play better than Henning in his 30nm run in 29:39, https://www.youtube.com/watch?...
Rest in peace Malin "looxn" Kristiansen. We miss you...
...like all bots do.
All the way back in the original Quake there was a really nice learning AI written in quake C. One version allowed you to add practice bots to work on your deathmatch strategy.
Similar to the AI described in this article, the AI in this mod was ignorant of the map and had no preset patterns. It learned by doing. So as you began playing against them they were easy kills in the early rounds. They'd often just stand there and get shot. And they couldn't hit the broad side of a barn.
But they learned the map. And they learned your moves. And within a few rounds you'd be lucky to stay alive long. And finally they would learn enough to get you every time. They'd know which direction you were going to dodge before you did. And they kept track of every resource in the game and all of the respawn times, so they'd deny you any ammo or health by timing their movements perfectly to collect all spawns instantly.
It was very cool.
Then the guy who wrote it used his AI to replace the original game AI for all of the enemies. Wow. It made the game into an entirely different experience.
After about a half-level, the enemies would learn to avoid you, go out and recruit all the bad guys from the level and return in force. After a couple of more levels they'd learn to ambush, flank and surround you. They'd team up their fire, so you'd dodge a fireball to the left and strafe right into another fireball.
It was really interesting, but ultimately unplayable. It really gave me an appreciation of the level of "balancing" that goes into creating a proper game AI. It certainly isn't about the same thing as making a chess AI that can beat Kasparov. It requires a great deal of work to make the enemy realistic and interesting and difficult but ultimately beatable.
Unless it also uses an aim bot, it isn't winning vs rail gun wielding aim bot players.
God spoke to me
It's difficult, and more generally applicable, to make a bot that learns to play by watching only the same info the human players get
That's not what's happening.
We've seen similar bullshit headlines like this before that imply that the program learned to play the game just from watching the screen. That couldn't be farther from the truth.
Prepare yourself for some serious disappointment and read the paper.
Required reading for internet skeptics
I don't buy that at all... the computer in this case is still learning through visual feedback only. If you replaced the screen buffer with a camera, the neural net would very quickly learn to ignore the areas outside of the monitor. Similarly, using a mouse and keyboard would be more of an accomplishment for the hardware engineers rigging up the appropriate servos than for the AI learning to use them.
They took the same reinforcement learning techniques from the Atria 2600 AI and applied them to Doom. It worked. Next they plan to do it on Quake. Let me give you a hint for the future: It'll work.
The theory is sound, it'll eventually learn to do whatever goal you program it to reach. This isn't new AI research, it's confirming things we already know will work do work. That's something that still needs to be done, but it's not very noteworthy nor does it advance the state of the art.
Also, they gave it specific training scenarios. They didn't just let it lose on a map. Basically that just reduces the training time, but it's an important bit of info since it means the AI has more human biases in it due to it being trained on more human chosen goals.
You should buy that.
Until there's a link between brain and game to manipulate the controls, the Script has an unfair advantage. I have to use a keyboard, the script doesn't.
_ _ _ Go for the eyes Boo! GO FOR THE EYES!
Why is the skynet tag missing?
"In other words, this AI learns to play in exactly the same way a human player learns to play."
Just because it has literally one similarity to how humans learn to play doesn't mean it learns exactly the same way as humans.
Exactly. All of these things introduce lag and imprecision. Do I doubt it can be done eventually ? No. But I want to see a gamer bot do it. Not code that has zero second input lag (plus a half frame real advantage no ?)
Plus he has to have his mother yelling at him while he plays.
we need an undo button
But those are basically trivial. Get a good quality camera and throw some hefty processors in and you can pretty well recreate the GPU buffer. Nothing novel at all. Similarly getting the AI to control the mouse/keyboard: throw some fast actuators and motor control and your done. The AI is the hard bit, not these.
The AI bot never misses a shot, or is it programed to miss? Does it learn how much is should miss by watching other players? "beep, I should miss at 80% rate, adjusting fire to miss"
We've seen similar bullshit headlines like this before that imply that the program learned to play the game just from watching the screen. That couldn't be farther from the truth.
What bullshit headlines, like "New AI is capable of beating humans at Doom"? That is quite directly true, and says so little you can't argue it is bullshit.
Or do you mean bullshit summary instead of headline? Because the summary is also quite direct and true. "What that means is that the program learns by interpreting what is happening on the screen as opposed to following a pre-set series of command instructions alone. " This is exactly what was done, it wasn't just hard coded to do stuff, but went through a learning process where it was fed only an objective and the screen buffer. Yes, it was programmed with how the game work and yes it had an objective programmed, but the summary doesn't say otherwise.
That couldn't be farther from the truth.
How could you possibly argue it is that wrong, when that is what is described in the paper? Are you suggesting the paper lied too? Or are you just upset that the objective function is not counted as part of the input? In that case, you're arguing a much subtler issue, and the fact still remains the program depends upon the screen buffer as an input. Disagreeing on subtleties like that is not "couldn't be farther from the truth" and saying such is disingenuous, as if you are hoping to convince people through display of confidence instead of display of actual information.
Prepare yourself for some serious disappointment and read the paper.
Yes, read the actual paper, and see how some of the things you've posted multiple times is just flat out false.
Blake Stone is calling your bluff, long before Doom was even around.
cant hide from.... Dukebot.
It is inevitable.
Good part is - we short circuit the entertainment industry like this means we win some private time we can do something else....
That's not what's happening.
The summary, article, and paper all say you are wrong.
OK, I've read it. It's a deep network. Input is the screen buffer. Output controls the game. I think that's pretty damn impressive. What's disappointing to you exactly?
Ahh, my apologies for the previous post.
You've posted your lies 7 times in this thread so are clearly just purposefully trolling.
Go ahead and disregard all the facts and don't bother replying to me.
Valve's NextBot implementation has simulation for input lag and even mousepad area, such that the bot has aiming delays when it reaches the edge of its virtual mousepad to pick up the mouse and recenter it.
I'd argue that still isn't equivalent. A human can focus on a very small area. Your peripheral vision only really detects movement which you can then turn to focus on. A AI would be able to focus on 100% of the screen at a time. Therefore they should be able to spot details and react quicker than a human. Their timing is also more exact so they'll be able to turn to aim precisely far more accurately than a human, except possibly a professional player.
Not quite, unless you're talking actually modifying hitpoints via memory.
Around 1999-2000 there was a bot for Quake 2 called Ratbot. I beat a people who were using it because I had hundreds of hours of map knowledge and tactics. Raw aim didn't mean anything. You could tell they were using a bot because it had this handy feature. Whenever anyone said "ratbot" in chat, the bot would respond with "I am ratbot!"
I'd think direct access to the screen buffer where 100% of it is in focus is a bigger advantage. You want to use a servo-mounted camera with limited detail in its peripheral vision to model the human vision system to even out that advantage.
And the next round: developers add keyboard and screen to the AI and it still beats the human. Then the humans tell "but it is our brain that is creating the lag, that is so unfair, destroy the overlords!".
Video or it didn't happen.
Looks like someone didn't read the paper. It's free, so it's not like it'll cost you anything to read it.
Here's a helpful hint: When you hear than an AI "taught itself" anything, it's guaranteed to be bullshit for the foreseeable future. Simply things like determining success criteria are far-beyond what so-called AI can actually do that it's laughable to say it "taught itself" anything.
They trained a NN just like we've been training NN's for ages. It's about as interesting as an undergrads NN project.
Required reading for internet skeptics
What about my post was false?
This toy did not "teach itself" to play the game using only feedback from the screen buffer. That is a very simple and obvious fact. How you can believe otherwise is astonishing. Read the damn paper.
I know. No one likes to have their silly fantasies shattered by the cold light of reality, but enough is enough. Face facts, read the paper, and get over it. The last thing we need is another technology-based religion like the "less wrong" group or the Kurzweil cultists..
Required reading for internet skeptics
None of the headline, the summary, the paper or the post you replied to said "taught itself".
"I don't buy that at all... the computer in this case is still learning through visual feedback only."
Instantaneous visual feedback. Humans have serious input and output lag in comparison. Give the bot the exact same latency we've got and see how it does.
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
sounds like the base logic for a terminator. Stick the same code in a robot with a gun and we're all screwed.
No one likes to have their silly fantasies shattered by the cold light of reality, but enough is enough.
Yeah, it really sucks to see someone keep posting the same thing over and over again, being as obtuse and dense as possible, as if trying to protect themselves from admitting they are wrong.
Face facts, read the paper, and get over it.
Yeah, said person needs to read the paper and get over how wrong they keep saying is. They keep talking about the paper, but don't even read the parts that have been directly quoted in comments. But, the paper (and quotes) is at least there for others to read, so it should be pretty obvious to nearly everyone else who is incorrect.
The last thing we need is another technology-based religion like the "less wrong" group or the Kurzweil cultists..
Yes, we can now see this as a religious argument, which might help explain the blind faith in direct contradiction to facts. Can't risk letting those cultists win, even if they aren't actually here.
Here's a helpful hint: When you hear than an AI "taught itself" anything, it's guaranteed to be bullshit for the foreseeable future.
Except the only place we hear that with this story is in your strawman arguments.
How is this not using the screen buffer?
I read the internet for the articles.
Who said it didn't use the screen buffer?
My point was that it didn't learn to play the game exclusively using feedback from the screen buffer, like the magical thinkers here seem to believe.
Required reading for internet skeptics
SOS...They are training Skynet to shoot people. Must Stop them
So why hasn't it been solved yet? I don't see I physical bots doing this. No.
You posted as AC but signed your post anyway? What is wrong with you? Get over yourself.
I PLAY NIGHTMARE IN MY SLEEP
I STRAIGHT UP GOT A CARBON FIBER HAND DESIGNED BY NASA FOR WASD MOVEMENT
RIP N' TEAR!
(Please don't use so many caps, it's like YELLING)
(Please don't use so many caps, it's like YELLING)
(Please don't use so many caps, it's like YELLING)
(Please don't use so many caps, it's like YELLING)
From the actual paper: "Game Settings: The game’s state was represented by
a 120 × 45 3-channel RGB image, health points and the
current tick number (within the episode). Additionally, a kind
of memory was implemented by making the agent use 4 last
states as the neural network’s input. The nonvisual inputs
(health, ammo) were fed directly to the first fully-connected
layer."
So yes, they gave it two integers from the internal workings of the game, solely because the player gets those values from the visuals, but they are using such a lo-res version of the visuals that the program can't get that info from it. Seriously, the AI is getting it's information from the screenbuffer, albeit a very very lo-res screenbuffer (because each pixel is an input to a neural net, they don't want to overload the network with too many inputs).
The last second to last sentence of the article says, "...the developers claim that their AI won one of the competition games by learning to duck and therefore making itself much harder to hit." What version of Doom are they playing? As a teenage I played countless hours of Doom 1 & 2 and I don't remember a duck/crouch button.
Great post
Finding Source - join the movement www.findingsource.org