Slashdot Mirror

← Back to Stories (view on slashdot.org)

New AI Is Capable of Beating Humans At Doom (denofgeek.com)

Posted by BeauHD on from the Skynet-approved dept.

An anonymous reader quotes a report from Den of Geek UK: Two students at Carnegie Mellon University have designed an artificial intelligence program that is capable of beating human players in a deathmatch game of 1993's Doom. Guillaume Lample and Devendra Singh Chaplot spent four months developing a program capable of playing first-person shooter games. The program made its debut at VizDoom (an AI competition that centered around the classic shooter) where it took second place despite the fact that their creation managed to beat human participants. That's not the impressive part about this program, however. No, what's really impressive is how the AI learns to play. The creator's full write-up on the program (which is available here) notes that their AI "allows developing bots that play the game using the screen buffer." What that means is that the program learns by interpreting what is happening on the screen as opposed to following a pre-set series of command instructions alone. In other words, this AI learns to play in exactly the same way a human player learns to play. This theory has been explored practically before, but Doom is arguably the most complicated game a program fueled by that concept has been able to succeed at. The AI's creators have already confirmed that they will be moving on to Quake, which will be a much more interesting test of this technologies capabilities given that Quake presents a much more complex 3D environment.

89 of 170 comments (clear)

  1. I can beat AI with a hammer. by zenlessyank · · Score: 1

    Bring on your pesky AI. What I don't break with the hammer, I will break with my urine. Bzzzt!!!

    1. Re:I can beat AI with a hammer. by wonkey_monkey · · Score: 3, Funny

      I will break with my urine. Bzzzt!!!

      If that's the sound you make when you pee, you should see a doctor.

      --
      systemd is Roko's Basilisk.
    2. Re:I can beat AI with a hammer. by zenlessyank · · Score: 1

      It is the sound of electronics shorting out.

      Paying a doctor to look at my dick is a dumbass idea.

  2. Doom, the game where you can't look up or down. by Anonymous Coward · · Score: 2, Funny

    Well I for one welcome our new 2d overlords..

    1. Re:Doom, the game where you can't look up or down. by lxs · · Score: 1

      Pioneered an industry? The game you're referring to is called Pong. The industry was in its third decade by the time Doom came along.

      Kids these days...

  3. Don't most games do this... by SuperKendall · · Score: 1

    It's funny to see some mainstream press outlets freak about about this like it was Skynet when three is already so much anti-layer going on in games already...

    Many games have creatures that hunt players, usually programmed with a somewhat limited view of the entire internal game world just as this AI has only the screen to understand where objects are.

    So I guess I'm missing what is really new about this, I assume some in-game AI in a few games somewhere has already had behavior programmed by some kind of learning mechanism.

    --
    "There is more worth loving than we have strength to love." - Brian Jay Stanley
    1. Re:Don't most games do this... by lgw · · Score: 4, Informative

      The point is that the AI learned to play the game from only screen data. No maps, no preset strategy, just visual data. So, it has to learn to recognize threats and obstacles, and what to do when it does.

      Beating humans is a good test, because humans are good at exploiting patterns, so shortcuts like always taking a fixed route wouldn't work for long.

      --
      Socialism: a lie told by totalitarians and believed by fools.
    2. Re:Don't most games do this... by Wraithlyn · · Score: 2

      There's a pretty big difference between a game AI (which is fed machine-centric game state information, and has an extensive pre-programmed ruleset) adapting marginally to a player's actions, vs learning to play (and master) the entire game via screen inspection.

      --
      "Mind, as manifested by the capacity to make choices, is to some extent present in every electron." -Freeman Dyson
    3. Re:Don't most games do this... by swb · · Score: 4, Informative

      In-game bots may be operating on a limited view, but they're operating on actual hard data in basically machine-usable form. What's impressive about this is that it learns from what's on the screen -- distances, obstacles, paths, its location are all learned from visual input.

      What I'm curious about is how adaptable their visual learning system is or whether it's extremely Doom specific. I'd also be curious at how long it took to learn to play. I'd also be curious what the learning curve was -- linear, non-linear, flat, steep or what.

    4. Re:Don't most games do this... by R3d+M3rcury · · Score: 2

      Right. So if you think you're going to "out-draw" the AI, you're probably SOL.

      Which means you have to out-think it. Snipe it, staying out of it's vision.

    5. Re:Don't most games do this... by lgw · · Score: 1

      Which is what makes it an interesting test of an AI. It's not like aimbots are new or anything, but an AI that humans fail to outsmart is something. Of course, the devil is in the details.

      --
      Socialism: a lie told by totalitarians and believed by fools.
    6. Re: Don't most games do this... by limaxray · · Score: 2

      This is what is called 'weak AI' - what you're referring to is 'strong AI', and we're no where near that point yet. Yes, they probably had to hardcode the objective of the game and some core rules, otherwise it would have no idea what to do. But even natural intelligence has this, we usually call it instinct. The point though is that it can actually learn how to play from there and adapt to improve its performance. Also understand that there's probably an ton of computation to do this, so the reaction time probably isn't as fast as you'd think, and I'd imagine a well trained weak AI could beat any human player even if it's reaction time was handicapped. It may not be as complicated as the AI in a fully autonomous car, but it's still impressive.

      The real problem will be this will make detecting cheats even harder. You could run the game in a VM or a dedicated machine and have your AI play for you using a virtual frame buffer and input devices - the game will have no idea the OS isn't displaying it to a real display and being driven by a real mouse/keyboard. You could do the same thing with a console by feeding the HDMI into a capture card and hacking a controller to be controlled by your AI computer.

    7. Re:Don't most games do this... by maugle · · Score: 1

      They say they're moving on to Quake but I can guess that, because of how machine learning generally works, the AI they've trained for Doom will be utterly helpless at Quake until fully retrained.

    8. Re:Don't most games do this... by sg_oneill · · Score: 1

      That is a very false statement.

      Or you could actually read the article, I know I know its hard so I did it for you. From the abstract of the journal article;-

      The software, called ViZDoom, is based on the classical first-person shooter video game, Doom. It allows developing bots that play the game using the screen buffer. ViZDoom is lightweight, fast, and highly customizable via a convenient mechanism of user scenarios. In the experimental part, we test the environment by trying to learn bots for two scenarios: a basic move-and-shoot task and a more complex maze-navigation problem. Using convolutional deep neural networks with Q-learning and experience replay, for both scenarios, we were able to train competent bots, which exhibit human-like behaviors. The results confirm the utility of ViZDoom as an AI research platform and imply that visual reinforcement learning in 3D realistic first-person perspective environments is feasible.

      So yes it is in fact learning to play from the screen data.

      --
      Excuse the Unicode crap in my posts. That's an apostrophe, and slashdot is busted.
    9. Re:Don't most games do this... by Anonymous Coward · · Score: 1

      > The bot has access to the depth buffer.
      Doom doesn't /have/ a depth buffer. And the paper is quite explicit that the only input is an RGB framebuffer.

    10. Re:Don't most games do this... by Dutch+Gun · · Score: 3, Informative

      Ehh, a first-person shooter is not really all that great a test for AI. Being a videogame programmer, and one who has programmed his share of AI in commercial videogames, the trick is not to create unbeatable AI, but to create an interesting experience for the player. Granted, we use a lot of internal data structures to assist the AI (like for navigation), and we obviously approach things from a completely different angle, but we also have to drastically handicap the AI's responsiveness and aim in shooter-type games.

      Remember, it's trivial for a computer to paint a bulls-eye between your eyes from 500 yards out with a sniper rifle no matter how you're moving or hiding. It's still reasonably easy even with true projectiles, as the AI can calculate perfect flight trajectories so that rocket will precisely intercept a moving target. An AI can't get disoriented, or confused, and has near-instant reflexes that no human can match.

      One trick I've used for shooter bots is to incorporate virtual springs attached to the bot's targets, helping to throw off their aim according to how the target is moving. You can also dynamically adjust the target spring tension or based on other factors, like difficulty scaling, whether the AI agent is running, jumping (throwing off his own aim), etc. That sort of thing, along with adding blind spots, artificial reaction times, intentional mistakes, and so on, are the things you have to do to keep the bots from kicking the crap out of human players just because they could instantly headshot you from across the map otherwise.

      Don't get me wrong... this is a neat little project. But beating humans in a shooter where fast reflexes and perfect aim dominate isn't really the end-all and be-all of AI tests, because our strengths and weakness are in different areas than for computer-based opponents.

      --
      Irony: Agile development has too much intertia to be abandoned now.
    11. Re:Don't most games do this... by Anonymous Coward · · Score: 3, Informative

      From TFP

      b) Game Settings: A state was represented by the most
      recent frame, which was a 60 × 45 3-channel RGB image.
      The number of skipped frames is controlled by the skipcount
      parameter. We experimented with skipcounts of 0-7, 10, 15,
      20, 25, 30, 35 and 40. It is important to note that the agent
      repeats the last decision on the skipped frames.

      and

      b) Game Settings: The game’s state was represented by
      a 120 × 45 3-channel RGB image, health points and the
      current tick number (within the episode). Additionally, a kind
      of memory was implemented by making the agent use 4 last
      states as the neural network’s input. The nonvisual inputs
      (health, ammo) were fed directly to the first fully-connected
      layer. Skipcount of 10 was used.

      The only thing they gave the AI for the experiments were the screen image, the time, and in the second case, the health.

    12. Re:Don't most games do this... by Anonymous Coward · · Score: 4, Insightful

      Modern re-implementations of Doom and Wolfenstein have a depth buffer. The original do not.

      Doom used BSPs to sort walls from front to back and a scanline like algorithm that guaranteed no overdraw, so there is no use for a depth buffer. Sprites were sorted from back to front, and occlusion with other sprites was handled by painter's algorithm, and occlusion with walls was done by using a list of drawn walls and direct intersection testing. Again, no use for a depth buffer.

      The algorithms in the original Doom wasn't perfect, and there are some severe artifacts to find if you know where to look, especially if sprites where near a step. Also, re-implementing Doom in something with a depth buffer like OpenGL either requires modification to how sprites are rendered, or you will get a visual difference from the original game. In Doom, the lack of clipping of sprites against the floor and ceiling is usually easy to handle by just shifting of sprites vertically, for example. But some other games made with the Doom engine depended more heavily on the lack of such clipping in the perspective used by the artists creating the sprites, and it can take some effort to get OpenGL to not clip things clip when that weren't clipped in the original game.

    13. Re:Don't most games do this... by Maritz · · Score: 1

      So I guess I'm missing what is really new about this

      Looks like.

      --
      I do not want your cheap brainburning drugs. They are useless for work. And I am a working man today.
    14. Re:Don't most games do this... by Maritz · · Score: 1

      You keep saying that, and you seem to be getting flatly contradicted. How come?

      --
      I do not want your cheap brainburning drugs. They are useless for work. And I am a working man today.
    15. Re:Don't most games do this... by Big+Hairy+Ian · · Score: 1

      It has interesting implications for software automation

      --

      Build a Man a Fire, and He'll Be Warm for a Day. Set a Man on Fire, and He'll Be Warm for the Rest of His Life.

    16. Re:Don't most games do this... by rxmd · · Score: 1

      Actually the paper states the opposite. From page 4: "3) Depth Buffer Access: ViZDoom provides access to the renderer’s depth buffer (see Fig. 3), which may help an agent to understand the received visual information. This feature gives an opportunity to test whether the learning algorithms can autonomously learn the whereabouts of the objects in the environment. The depth information can also be used to simulate the distance sensors common in mobile robots."

      --
      As a state gets corrupt, its laws multiply; the most corrupt states have the most numerous laws. (Tacitus, Annales 3:27)
    17. Re: Don't most games do this... by religionofpeas · · Score: 1

      Every AI related problem that has been solved automatically becomes 'weak AI'.

    18. Re:Don't most games do this... by Bob+the+Super+Hamste · · Score: 1

      My bet is mental deficiency.

      --
      Time to offend someone
    19. Re:Don't most games do this... by lgw · · Score: 1

      You've yet to explain your point. It learned to play the game, which is in fact somewhat like the average undergrads NN project (perhaps a bit more elaborate in scope). And ...?

      --
      Socialism: a lie told by totalitarians and believed by fools.
    20. Re:Don't most games do this... by lgw · · Score: 1

      You've missed the point I think. This isn't an aimbot.

      This is a neural net that needed to learn on its own what an enemy looked like on the screen, and what obstacles were, and how to move around them, and so on. The basic "strategy" logic of the bot was fairly simple: a maze explorer, plus an aimbot. The interesting part was doing that given only the screen buffer to work with (well, they cheated a bit by giving the health explicitly).

      --
      Socialism: a lie told by totalitarians and believed by fools.
    21. Re:Don't most games do this... by Anonymous Coward · · Score: 1

      The novelty is not in the AI itself but rather that it's only being fed the exact same video data that a human player sees. Essentially applying the same technology that a self-driving car uses, mashed up with a neural net and instructed to learn to play.

    22. Re:Don't most games do this... by narcc · · Score: 1

      Yes, it plays with only screen data. It did not learn how to play using only that data. Just like a zillion similar NN projects before it.

      Sorry if this hurts your religious beliefs, but reality is indifferent to your fantasies.

      --
      Required reading for internet skeptics
    23. Re:Don't most games do this... by narcc · · Score: 1

      Nonsense. In Wolfensten, for example, a 1D depth buffer is NECESSARY to paint partially eclipsed sprites, and to avoid painted completely eclipsed sprites.
      This is how it works: On the first pass, a ray is cast for each vertical column on the display. When a wall is encountered, the distance to the wall and the position of the ray on the way is determined and used to draw a vertically-centered column scaled by distance. The distance is stored in a buffer the width of the screen. (A Depth Buffer!)

      On the second pass, sprites are scaled and drawn. For each column of the sprite, the buffer is consulted. If the depth of the sprite is greater than the depth of the buffer, the column is not drawn.

      How the hell do you think it worked? Magic?

      --
      Required reading for internet skeptics
    24. Re:Don't most games do this... by narcc · · Score: 1

      It's not difficult. The implication here, from the article and summary, is that the program learned to play the game using only feedback from the display buffer. That is, quite obviously, false.

      As I pointed out earlier, it did not, and can not, determine success criteria. That is the assumption you see endlessly here, implicit in absurd statements like "the computer in this case is still learning through visual feedback only", "The point is that the AI learned to play the game from only screen data. No maps, no preset strategy, just visual data", "This bot was just shown a video of what's happening and then learned how to play exactly like a human player would.", and a host of other, similar, nonsense statements. That would indeed be an impressive accomplishment. That, quite obviously, didn't happen.

      This is, as I've said, no different than any other NN project. To claim otherwise is absurd.

      Why this is controversial is beyond me. It's not exactly complicated.

      --
      Required reading for internet skeptics
    25. Re:Don't most games do this... by lgw · · Score: 2

      You seem to be slicing something very fine that seems of minor importance, unless I'm missing something. When a human learns to play Doom, he starts knowing the success criteria, he knows upfront that the basic tasks are "maze explorer" and "move and shoot", and what the health bar is. Yet we still say he "learns to play Doom".

      From what I read, the bot learned the map, learned to move and shoot in an effective way, and had only the screen buffer and a health number as an interface with the game - not the additional side-channel info (map, player locations, etc) that someone writing a normal aimbot would use.

      --
      Socialism: a lie told by totalitarians and believed by fools.
    26. Re:Don't most games do this... by lgw · · Score: 1

      I guess I still agree with like "the point is that the AI learned to play the game from only screen data. No maps, no preset strategy, just visual data". We're arguing about what "learned to play the game" means. You seem to be objecting that "that part the computer can't do yet, that's the important part", which people have been saying about AI research for 50 years of progress.

      --
      Socialism: a lie told by totalitarians and believed by fools.
    27. Re:Don't most games do this... by narcc · · Score: 1

      The fact remains that statements like "the AI learned to play the game from only screen data" are completely false. It had far more than data from the screen buffer available to it.

      We're not arguing over what "learned to play the game" means, we're arguing over the claim that the program learned having access only to the screen buffer, which is completely false.

      Why does that matter? Because the uninteresting part, they trained a NN by traditional means, isn't what people are claiming. They're claiming that a much more difficult problem was solved.

      Take a look at this, disturbing, belief:

      This bot was just shown a video of what's happening and then learned how to play exactly like a human player would.

      That's an actual quote from someone here. You'll find similar statements all over this thread. That is, apparently, what people believe. You can not argue that this is true in any way. It's a completely false statement. The claim made here is so far removed from reality that I can't believe you think I'm splitting hairs!

      This isn't akin to a "God of the gaps" argument, nor are we talking about strong AI, consciousness, or anything like that. This is a bullshit article conning uninformed people in to believing (and spreading the false belief) that some advancement was made that has not, in fact, been made.

      Can you still stand behind the above quote? If so, how? If an actual advancement were made that allowed the program to learn to play solely from feedback from the display buffer, would you say that it had already been done, referencing the paper here? If not, why do you insist on promoting this absurdly false belief?

      --
      Required reading for internet skeptics
    28. Re:Don't most games do this... by mobby_6kl · · Score: 2

      Human newbies have a ton of context to determine what's success in the game. I think you're the one who seems to expect a Doom bot to have solved all of AI forever.

      This appears to be a significant improvement over something like Breakout or even Mario, and just because it was partially supervised, doesn't make it less of a progress.

    29. Re:Don't most games do this... by narcc · · Score: 1

      I think you're the one who seems to expect a Doom bot to have solved all of AI forever.

      No, I'm saying that the article and a lot of readers seem to believe that a significant problem has been solved, which, obviously, has not been solved.

      You seem to want to believe there is far more to this project than is actually there. Wishful thinking is fine, but don't pretend that it's reality.

      --
      Required reading for internet skeptics
    30. Re:Don't most games do this... by lgw · · Score: 1

      The (reasonable) claim isn't that the bot "learn[ed] to play solely from feedback from the display buffer", unless you add "exactly like a human would". A human would bring a lot of context before he started "learning". Sure, we can slice it very fine about whether a human could figure out what a health bar was, eventually, without knowing ahead of time, but then, perhaps that NN would too given a longer training time.

      The main thing is, if we go back to when Doom was new, the human already knew that dying was the failure state, that creatures attacking you are bad, and so on. When id wrote the game, they wrote it around those assumptions - it was as natural to the devs as the players. So, when something that looks like a demon (or an opposing player) shoots you with something that looks like a gun, you bring prior knowledge that that hurts - you can guess the function of the health bar pretty quickly from that prior knowledge.

      I dunno, perhaps some people though the bot learned to live in the game world the same ay a baby learns to live in the would, tabula rasa? That's silly, but I don't read the thread that way. Most people are asking how this is different from a classic aimbot, and the difference is it only gets the screen buffer, not the side channel (aimbots get all the other prior knowledge of failure states etc. too, you see).

      --
      Socialism: a lie told by totalitarians and believed by fools.
  4. no video? by spongman · · Score: 3, Insightful

    this is a video game, after all...

  5. Nice? by Carewolf · · Score: 2

    I would have thought AI had already beaten us at those games long before beating us at chess or go. Note sure if I am supposed to feel good as a human or bad as a programmer now.

    1. Re:Nice? by Narcocide · · Score: 1

      The novel part isn't that it can beat humans. As previously observed, that is a simple job for an aim-bot combined with some predefined, map-specific movement patterns. The novel part here is that it taught itself how.

    2. Re:Nice? by narcc · · Score: 1

      It did no such thing.

      --
      Required reading for internet skeptics
  6. They are called... by ark1 · · Score: 1

    Aim bots.

  7. Re:The screen buffer? Really? by Gibgezr · · Score: 5, Insightful

    You totally miss the point: it's trivial to write a perfect bot that hooks into the game's internals and always wins. It's difficult, and more generally applicable, to make a bot that learns to play by watching only the same info the human players get: the screen buffer.

  8. So... in a few years... by WheezyJoe · · Score: 4, Insightful

    Can I rig Call of Duty with an AI auto-pilot plug-in, and just sit back and watch it steam-roll over all the sucker humans in the game? If I play an online game like Overwatch and get smeered over and over by an opponent(s) with perfect aim and lightning-quick moves, will I just assume someone's introduced a bot into the game and I'm wasting my time with my hopelessly inferior carbon-based reflexes? Gaming may need its own version of the Butlerian Jihad.

    --
    Take it easy, Charlie, I've got an Angle...
    1. Re:So... in a few years... by houghi · · Score: 2

      I think I have choosen the wrong carrer path.
      First I worked in a factory where I was replaced by a a robot. Next I woreked at a food chain, where I was replaced by the self-checkout. Next I became an Uber driver and I am getting replaced by self driving cars, so I have invested in becoming a professional gamer and now THIS?

      --
      Don't fight for your country, if your country does not fight for you.
    2. Re:So... in a few years... by Gussington · · Score: 1

      Always thought it would be possible to build system with camera and servos to drive WASD keys and a mouse, that could be programmed for say the Dust 2 map in CSGO. Fire fights are generally in the same places on the map, so once it learns the background imagery it just has to shoot at anything different.
      It sounds pointless, but CSGO has a real world economy so this could be used for real financial gain.

    3. Re:So... in a few years... by randomlygeneratename · · Score: 1

      Isn't that already how it is? Play CoD for more than 5 minutes without someone being accused of hax...

  9. Re:Fucked up by Guspaz · · Score: 1

    Being able to beat a human isn't a big deal. Being able to do so while using the exact same inputs (key presses) and outputs (a picture of the screen) as a human is a big deal. Doom is definitely a simplified problem set (a given sprite only ever varies in scale and X/Y position), but it's still an impressive feat of machine vision and machine learning.

  10. Re:The screen buffer? Really? Consider... by Anonymous Coward · · Score: 1

    The scary part is not an AI bot beating you in a game.

    The very scary part is an AI bot (with physical presence and simple vision) beating, maiming, or killing you into submission when you and [hundreds, thousands, millions] of your fellow humans rebel against an oppressive, intrusive state that uses these bot to control the "rabble", justified by "Law and Order".

    - Leonard

  11. Not Impressed by Herkum01 · · Score: 2

    Until's middle finger hurts from pressing the "W" for five hours straight, I will not be impressed. (yes I did this in 1994).

    1. Re:Not Impressed by freeze128 · · Score: 2, Insightful

      Reconfigure your keyboard. You don't need to use WASD for doom. Try Right-Mouse for FORWARD instead. Your finger will thank you.

      Sorry for the 20-year late message.

    2. Re:Not Impressed by kackle · · Score: 1

      "Straight"? You know, you're allowed to release the "W" key once in a while...

    3. Re:Not Impressed by fuo · · Score: 1

      Doom doesn't have Jump, so S=Forward, Space=Back is better imo, it allows you rock forward and backwards easier since the same finger doesn't control both buttons.

    4. Re:Not Impressed by the+Gray+Mouser · · Score: 1

      I remember computer games that used WASD for movement in the 1980's.

  12. Ramifactions for the Future of Gaming by TranquilVoid · · Score: 4, Insightful

    It will be interesting to see how future games develop to keep them fun for humans in an AI-filled world.

    Imagine your AI setup gets to the point where it truly has the same input, not needing to be directly fed the screenbuffer but can use a camera pointed at your monitor. Suddenly current anti-cheating technologies mean nothing, and enough people using these would quick ruin a game.

    1. Re:Ramifactions for the Future of Gaming by Anubis+IV · · Score: 1

      Reminds me of an xkcd about reputation-based spam-blocking. Seems we might have a similar situation here, except that we end up with everyone benefitting from being able to play against the most enjoyable opponents all the time.

    2. Re:Ramifactions for the Future of Gaming by Dog-Cow · · Score: 1

      So you'll end up with bots as teammates and worse players as opponents.

    3. Re:Ramifactions for the Future of Gaming by Solandri · · Score: 1

      Then maybe game companies will stop trying to control how everyone plays by forcing them to play on company servers only. And they'll put LAN gameplay capability back into games (which ironically was the only way multiplayer Doom could be played), where you can physically confirm who you're playing with and that they are not cheating.

    4. Re:Ramifactions for the Future of Gaming by OpenSourced · · Score: 1

      Suddenly current anti-cheating technologies mean nothing, and enough people using these would quick ruin a game.

      Contrariwise, imagine a world where you can play in your computer against any number of AI opponents, regulated to the level that makes the game interesting to you. Then you don't need other people and cheating becomes meaningless, as it should be in a game.

      --
      Rome taught me patience and assiduous application to detail. Virtues which temper the boldness of great, general views.
    5. Re:Ramifactions for the Future of Gaming by Big+Hairy+Ian · · Score: 1

      This goes beyond gaming. Imagine when this AI gets turned on all those mundane jobs you do around the office! We're going to have a lot of unemployed fax monkeys!

      --

      Build a Man a Fire, and He'll Be Warm for a Day. Set a Man on Fire, and He'll Be Warm for the Rest of His Life.

  13. Marvel's way ahead of you.... by Chris+Mattern · · Score: 1

    They've had Doombots for years.

  14. It's not about the performance by freeze128 · · Score: 1

    You should continue to feel good as a human. Humanity needs that. The other posters will want to tell you that it's not that an AI beat a human, but HOW it did: By learning. I'm taking a different approach: It's not about the performance of the AI. It's all about how playing the game makes you FEEL. Sure, the AI got the most kills, but what KIND of kills, and at what cost? Did it seek out and kill the only players who were out of ammo, and low on health? Did it ever sacrifice itself to protect an asset? Deathmatch was designed for humans to play against humans. So you can choose co-op on a whim to overcome a more powerful enemy, and then betray your allies at the last second. The AI cannot experience that.

    So go ahead. Feel good as a human. And you can feel good for the fact that you know that the AI can NEVER feel good.

  15. Good thing it didn't go up against me by damn_registrars · · Score: 1

    We would have both died in the radioactive waste. I never mastered that game, and was so utterly hopeless at Quake (and the zillions of Quake clones that came after it) that I would be more useful at teaching the AI how not to play that game.

    --
    Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
  16. Impressive, but flawed by zokum · · Score: 2

    There's one glaring problem with this. The bot is good enough to beat a human. Most humans don't play Doom very well. If it beat a well known good player like Ocelot, Sedlo or Johsen, then it would be impressive. It's similar to writing a chess AI that can beat a human. This was done 30 years ago, but can that AI beat a grand master? Judging by the articles, the headline is somewhat misleading.

    Doom may seem simple compared to Quake, at least superficially, but Doom features the BFG 9000 which a good player can do some fairly impressive things with, that would be VERY hard to deduce from simply observing. How the BFG worked wasn't really worked out in full detail until the source code was released. The BFG9000 is probably one of the most complex FPS weapons in any mainstream game. Then there are techniques like wall running, bumping, silent BFG shots etc. Knowing about these and when they are of use, can give a player a huge edge. Can the bot discover, use and master this? Such techniques are vital on the most common deathmatch maps, map01 and 07 in Doom 2.

    Doom deathmatch can also be played in altdeath mode, typically map11 or maybe map16 are used for this type of play. This introduces many new skills, and downplays other. It is a rather different experience. Does the bot handle this? Navigating the 3d space of map11 is a lot more complicated than map07, which is basically flat. Figuring out the map, teleporters, secret doors, trigger lines that activate elevators, etc is pretty complicated stuff.

    Given phrases like "Their agent, he said, “was ducking most of the time and thus was hard to hit.” I suspect a good human player would outskill the bots here. From the ViZDOOM paper (https://arxiv.org/abs/1605.02097) "we test the environment by trying to learn bots for two scenarios: a basic move-and-shoot task and a more complex maze-navigation problem."

    When it comes to singleplayer, I would love to see bots play better than Henning in his 30nm run in 29:39, https://www.youtube.com/watch?...

    --
    Rest in peace Malin "looxn" Kristiansen. We miss you...
  17. There was a nice AI bot mod in Quake C by Cytotoxic · · Score: 4, Interesting

    All the way back in the original Quake there was a really nice learning AI written in quake C. One version allowed you to add practice bots to work on your deathmatch strategy.

    Similar to the AI described in this article, the AI in this mod was ignorant of the map and had no preset patterns. It learned by doing. So as you began playing against them they were easy kills in the early rounds. They'd often just stand there and get shot. And they couldn't hit the broad side of a barn.

    But they learned the map. And they learned your moves. And within a few rounds you'd be lucky to stay alive long. And finally they would learn enough to get you every time. They'd know which direction you were going to dodge before you did. And they kept track of every resource in the game and all of the respawn times, so they'd deny you any ammo or health by timing their movements perfectly to collect all spawns instantly.

    It was very cool.

    Then the guy who wrote it used his AI to replace the original game AI for all of the enemies. Wow. It made the game into an entirely different experience.

    After about a half-level, the enemies would learn to avoid you, go out and recruit all the bad guys from the level and return in force. After a couple of more levels they'd learn to ambush, flank and surround you. They'd team up their fire, so you'd dodge a fireball to the left and strafe right into another fireball.

    It was really interesting, but ultimately unplayable. It really gave me an appreciation of the level of "balancing" that goes into creating a proper game AI. It certainly isn't about the same thing as making a chess AI that can beat Kasparov. It requires a great deal of work to make the enemy realistic and interesting and difficult but ultimately beatable.

    1. Re:There was a nice AI bot mod in Quake C by Gussington · · Score: 1

      All the way back in the original Quake there was a really nice learning AI written in quake C...But they learned the map. And they learned your moves. And within a few rounds you'd be lucky to stay alive long. And finally they would learn enough to get you every time. They'd know which direction you were going to dodge before you did. And they kept track of every resource in the game and all of the respawn times, so they'd deny you any ammo or health by timing their movements perfectly to collect all spawns instantly.

      It was very cool.

      I used to play quite A_LOT of Quake and Quake 2 and 3 and don't recall this. Bots have always been a split between extremely useless or extremely impossible (ie they know where you are and head-shot you before they've seen you). Even now in the latest games the bots are easily identifiable for the same reasons, either too crap or impossibly good.

    2. Re:There was a nice AI bot mod in Quake C by Sumus+Semper+Una · · Score: 2

      I used to play a ton of Quake and Quake 2. I *think* Cytotoxic is referring to Eraser bot. As noted here, the bot will learn maps it has never seen before. Now, I don't remember ever seeing any documentation about the bots learning your play style or anything, but I do remember most of the rest of what Cytotoxic said.

      A Quake 2 version existed as well, so a friend and I used these bots in Quake 2 to test custom levels. At first, some would run around, not picking up much. Others would just sit still until you killed them and would only fire at you if you fired at them first. After several minutes, the bots would usually quickly start to get a lot better. I do remember some times when the bots continued to be stupid for a lot longer than you would expect. I never found out why, but it had something to do with how you interacted with them and how they interacted with each other in their learning match. I believe at some point you could save what the bots had learned to a file so that they wouldn't have the "stupid" period of startup on the map. Once you had the bots trained, they knew where powerful weapons were and exactly how long the weapons had been absent, so they would often arrive at a weapon location EXACTLY as it respawned so that they could either get the weapon or grab the ammo and deny the weapon to their opponents.

      It was a little buggy, but it did have some real learning algorithms to it and generally worked quite well and could provide either a brutally unforgiving experience (if you allowed them to play at their "hardest" skill level) or an enjoyable testing experience for new maps you had downloaded or created if you set the "skill" to the appropriate setting once they had learned a map. Definitely not as impressive as what's described in this article, but I was certainly impressed with it at the time.

  18. Re: Fucked up by Guspaz · · Score: 1

    In what way? The entire point of this project is that it relies on the screen buffer, hence the name "VizDoom".

  19. Hah, it can't beat humans at quake unless... by GoodNewsJimDotCom · · Score: 1

    Unless it also uses an aim bot, it isn't winning vs rail gun wielding aim bot players.

    --
    God spoke to me
    1. Re:Hah, it can't beat humans at quake unless... by jandrese · · Score: 1

      If it's using a Rail Gun in the original DooM then it already has a big advantage over the other players.

      --

      I read the internet for the articles.
  20. Re:The screen buffer? Really? by narcc · · Score: 2

    It's difficult, and more generally applicable, to make a bot that learns to play by watching only the same info the human players get

    That's not what's happening.

    We've seen similar bullshit headlines like this before that imply that the program learned to play the game just from watching the screen. That couldn't be farther from the truth.

    Prepare yourself for some serious disappointment and read the paper.

    --
    Required reading for internet skeptics
  21. Re: The screen buffer? Really? by maugle · · Score: 2, Insightful

    I don't buy that at all... the computer in this case is still learning through visual feedback only. If you replaced the screen buffer with a camera, the neural net would very quickly learn to ignore the areas outside of the monitor. Similarly, using a mouse and keyboard would be more of an accomplishment for the hardware engineers rigging up the appropriate servos than for the AI learning to use them.

  22. Re: The screen buffer? Really? by the_Bionic_lemming · · Score: 2

    You should buy that.

    Until there's a link between brain and game to manipulate the controls, the Script has an unfair advantage. I have to use a keyboard, the script doesn't.

    --
    _ _ _ Go for the eyes Boo! GO FOR THE EYES!
  23. Re:We already have that! by Bob_Who · · Score: 5, Funny

    Yeah, but does the AI live in its parents basement and pee in empty Mountain Dew bottles ? I think not!

    AI may win at DOOM, but we're superior LOSERS!

    So there...

  24. Re: Fucked up by Anonymous Coward · · Score: 1

    Pointing a camera at the screen is nothing like how human vision works, as human vision is a complicated mess of dynamic gain and feedback, not a simple rectangular array of values. A mechanically controlled mouse and keyboard is nothing like a human, as you can easily move it faster and with more precision than any human ever could. Adding a camera and mechanical controls could easily be simulated by just adding lag to outputs and noise to inputs, but how lag affects a control system and noise affects computer vision is pretty well established. You would learn nothing much, for a lot of effort, while still not really doing anything to approximate a human any better.

    Too many people on Slashdot look at a tree and can't see the forest. The point of most AI research, especially practical parts of it, is not to duplicate humans as best as possible, but to accomplish various tasks that have traditionally been easy for humans and difficult for machines. Exploring an environment from visual information is one such task, and many applications will have other aspects nothing like a human. The fact that this AI doesn't deal with slightly blurrier image is as inconsequential to the point as complaining the AI doesn't deal with drowsiness or aging the same way a human player would.

  25. Re:The screen buffer? Really? Consider... by umghhh · · Score: 1

    It is inevitable.
    Good part is - we short circuit the entertainment industry like this means we win some private time we can do something else....

  26. Re:The screen buffer? Really? by dissy · · Score: 1

    That's not what's happening.

    The summary, article, and paper all say you are wrong.

  27. Re:The screen buffer? Really? by dissy · · Score: 1

    Ahh, my apologies for the previous post.
    You've posted your lies 7 times in this thread so are clearly just purposefully trolling.

    Go ahead and disregard all the facts and don't bother replying to me.

  28. Re:Results As Expected by mugurel · · Score: 1

    They took the same reinforcement learning techniques from the Atria 2600 AI and applied them to Doom. It worked. Next they plan to do it on Quake. Let me give you a hint for the future: It'll work.

    It is still noteworthy, since the Atari visual frames are built up from sprites of limited size that are very convenient for a convolutional model: once it has learned to detect the sprites, it has an accurate representation of the location of the relevant objects in the scene. In doom, since it is a 3D scene, there is no such one-to-one correspondence between object identities and visual appearance. The fact that the method still works is not at all trivial.

  29. Re:everyone STFU! by Anonymous Coward · · Score: 1

    as great of a game that was, wolf3d was household name, not blake stone.

  30. Re:The screen buffer? Really? by narcc · · Score: 1

    Looks like someone didn't read the paper. It's free, so it's not like it'll cost you anything to read it.

    Here's a helpful hint: When you hear than an AI "taught itself" anything, it's guaranteed to be bullshit for the foreseeable future. Simply things like determining success criteria are far-beyond what so-called AI can actually do that it's laughable to say it "taught itself" anything.

    They trained a NN just like we've been training NN's for ages. It's about as interesting as an undergrads NN project.

    --
    Required reading for internet skeptics
  31. Re:The screen buffer? Really? by narcc · · Score: 1

    What about my post was false?

    This toy did not "teach itself" to play the game using only feedback from the screen buffer. That is a very simple and obvious fact. How you can believe otherwise is astonishing. Read the damn paper.

    I know. No one likes to have their silly fantasies shattered by the cold light of reality, but enough is enough. Face facts, read the paper, and get over it. The last thing we need is another technology-based religion like the "less wrong" group or the Kurzweil cultists..

    --
    Required reading for internet skeptics
  32. Re: The screen buffer? Really? by Khyber · · Score: 1

    "I don't buy that at all... the computer in this case is still learning through visual feedback only."

    Instantaneous visual feedback. Humans have serious input and output lag in comparison. Give the bot the exact same latency we've got and see how it does.

    --
    Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
  33. one small step for man... one giant leap for .... by sdinfoserv · · Score: 1

    sounds like the base logic for a terminator. Stick the same code in a robot with a gun and we're all screwed.

  34. Re:The screen buffer? Really? by jandrese · · Score: 2

    b) Game Settings: A state was represented by the most recent frame, which was a 60 × 45 3-channel RGB image. The number of skipped frames is controlled by the skipcount parameter. We experimented with skipcounts of 0-7, 10, 15, 20, 25, 30, 35 and 40. It is important to note that the agent repeats the last decision on the skipped frames.

    How is this not using the screen buffer?

    --

    I read the internet for the articles.
  35. Re:The screen buffer? Really? by narcc · · Score: 2

    Who said it didn't use the screen buffer?

    My point was that it didn't learn to play the game exclusively using feedback from the screen buffer, like the magical thinkers here seem to believe.

    --
    Required reading for internet skeptics
  36. Re:The screen buffer? Really? Consider... by erapert · · Score: 1

    You posted as AC but signed your post anyway? What is wrong with you? Get over yourself.

  37. BRING IT ON by ArylAkamov · · Score: 1

    I PLAY NIGHTMARE IN MY SLEEP

    I STRAIGHT UP GOT A CARBON FIBER HAND DESIGNED BY NASA FOR WASD MOVEMENT

    RIP N' TEAR!

    (Please don't use so many caps, it's like YELLING)
    (Please don't use so many caps, it's like YELLING)
    (Please don't use so many caps, it's like YELLING)
    (Please don't use so many caps, it's like YELLING)

  38. Re:The screen buffer? Really? by Gibgezr · · Score: 1

    From the actual paper: "Game Settings: The game’s state was represented by
    a 120 × 45 3-channel RGB image, health points and the
    current tick number (within the episode). Additionally, a kind
    of memory was implemented by making the agent use 4 last
    states as the neural network’s input. The nonvisual inputs
    (health, ammo) were fed directly to the first fully-connected
    layer."
    So yes, they gave it two integers from the internal workings of the game, solely because the player gets those values from the visuals, but they are using such a lo-res version of the visuals that the program can't get that info from it. Seriously, the AI is getting it's information from the screenbuffer, albeit a very very lo-res screenbuffer (because each pixel is an input to a neural net, they don't want to overload the network with too many inputs).

  39. They claim the AI Doom marine ducked ...WTF? by Ranbot · · Score: 1

    The last second to last sentence of the article says, "...the developers claim that their AI won one of the competition games by learning to duck and therefore making itself much harder to hit." What version of Doom are they playing? As a teenage I played countless hours of Doom 1 & 2 and I don't remember a duck/crouch button.