A Look At Modern Game AI

IEEE Spectrum is running a feature about the progress of game AI, and how it's helping to drive AI development in general. They explore several of the current avenues of research and look at potential solutions to some of the common problems. "The trade-off between blind searching and employing specialized knowledge is a central topic in AI research. In video games, searching can be problematic because there are often vast sets of possible game states to consider and not much time and memory available to make the required calculations. One way to get around these hurdles is to work not on the actual game at hand but on a much-simplified version. Abstractions of this kind often make it practical to search far ahead through the many possible game states while assessing each of them according to some straightforward formula. If that can be done, a computer-operated character will appear as intelligent as a chess-playing program--although the bot's seemingly deft actions will, in fact, be guided by simple brute-force calculations."

Comments

Holy Acronym Overload

[cjfs]

F.E.A.R. , short for First Encounter Assault Recon .... University of Alberta GAMES (Game-playing, Analytical methods, Minimax search and Empirical Studies) .... called STRIPS (for STanford Research Institute Problem Solver)

Combine that with such gems as:

players view the virtual world from the perspective of the characters they manipulate, making Counter-Strike an example of what's known as a first-person-shooter game.

and I'm not sure that belongs here.

Then again, maybe I'm just bitter that I still can't beat GNU chess.

College AI Project

[Dripdry]

Back in college I worked with a guy, Jeff, on an AI project. We were to play the game Freecell through to its finish.
(if you're reading this, jeff, I'm still sorry I didn't do more coding on that and I owe you one)

While I can understand the difficulties in doing a brute force search, and that a simplified "version" of the game could be helpful, OR even that parsed "states" or "instances" of situations in the game could be broken down and analyzed, wouldn't a simpler way be to use a fitness test on various actions? No, no... I lose points for not reading the article, perhaps.

We used a combination of fitness and searching to determine a way to win a Freecell setup. Admittedly this is VERY simplified, and done in a sort of static system as opposed to a (usually) dynamic one in games.

If there is less memory, the obvious answer seems to be to use a system to determine better ways of doing things. Rather than simplifying the game, couldn't the AI have a library of responses designed to fit certain situational profiles, then act in a (perhaps semi-random) manner that fits ? Perhaps the responses could be genetically determined, even.

Also, this use of situations versus individual actions could help lengthen the time the AI has to come up with a response.

Just some thoughts, though I'm sure others more experienced than me have these on the brain. I'm looking forward to the responses on this topic.

Re:College AI Project

[Mad Merlin]

Games of perfect knowledge versus an opponent are pretty simple to solve. You'll find they all basically boil down to minimax applied to game trees plus an evaluation function (which gives you a fitness value). There's also alpha-beta pruning and things like Negascout which are just optimizations for Minimax. The trickiest part of this is writing an effective (and fast) evaluation function.

Freecell is a bit different because it's a single player game, but ultimately you can apply a similar method as above.

Real time decision making in games is often quite different. One problem is that you don't necessarily always want to make the "best" move. In Game! for example, each monster has a regular attack and may have one or more special attacks. Using simple AI to pick one (such as, pick the attack that does the most raw damage) each time wouldn't be as interesting as picking randomly. Say one of the special attacks for the monster is to steal some gold from the player, why would the AI ever pick that? It doesn't benefit the AI at all, but it does make the monster more interesting to fight for the player. Similarly, if one monster has an absolutely devastating attack, a "smart" AI would always use it. But if the AI always uses the devastating attack then either that monster will be impossible to kill, or the regular attacks must be really boring. But, if the monster with the devastating attack only uses it occasionally, it keeps the player on their toes, perhaps they'll heal more often, or use more powerful attacks to try and dispatch the monster faster.

Having said all of that, random picking isn't always the best way to go (although it's quite efficient with CPU time). The main problem with game trees is their branching factor. Chess is a fairly CPU intensive game for AI to play, as it has an average branching factor of ~36. For real time games, it's likely that you can use domain knowledge to substantially prune the branching factor, which makes the problem much simpler. For example instead of considering to, say, turn left 1 degree, or 2 degrees, or 3 degrees or... you could just consider turning left 90 degrees or 180 degrees. If you only end up with a dozen options left to pick from, you can fairly quickly expand several levels of the game tree and then make an informed decision.

However, some games are not games of perfect knowledge (Backgammon, for example), often they rely on chance. In this case, the value of deeper game tree expansion rapidly diminishes, and you simply need to temper your fitness values based on the expected probability of that move being possible. The other problem with games of chance is that the branching factor is usually very high, which typically makes it unfeasible to expand too many levels in the game tree anyways.

Of course you can precook a number of situations, most good Chess AIs have a large collection of book openings that they use. It's really just an application of domain knowledge again, then you can reuse your game tree expansion with evaluation function on each of the book openings to find the most appropriate one instead of doing an exhausive search of all possible moves.

Re:College AI Project

[bishiraver]

Since you used an example from a role playing game , I'll respond with similar. Disclaimer: I'm totally hot for genetic algorithms, ever since I saw this article on pathfinding.

Oftentimes, a player character's "best" moves are limited by other factors rather than just needing to push the button - does he have enough combo points? does he have enough mana? is he in range? and so forth, before it's even a valid choice. Using a more complicated getInformation set than is outlined in the pathfinding program linked above, let's lay out what the mob can find out:

How many hostile enemies are there?
What kind of targets are the hostile enemies (ranged, melee, soft [rogue,mage,etc], hard [warrior,paladin])?
How hurt is each target?
How much damage has each available target done to me?
How much healing has each available target done to other hostiles?
How devastating have the non-damage abilities of available targets been to friendlies?
Which abilities are available to me?
Which status ailments do the available targets have currently?
Add up levels of opponents and levels of allies. Who is larger?
Which ability did I use last? .. And so on - I'm sure there are more checks you could give the AI access to - probably even depending on its intelligence.

Then there are the actions the MOB can take:

Choose target (can target self)
Attack target (melee)
Attack target (ranged)
Use ability on target (repeat for however many abilities are available to the MOB - limited by mana, and so forth)
Run away
Close distance
Run to ranged distance

It would take quite a bit of training (you could probably automatically cull the first several generations, but later on you might actually have to interact with it yourself), but this kind of technique could end up with some very "smart" AIs that are fun and challenging to play against. You don't get God AIs, because they have limited information. You don't get God AIs, because their abilities are limited - and not by simple randomness. You might actually get an AI that stuns you and runs if it realizes it's outmatched, instead of stupidly sitting there and whaling at you with its rusty sword of crumbling.

Granted, genetic algorithms have some HUGE drawbacks:

The decision tree can be quite large, and it can take quite a few cycles to evaluate. Of course, as your fitness check you could check how long it took to execute.

It can take quite a bit of training (hundreds of generations, with thousands of entities each) before you get something that resembles an intelligent algorithm.

Meanwhile, it might generate something that checks for contingencies you never thought to bake into the AI script.

one important point

[SirSlud]

Article is pretty bang on. Adaptive AI is tough to do, as is balancing being a tunable-level of smart and being beatable. One thing I have not seen enough of in games is AI agents communicating with each other about intentions. More often it is simple a matter of saying, "I'm in this area, so don't try and go here." I've yet to really feel in a game that the enemies are working together. I saw a very nice presentation on Halo 3 high level AI at GDC 08 that kind of nailed some of these problems with a pretty simple solution - there should be some top level AI manager that handles requests from AI agents on what to do next when a high level goal becomes useless to attempt to achieve. Left4Dead sort of deals with this, not by talking to agents that are still alive, but by deciding when to introduce new agents, but the Halo 3 approach to me seemed very elegant. It was higher level AI than the article was talking about, but in effect it was a similar setup: AI achieves something, and says, "What's next?" Since the AI manager would know the state of the other enemies in its unit, it could decide that you might as well not start firing at the player since the two others were doing that. Maybe some other game vets could clue me in, but I havn't seen too many games like that where a module is advising the AI based on balancing attack/protect/advance ratios during gameplay. /framework/tools programmer //not AI programmer

Re:one important point

[hax0r_this]

Actually, what the article is talking about is typically applied in systems of multiple levels of AI. Consider an ideal squad based shooter:

1. Command AI issues squads orders (do/accomplish something) based on a very simple model of the battlefield
2. Squad AI issues individual units orders (go somewhere) based on a more detailed model of the immediate area.
3. "Conscious" individual AI computes a good way of following orders from the squad AI based on yet a more detailed model.
4. "Subconscious" individual AI makes moment-to-moment decisions, for example about how to avoid minor obstacles that the "conscious" AI ignored.

Of course that is very idealized.

Game AI For Fun

[Nanidin]

The ACM Chapter that I preside over at Missouri S&T (Formerly the University of Missouri - Rolla) has been writing simple RTS games with AI APIs for the last two semesters. We're currently working on a third game to add to our repertoire. We host a tournament at the end of each semester and invite anyone that will come - the main site is at http://acm.mst.edu/~mstai. The API is easy enough to get a handle on that a C++ novice could pick it up and do something with it within a few hours. Competitors are given 24 hours to write their AI, then we pit them against each other. Generally speaking, for the RTS style games we have written, AIs that act on an individual unit level only perform the best (both in execution time and scoring). This is probably due to the 24 hour time limit imposed, but it does show that even simple/greedy algorithms can perform well in game AI situations. I believe the winning team of our first tournament had an algorithm that went like this: for each unit: doBestActionForUnit(unit)

Re:AI? In video games?

[acidrainx]

I have to say that the AI in Far Cry 2 is definitely one of the worst of current generation video games. I couldn't play that game for more than a couple days before getting utterly bored and frustrated at the idiotic AI.

Enemy Territory: Quake Wars, on the other hand, has some of the best AI I've seen AND its a multiplayer game. The bots' ability to attack and defend objectives while using infantry and vehicle skills against the random actions of human players is incredible.

Re:AI? In video games?

[Eivind]

That is true. The computer is simply very different, so modeling our strengths is just as hard as our weaknesses.

It's trivial for an AI-controlled enemy to get headshots all the time. It's trivial for the AI to have complete knowledge of the battlefield and state of all items and characters on it. Humans can't do that.

It's a lot -less- than trivial for an AI to notice patterns in the enemy and exploit them. Thus the same approach tends to work 100 times against the same AI. It can't learn from its mistakes.

Re:AI? In video games?

[bishiraver]

It could if the AI decision tree were a genetic algorithm.... each entity gets its own decision tree, and the ones that survive mate. :P

Of course, that only really makes sense in an MMO 'verse.

You could do some AI juggling, so that after every map (or every time the AI loses), it runs its algorithms against all previous scenarios until it wins (or at least, gets better at not losing).

But then you end up with an AI that wins all the time, and a huge amount of CPU cycles.

Re:AI? In video games?

[stjobe]

But then you end up with an AI that wins all the time

And we don't want that. We want an AI that wins some of the time, and that is beatable. That is, it should present us with a challenge, but the challenge can't be too great because then the game will be no fun.

So, we only want a smart-enough AI, not a god AI.

The problem

[Anonymous Coward]

The problem with game AI isn't that we can't make better AI, it is that we don't make it a priority. Todays machines are powerful enough to give us good visuals but not powerful or memory spacious enough to really devote resources to too much beyond that. In Mass Effect I want to say we devoted something like 75% of the memory budget to textures, and we still had to downgrade the textures before the final ship. I don't know what the final stats were, but I wouldn't be surprised if about 90% of the budget was allocated to textures and polygons.

That's not to say if you were to quadruple the memory on today's machines that AI would suddenly improve drastically, though. Many teams don't have the resources to devote to programming, so they need to take whatever is in the package. There's room for some entrepenureal spirits to create snap in AI programs, like what Havoc does for physics. Get started now, and you may have a refined product ready for the next generation of consoles.

As an animator, I just want to point out that most anything smart you see in a game is a scripted sequence. An AI marine flipping a table and taking cover is mostly animation work. The only real code there is is a simple set of conditions that determine if the animation should be played and then some state changes to coincide with the animation. The measure of an AI isn't what kind of cool things it can do, because that's animator work, it's how quickly it figures out what it should do, and how well it figures out the quickest way to do it. When you see AI running out in the open, taking the long route to cover, getting hung up on corners or doing circles, that's bad AI.

To give credit (and blame) where credit (and blame) is due, designers choose what kind of behaviors that are possible, so they too are highly responsible for the final appearance of the AI. If a designer neglects a cover system, then it can make even an intelligent AI look stupid by just having enemies stand in harms way. If a designer includes a visceral chainsaw attack, even a poor AI that gets a kill can still seem impressive.