Slashdot Mirror
StarCraft AI Competition Results
bgweber writes "The StarCraft AI Competition announced last year has come to a conclusion. The competition received 28 bot submissions from universities and teams all over the world. The winner of the competition was UC Berkeley's submission, which executed a novel mutalisk micromanagement strategy. During the conference, a man versus machine exhibition match was held between the top ranking bot and a former World Cyber Games competitor. While the expert player was capable of defeating the best bot, less experienced players were not as successful. Complete results, bot releases, and replays are available at the competition website."
am terrified we're actively pursuing the ability for our robot overlords to have the perfect micromanagement strategies.
By default, I'd say that a well-designed bot in an RTS would have an advantage over all but the best players. Since there is so much to keep track of, software would win out over wetware.
Bots in an FPS are one thing, but when you have dozens of units, a build order, multiple fronts, resources, and more to track all at the same time, the infinitely scalable multitasking of a bot would certainly come in handy.
Or maybe I'm off base?
Living With a Nerd
Cool.
Now someone write me a bot which helps me track down those bastard P'toss players who keep tower rushing the outside of my base, so I can teleport my fist in their respective faces.
lol, captcha: crystal.
Uses Broodwar, not Starcraft 2, and not just the original.
Just saying. (Cause when they mentioned Ultralisk Microing, I thought about SC2 and how Ultralisks are terrible units there simply because they block your units making Micro a huge pain, and it wasn't so bad in BW when your units could take a bit more of a beating).
As a casual player of SC & SC2, I've always found the bots to be pretty competent and provide a decent challenge. There are some tricks to halt the AI early game but ignoring them and playing normally they provide a decent challenge. There's only been a few times it flakes out early game and doesn't climb the tech tree, usually happened with Zerg.
When I was a new player with SC I actually thought the bots were utterly brutal and was a little annoyed with no difficulty setting. After awhile I got used to how the game essentially threw you in the deep end and was able to fend for myself.
machine is capable of reading and executing way more instructions per sec than a human can ever do, however the difficulty is deciding which instructions it should execute, thats always the hard part with writing an AI. Writing a very good reactive bot is not an easy thing to do, because people can be unpredictable. my 1st ever /. post!
So the winner is just a Muta harassment bot? I have to wonder if the top level human player just spammed some Corsairs or Valks? It didn't seem like the AI was particularly good at changing strategy if the opponent countered. Spreading out the Mutas would help against a Corsair or Valk counter though, since both of those units rely on the Mutas natural tendency to clump to get maximum effectiveness.
I read the internet for the articles.
Anyone know if anyone has ever created a bot that has the ability to learn from losses and wins, to figure out what works and what doesn't (both what it is doing and what the enemy is doing, then use that in the future to predict what the other player is *trying* to do and come up with a counter)?
I've not played a lot of RTS's, but I've played a few, and the thing I've noticed is, if a strategy works against a bot, even if there's a fairly obvious counter, it will always work against that bot.
The RTS I've played the most is Supreme Commander: Forged Alliance, so I'll use an example from that: There are a collection of different bots available which will use different strategies. Now, the player can build a strategic missile launcher, which builds nukes. You can also build a strategic missile defense which will build 'seeker' missiles which will shoot down incoming nukes.
Some of the bots will build SMD's, some won't, but in no case does it build an SMD based upon the player/opponent's actions. That is, it doesn't check to see if you have started to build any Strategic Missile Launchers before it begins to build the defenses. The bots that don't build defenses won't build them even if you are building one. This means on the one hand that the bot wastes resources which it could have used otherwise, to build defenses, while on the other hand, if it doesn't build them, you can pretty easily and quickly defeat the bot with a nuke or two. Alternatively, instead of building a strategic missile defense, the bot *could* try to use someattack method (for example, if you don't have good air defenses, it could hypothetically try a targeted attack with a bunch of bombers or gunships to either destroy the SML, or the engineer units which are constructing the SML).
However, the bots never seem to be smart enough to attack the obvious threat of a strategic missile launcher. It seems like the only way the developers found to make the bots harder is to make them much more efficient at building up their economy and spamming out lots of land-units to try to attack the player.
When I get a chance, I want to try SC2, but right now, I'm in a period where I'm not playing games as much as I used to, and trying to reduce my gaming down to almost none while I get some more important things done in RL. It'll be interesting to see how the AI differs in that game.
While a bot will certainly be able to macro and micro better than a human, human players are closing the gap and I don't think that the current gap is all that much between humans and machines.
On the other hand, the creativity gap between humans and machines is always in the favor of humans. At best (as of now), machines can only be designed to react, and their "creativity" is limited to a book of plays. Once you know the machine's playbook, you can easily defeat them. So they are only as good as how well their playbooks can be kept secret. Their ability to make a gamble during the heat of the battle (be it in attacks, build switching, etc) is still limited compared to humans (in my opinion). And this is often where tides turn in such games.
I've seen a lot of AIs that have a general strategy that they stick to like glue. I'd love to see an AI that continually evaluates its current position, and if it sees a loss trend, switches tactics.
When you're doing the right thing, but slowly bleeding out to inevitable death, you might want to lower your risk aversion and try something else.
Beyond that, some way of evaluating a player and keeping track over multiple games - and comparing known players and their styles to new players to make predictions, would be nifty.
There are software competitions in chess, and they're getting pretty exciting now that the software can play at a grandmaster level. I think it would be pretty cool if this sort of thing made it into games like Starcraft. In any case, I'm happy that bots and their authors are getting some prizes and recognition, because I think their work is incredibly important and fertile. Bots are out future.
Competitions like these are great, but they (and games like SCII) really make me pine for an RTS with a robust in-game scripting language. I would like to write complex auto-executing instructions for my units like "Pursue but never enter the firing zone of a known enemy turret or siege tank" or "If your energy > X and enemy of type Y in range R, cast spell S" or "If you are unit type X, always position yourself between friendly unit type Y and the enemy". You could confine the script to some reasonable specifications (say, no more than 1000 queries and 100 orders given per second) if you want to deter brute-force approaches.
This is a totally different problem than writing a good AI, you would be focused on writing powerful tools that aid, not replace, a human player by letting him specify his intent on a higher level than "go here" or "attack that". Better visualization of what's happening would be an integral part of this too. I would love to have SCII give me an overview of what units/building I have, what they are doing/queued to do -- even better if they are grouped into functional 'squadrons'. Being able to have multiple panes/monitors looking at different things would help to.
I guess what I'm saying is that I'm old and slow and want to leverage my experience writing rule-based logic to beat the whippersnappers that can click faster than I can and keep more things going in their heads at once :-P.
There seems to be a very wide range in the abilities of the winner and runner up bots that might not be noticed by someone unfamiliar with Starcraft. In order of appearance:
1) In the flying units versus flying units match, mutalisks (guys with wings) should have focused the scourges (little 'c' shaped guys) because scourges have about 1/5th the health of a mutalisk, but can suicide to take out 5/6ths of a mutalisk's health. Red ai focused scourges, the other ai didn't, with disastrous results for the other ai.
2) In the match with infantry, the players had medics, which heal other units but can't attack, and marines/ghosts, which can do damage but can't heal each other. One ai moved medics with ghosts such that the medics could actually heal. The other ai just left the medics a mile away from the combat, and got slaughtered. Furthermore, the AIs didn't bother with formations, which meant that half their units spent the entire fight trying to get into range. A precombat formation lets almost all the combat units start firing as soon as the fight begins.
3) The red zealots retreated in the face of numerically superior opponents, while the teal zealots just attack moved no matter how many they had. Teal zealots didn't focus fire, which meant that they lose units sooner, and thus had less damage output compared to red. In addition, red failed to kill the pylons (600hp) powering the buildings (a lot more than 600hp). Neither player built their bases to maximize the number of pylons powering their vital gateways; each pylon usually powered only one building.
4) In the fight with dragoons (orange spider things) versus tanks, the protoss (orange) could have frozen half the enemy tanks with a single stasis spell by sneaking the arbiter (flying spider thing) to the back of the tank formation. Furthermore, the protoss could have focused the science vessel (floating teal circle) that was preventing them from being invisible. It would have been a slaughter if the vessel had been focused, as teal would have had no real way to hit the orange units.
5) In the match between Overmind and Krasi0, the article talks about mutalisk clumping preventing some of the mutalisks from attacking. However, the point of the stack is that when one guy is in range, everyone else is. Also, the attack animation is so fast that for all practical purposes, the flyers can shoot while moving. They use their mobility to get out of range of infantry units, then zoom in to pick off outliers when their attack cooldown is finisihed. In actual competition, the terran player would usually rely on a strategic placement of static defenses and a highly mobile cluster. However, the terran (defending force) built tanks, vultures, and goliaths (mech guys that shot missiles) with the flaw that tanks and vultures can't shoot air, and are thus almost useless versus the mutalisks. The mutalisks spent the second half of the clip shooting tanks rather than focusing down the goliaths volleying missiles into them. The terran AI prioritized repairing tanks as much as goliaths, and didn't place tanks next to goliaths to soak up bounce damage from a mutalisk.
In general, the AI had problem with understanding the priority in a fight. That is to say, they often had no sense of what units are critical to a position or what units pose the most threat. They didn't arrange their units to maximize their effectiveness, and often failed to alter their behavior based on the other party. It's a fun contest, but I'm not sure the AIs could beat a moderately skilled player who understands tactics AND strategy.
Signatures are the new names.
Taken to the logical extreme, it sounds like you just want a game where you don't have to micro your units.
(Assume you write the script, publish it, and the community picks from a set of micro scripts)
SC2 would then devolve into "I think he's going strategy Y, computer deploy counter strategy Y-c".
Which then devolves into "the AI deploys strategy Y-c because it is the most effective strategy against the observed strategy Y"
Which then devolves into Progress Quest.
YES.
YES.
YES.
YES.
I remember thinking something like this when I played Mechwarriors; HOW COOL would it be if your Mech had an API and you could program or customize behavior?
In fact, now I want a Mech simulator that gives you generic programmable parts, and you can build your mech from scratch programming things like the gyro, actuators and so on.
Or maybe I should just get into electrical engineering and/or robotics and/or lay off coffee forever.
You should look at the Spring engine. I'm pretty sure unit behaviors can be scripted using things like Lua.
That was me BTW.
For instance, imagine you're given a set of parts with simulated physics and physical+logical interconnects. .... we could have competitions to see who can program the fastest running bot!
I figure with real robots there would be a lot of welding and metal-piece-shaping involved, but imagine if somehow all that was virtual/programmable, and the whole thing moved according to your programming (lower leg goes up to xx degrees, has such and such torque/force/movement curve).
BOT DANCE COMPETITIONS!
AMERICA'S NEXT TOP ROBOT!
A group of my friends have been working (slowly) on such a project for a while now. Though it's pretty much ended up being a client-server RTS where everyone writes their own client.
I am trolling
Have a look at the Spring Engine if you haven't already. There are a variety of RTS games, including some high-quality variants of Total Annihilation, which use the Spring Engine and allow for all sorts of client-side scripting through Lua. There are a variety of client-side lua "gadgets" that players have written already. You can move your units into custom formations by drawing lines or squiggles with the mouse; there are widgets to automate using air transports to ferry units between factories and rally points; there are even widgets to automatically alert the player when certain dangerous units are spotted. IIRC, someone was even working on a script for kiting with long-range units.
I kinda liked the "less control" approch of Majesty.
Your units actually think for themselves, and respond differently to your "reward" incentives. To me this makes for a more "realistic" experince than the "hive mind" approach of the standard RTS (Where the player controls exactly what every unit does.)
monkey spunk
I think quite a bit of this can be attributed on generic versus specific behavior. I am not an AI programmer, and I just glanced at the page and videos, but from what I gathered most of the AIs had hardcoded strategies that they would follow blindly. Then, the strength of the AI was directly proportional to the quality of the strategy that was implemented. They would use very specific patterns and follow them systematically.
What I would be curious about would be a more generic AI. Instead of using known strategies, the AI would analyze the battlefield and would, for instance, focus fire on support units (not on "medics" or "science vessels", but using the stats of the unit to determine its usage and purpose) or the most dangerous units (air-to-ground would prioritize fire on ground-to-air units) or the most important buildings (using ratios and comparisons to determine this, instead of hardcoding that building A is critical). Technically, it could then be able to react to a lot more situations and would also be adaptable if the fight changes or if the enemy bot has a counter for whatever strategy had been planned. Obviously, a generic bot is harder to build and it might also not be as efficient when compared to singular strategies, but it would be an interesting experiment to make.
whilst the game is a simplification of what you're after, gratuitous space battles is a game entirely on this concept, you set up your units beforehand and then leave them to do their battle. http://www.positech.co.uk/gratuitousspacebattles/
Taken to the logical extreme, it sounds like you just want a game where you don't have to micro your units.
Actually, I think the extreme is a game where you write the script then you don't manage your units at all -- and it turns out that games like that are pretty awesome even when you only have one unit to manage, so I can only imagine managing a whole army would be much more fun (presuming the complexity wasn't overwhelming)
I mod down anyone who says "I will be modded down for this", regardless of the rest of their comment
jizzed in my pants when I read this finally I get to test the best of the best! and I met a AI sometime last month that *was* more flexible in that it could be any race. It was a little underdeveloped, because it was mostly just a structure for others to add their ideas/AI on. and writing mutating code is VERY hard, especially for SC. give them a break.... please.
Smoking cures cancer. Smoking also cures stupidity. check darwinawards . com for some stupid stuff
Did you ever play Carnage Heart? It sounds a lot like what you're thinking of, though this was from the mid- to late-90s. My friends and I played this all through college. We eventually came up with some pretty ridiculous strategies.
I'm glad to see a potential player base for this type of RTS. I would also love the ability to select from scripted build orders. Just let me fine tune bits of my second-nature strategies in game and let me concentrate on scouting and such.
I think this is an awesome idea. I would love to play a game like supreme commander but options like that. It would turn every game into a real battle of strategies, instead of who can best micromanage their resources.
hmm what if you had to code during the game (no way to save/restore programs), and you're also limited there by a resource? So you can't implement that 5000 lines of code AI, because you only have 10 seconds for it, and you only have collected enough energy for 5 lines of code.
I'm with the winning UC Berkeley Overmind team. ( http://overmind.cs.berkeley.edu/ ) We're very excited to have won the competition, and we're hopeful that soon we'll be able to create an agent that can beat the very best humans.
Here's an overview of our strategy. Broadly, our agent always plays Zerg, and its primary objective is to apply constant pressure to our opponents, which will let the agent continually expand to improve its economy. To that end, our agent had three primary tools: a scouting worker that harassed early, zerglings that provided early defense, and mutalisks which basically force the opponent to stay in their base. Our agent would make decisions, based on what it observed, to trade off between these different forces. It might build more zerglings if it sensed more early pressure, or it might skip them altogether if our opponent wasn't going to attack early.
Our choice of mutalisks was deliberate. They are a highly mobile all-purpose air unit that can mass up and cause significant damage in a short period of time. Also, they don't have bounding boxes, meaning that they're more amenable to computer control. That choice proved to be really valuable, because a lot of the opponents seemed to have preferred ground armies. However, there are many other strong units. For example, the other agent in the finals (Krasi0) was truly impressive with its ability to repair units.
Anyway, on our page we have a couple of videos, and we'll have several more by tomorrow (Saturday).
hmm what if you had to code during the game (no way to save/restore programs), and you're also limited there by a resource? So you can't implement that 5000 lines of code AI, because you only have 10 seconds for it, and you only have collected enough energy for 5 lines of code.
Yeah that sounds sweet, you could also just make it so the more complex the units code is the longer it would take to train. You could also unlock new functions as you progress, and have a spy unit to steal enemies various code.
That was me BTW.
For instance, imagine you're given a set of parts with simulated physics and physical+logical interconnects. .... we could have competitions to see who can program the fastest running bot!
I figure with real robots there would be a lot of welding and metal-piece-shaping involved, but imagine if somehow all that was virtual/programmable, and the whole thing moved according to your programming (lower leg goes up to xx degrees, has such and such torque/force/movement curve).
BOT DANCE COMPETITIONS!
AMERICA'S NEXT TOP ROBOT!
Hmmm... I may sound like I own a Wii :), but IMHO you don't want to make it too complex, just a simple macro based system, or perhaps an event based flow chart. There also needs to be some sort of penalty for overly complex orders.