Slashdot Mirror
BotPrize — A Turing Test For Bots
Philip Hingston writes "Computers can't play like people — yet. An unusual kind of computer game bot-programming contest has just been held in Perth, Australia, as part of the IEEE Symposium on Computational Intelligence and Games. The contest was not about programming the bot that plays the best. The aim was to see if a bot could convince another player that it was actually a human player. Game Development Studio 2K Australia (creator of BioShock) provided $7,000 cash plus a trip to their studio in Canberra for anyone who could create a bot to pass this 'Turing Test for Bots.' People like to play against opponents who are like themselves — opponents with personality, who can surprise, who sometimes make mistakes, yet don't robotically make the same mistakes over and over. Computers are superbly fast and accurate at playing games, but can they be programmed to be more fun to play — to play like you and me?"
Read on for the rest of Philip's thoughts.
Philip continues, "Teams from Australia, the Czech Republic, the United States, Japan and Singapore competed in the final. Competitors created bots to play a specially modified Unreal Tournament 2004 Death Match. Expert judges then tried to tell whether they were playing a bot or a human, just from their observation of the way they played the game. Judges included AI experts, a game development executive, game developers, as well as an expert human player. The result? The winning team AMIS, from Charles University in Prague, managed to fool 2 out of the 5 expert judges, and achieved an average 'human-ness rating' of 2.4 out of 4. All the human players were judged more human than the bots overall, but the judges were fooled often enough to suggest that in next year's contest, some bots may be able to pass the test by fooling 4 out of 5 judges. AMIS won $2,000 cash plus an all expenses paid trip to 2K's Canberra studio. You can check out the full results and competition videos, and try an online video quiz that lets you judge for yourself."
The fact that you're actually playing a human is a big factor too. Fast connections and low ping times aren't the only reason LAN parties were successful -- sometimes you just want to rub it in.
they want a bot that moves and fires randomly and then types "fuk u faggit" into chat every time you kill it.
lysergically yours
Just have the bot randomly jump around, and then stand over their kills and repeatedly crouch.
I strongly suspect that making a game bot truly act like a human calls for heuristics that approach those in real humans, meaning something like "true" artificial intelligence. Those heuristics would be be worth way, way, way more than a measly $7000 or $2000, and a trip. Billions, in fact.
Still, it'll be interesting over time to see if someone can, in fact, make a highly "human-like" set of heuristics without actually achieving this "true" artificial intelligence, or if someone does invent heuristics for "true" artificial intelligence then is naive enough to give it away for not peanuts, but a half a single peanut. Either way would say something important about so-called "human" intelligence.
A truly excellent pizza parlor is a delight unto the heavens. Treasure the sauce and the toppings!
As a casual gamer + AI observer, in my opinion the biggest / most obvious difference are human traits.
While this may sound obvious, let me elaborate:
- Traits are different to mistakes or intelligence. Mistakes are missing, shooting into walls, walking over edges, etc.
- Traits are: becoming too involved in a firefight, that you *know* you're going to lose, being so wound up on one enemy that you miss seeing others, hiding behind corners to wait for others to become injured, etc
Playing against humans has much more appeal than bots, because people are 'fun'. No bot is ever going to run at you with an axe ( or other lowest equivalent weapon ) when you've got the BFG - but humans will - and will often win with this tactic through sheer stupidity or blind chance.
I can only imagine programming human traits is a lot more difficult than 'standard' AI.
In the videos, I got most of the choices right by applying the question: Who is applying human behavioral patterns?
The only exception to this would be a difference matte of the player.
I would 'render' out a patch around any visible opponent to see *how* visible they are.
If they're standing still in front of a tree trunk the average luminance at the border of the player and tree need to measured. If the player is instead a black SWAT player on a white snow field then his visibility would be increased.
Motion should also be multiplied.
Contrast * %of Player Visible * percieved velocity (If they're crouching and creeping at '100 yards' they'll be moving slower in the bot's view than if they were 2" in front of them.
You want to make sure that just because the player model is visible it doesn't mean the player would actually be visible to an opponent. I can't count how many times a black hooded enemy in a darkened window has sniped me.
Also a bot should have its sound perception nerfed. Just because it hears a set of footsteps doesn't mean it shouldn't be biased to stereo or at least 5 channel restrictions.
Chatting is disabled so that the challenge isn't to write a chatbot.