Slashdot Mirror
AI Researchers Produce New Kind of PC Game
Ken Stanley writes "In an unusual demonstration of video game innovation with limited
funding and resources, a mostly volunteer team of over 30 student
programmers, artists, and researchers at the University of Texas at
Austin has produced a new game genre in which the
player interacively trains robotic soldiers for combat. Unlike most games
today that use scripting for the AI, non-player-characters in NERO learn
new tactics in real-time using advanced machine learning techniques.
Perhaps projects such as this one will encourage the video game
industry to begin to seek alternatives to simple scripted AI."
One of the earliest forms of AI I ever learned about was MENACE. A pre-computer means of training a system to play and win Tic-Tac-Toe. I will confess to loosing more than a little time "training" my system.
What do you know I wrote a novel
Joshua: Greetings, Professor Falken.
Stephen Falken: Hello, Joshua.
Joshua: A strange game. The only winning move is not to play. How about a nice game of chess?
For those of you who actually look on a user's history of posts, yes this is a variant of another post I did, however it's apropos here as well.
Sorry about the writing. Robot fingers, you know? Cliff Steele in DOOM PATROL #23
I implemented learning AI in a couple of popular video games (including at least one multi million unit PC title) more than 5 years ago, and I'm pretty confident I wasn't breaking any new ground.
"Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking
"Galapagos" by Anark had a robot creature with some kind of neural net, and you had to teach him to navigate around by providing him with appropriate stimuli and rewards.
It could get frustrating--sometimes if he hit a particular deadly obstacle too often, he'd become traumatized, and would then refuse to go anywhere near it, which could make the level impossible until you had allowed him to wander around and petted him and calmed him down.
Great game, though. I wish there were more like it.
GCHQ Quantum Insert installed. If only our tongues were made of glass, how much more careful we would be when we speak
and then there's the fun factor. i seem to remember an article about one of the Id games in which they developed all sorts of interesting behaviors for the AIs, played with in for a while, and eventually came to the conclusion that "turn and move toward player" gave much better gameplay.
on a separate note, i remember a game from the late 80's in which you had to program logic circuits to get a robot to perform tasks of increasing difficulty... not a game with a lot of commercial appeal, i'm sure, but i spent many hours trying to solve problems using those little graphical circuit boards...
daniel
"In the far-flung future of the year 2000, functional programming has taken over the world and so humans live in an almost unimaginable luxury. Since it's so easy, humans have used robots to automate everything, even law enforcement and bank robbery -- the only job left to humans is to write their robots' control programs." http://icfpc.plt-scheme.org/
http://www.poconorecord.com/report/iraqwar/000092. htm
___
It's the end of my comment as I know it and I feel fine.
As for what you call "ethical cheats", that is what evolutionary algorithms are really, really good at. Trust me. You have design your fitness function (scoring system) very carefully for this not to happen. It is a major source of frustration, disappointment and thoughts of getting a normal job among neuroevolution researchers. E.g., you want evolution to come up with a nice neural network that drives smoothly around a track, but evolution (that bastard!) finds out that it can actually score higher faster by creating something that drives in circles, bounces between walls etc.
I don't know about the other tactics, but it is certainly not impossible, given that NEAT is more open-ended that most NE systems out there. Let's find out!
What has become of simple HTTP downloads with relative paths? The whole binary could have been picked up by Coral. But nooooo, it has to be a fancy "download.php" with a parameter "go=yes"?! WTF? Is everyone growing retarded these days?
</rant>
Of course it runs NetBSD. BTC: 1NT7QvbetmANwaMzhpVL6
Does anyone remember a research 'game' which was sort of like Pacman but with real motivation. IIRC, the Pacman character was programmed to seek pleasure and avoid pain. Certain pellets were considered positive reinforcements and others were considered negative reinforcements. It ended up having some almost spooky emergent behavior, like hiding in a corner if there were too many negative reinforcement pellets. It seemed to develop responses almost like fear. Stuff like that. I can't recall the details unfortunately. I think it was done as a university project or something, maybe in the late 80s. The idea of generating unpredictable emergent behavior from a relatively simple computer program has stayed with me.
I think that will be the next stage of computer characters: to make them unpredictable even for the programmers. Rule-based learning can get you somewhat complex behavior, but it is all predictable. What we need is genuine example-based learning. So that the resulting behavior would be impossible for anyone to predict and constantly changing and evolving. Of course I am thinking along the lines of various neural network, connectionist architectures. Their unpredictability is generally considered a downside, but for a game the black box aspect seems perfect.
Quite an experience to live in fear, isn't it? That's what it is to be a slave.
Re: Ethical cheating, it will absolutely happen. One of the more famous cases may well be an urban legend but it's exactly the type of thing that happens every day: The army is training image analysis software to locate enemy tanks in the field. They feed it 10,000 images and over time develop the perfect weights so it performs expertly. They move on to test it in the field and find that it's not even close to accurate! What happened? In their test images, the pictures with enemy tanks had all been taken at night, and the others during the day. They had just spent hundreds of thousands of dollars developing a sophisticated way of telling night from day in pictures. Or you could say, the program learned an "ethical cheat."
I remember thinking (not very hard) along these lines some years ago. I was doing a PhD in machine vision and we were using Doom/Quake engines to generate simulated environments for testing robot navigation algorithms.
My thought was that you would train an entity yourself in a series of one-on-one battles or training bouts. These could be staged or otherwise constructed to make mini-games e.g. perhaps testing your entity in predefined scenerios. Once you were happy with its performance you could dump it onto a USB stick and take it around your friends house or upload it to a server for an online game. The main game would put your entity in an arena against a number of other 'gladiators'. They fight it out etc. Online this could allow for 'spectators' who watch the game and potentially even bet on the winner. This might allow for prize money or other revenue stream to be introduced.
-- "Can't sleep, clowns will eat me!"
Which means that you agree with the original poster that people are pretty bad at differentiating friendly from enemy fire.
As the original "grandparent" poster, I have one thing to say to that:
Humans may suck as telling friend from foe in the heat of combat, but right now AI is worse.
In the past, AI has not allowed people to make calmer, more objective decisions. Landmines, to take one example, kill civilians more easily than they kill soldiers, and without the accountability.
How do you mix landmines with AI? "Smart" landmines don't have any AI, they just have a timer or a radio frequency reciever so they can be safety disarmed after the war.
Anyway, the way of the future is going to be soldiers and AI working together, not competing against each other. Kind of like how they do now in the USAF. (The F-22 through the A-10 all have computers to help the pilot; the UAVs have a human to direct the killing.)
Two big ideas of note are the "Future Soldier" program, which is going to introduce a whole host of new tech to the army's riflemen, incudling a live-feed wireless situation transponder. (Kind of like those cameras on Aliens, but not as sucky.)
The second big idea (please pardon my sentence strucutre; it's too early in the am) is a robot-controlled sentry. AI is great for this mindless, repetitive job of looking for movement and firing when given a certain situation. (I.e., "kill anything that crosses that line"). And as with the UAVs, the robot can recieve live guidance from an officer of the military if it is found to have a questionable situation (i.e., "there's a person standing on the edge of the line, not moving forward.")
I remember programming AI for a mech combat game. Me and some friends would spend hours programming bots to hunt each other down and then do battle. I'm pretty sure this was on the PS1, so it's not even vaguely new.
Your sig(k) has been stolen. There is a puff of smoke!
The X-box racing game Forza motorsport already has something like this. You can train a "Drivatar" to race just like you. Once it's properly trained, it will take generally the same line as you, take corners the same way... and it also makes the same errors as you.
More info about it here: http://www.drivatar.com/
In my earlier graduate research, I had several instances where the GA would discover physically unrealistic solutions due to bugs or tuning problems in the model. The problem involved the evolution of a neural network to control a hybrid wheeled/legged robot (the legs were mounted similar to the two rear legs on a cricket). In the robot model, we used a spring/damper model to simulate the ground contact of the feet. However, our integration method was sensitive to high-stiffness equations, and ground contact is about as high-stiffness as low-velocity motion gets, so we had to be careful not to set the ground stiffness too high if we expected the model to conserve energy.
Anyway, the GA discovered with one iteration of the model parameters that it could just peg the feet on the ground with the actuators turned on, and the force from the actuators was sufficient to overcome the springs that prevented the legs from hyperextending. (In the physical robot, there are hard stops that prevent this, but modeling a hard stop would involve using high stiffnesses again.) We didn't model "knee" contact with the ground, since that could never happen with the physical robot. So, there it was, a simulated robot with its knees hyperextended and protruding beneath the ground. Its feet were still at ground level, and the whole setup was very high stiffness. So, all it had to do was hold that position, and the jitter from the numerical inaccuracies caused it to accelerate itself forward without actually having to do any walking.
There are other examples of cheating GAs in the literature. One example was (I believe) from Karl Sims's work, where he evolved virtual creatures both in structure and control. The task was to get the virtual creature to achieve as much altitude as possible, given physically realistic physics. However, there was a problem in the physics model, and the GA discovered a cheat. The result was creatures that would beat themselves over the "head" repeatedly, with each smack causing the creature to rise up into the air more and more.