Slashdot Mirror
Computer Learns Language By Playing Games
Frans Faase writes "By basing its strategies on the text of a manual, a computer infers the meanings of words without human supervision. The paper Learning to Win by Reading Manuals in a Monte-Carlo Framework (PDF) explains how a computer program succeeds in playing Civilization II using the official game manual as a strategy guide. This manual uses a large vocabulary of 3638 words, and is composed of 2083 sentences, each on average 16.9 words long. By this the program improves it success rate from 45% to 78% in playing the game. No prior knowledge of the language is used."
All Civilization-franchise manuals soon to be confiscated and destroyed in the name of national security.
It's always confirmation bias!
in schools? Get kids reading decent manuals (text-books) and perhaps they may actually learn something and find they can do decent things with the new-found knowledge.
Todd: I hope it proves as delicious as the farmers that grew them
And yet you were dumb enough to marry her?
I wonder how good that algorithm is at identifying people using their style, grammar and errorrs. Think Facebook and Google+.
If no prior knowledge of the language is used, how is the program able to determine word boundaries? Perhaps they meant the domain-specific language (meaning vocabulary), for example "unit" and "cell"? Otherwise, there's a plethora of bits of knowledge about English grammar and structure that they probably coded into the AI...
I also find it junk that the web.mit.edu link posts a screenshot of Civ 5, when the AI they are discussing runs against Civ 2... My bullshit meter is starting to tickle.
Every other word out of it's "mouth" would be 'anal'
Prof. Farnsworth - "Oh a lesson in not changing history from Mr I'm-My-Own-Grandpa!"
You don't have to be dumb to marry a realdoll.
The only winning move is not to play.
How about a nice game of chess?
Computers have always been good for doing tedious jobs that people don't want to do.
Like reading manuals.
Before "gaming" became synonymous with exclusively FPS "if you can see it, shoot it", there used to be all kinds of games available, often with interesting manuals.
Needless to say, the downloaded copies were better than store bought, because they didn't have copy protection / DRM, but obviously they didn't have the manual that came in the box from the store.
Section 6 of the paper seems to imply that even the most illiterate fool would still win about 30% more games by having a copy of the manual, no matter how illiterate they are. Even if the manual was written by the programmers in Hindi, even non-Hindi readers, at least as smart as a computer, would win 30% more games...
If only the games industry made non-FPS games, then they could use this to motivate people to buy the game with the manual, instead of just downloading... You'd still have to download the game anyway to avoid the DRM, but at least you'd win a minimum of 30% more games by having the manual.
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
Mostly the kind of language you don't use among polite company.
Call me when computers learn to swear idiomatically and emotionally appropriately.
Welcome to the Panopticon. Used to be a prison, now it's your home.
The baseline rate of winning was 46%. What level of strategy was being implemented at this baseline? It seems like there's a lot of room for interpretation. If I wanted exciting results (and had no actual interest in doing science), I'd probably do a combinatorial removing and adding of various algorithms from my baseline until "reading" the manual gave the highest jump in the win rate.
Nice, now I have an AI to play against that isn't completely retarded.
The next step is to get this sort of thing onto Battle.net.
If you read the paper, you see that they are using FreeCiv, and not Civilization II.
Also from TFA: initially, its behavior is almost totally random.
I have no idea what constitutes a "win" in this game, but if a "totally random" strategy can win 46% of the time it sounds a little cheesy. Sorta like life, I guess.
XML is like violence. If it doesn't solve your problem, you're not using enough of it. --AC
You've obviously never experienced life with a female.
I played Civilization 2 for years, and I still don't understand the rules.
How does it know it wins??
I think emotion would yield better results because bad emotions, such as losing tend to make people try harder and not lose rather than try sequentially try different strategies.
I would make them think further than just choosing different pathways in the game, as well as learn from their mistakes.
Previewing comments are for sissies!
If this thing gets a copy of the Bible, we are boned.
If Slashdot were chemistry it would look like this:Cadaverine
It could learn that "bad" is a noun and "fail" is an adjective.
-fb Everything not expressly forbidden is now mandatory.
I'd be curious to see how it fares in a roguelike game. That would be for me the ultimate test in learning to play by merely reading the manual.
Heh, it got crushed by the FREECIV AI, which is worse than the Civ 2 AI. Makes me feel better about the superiority of man.
Which makes it even better. Having the computer "read" the manual of one game makes it better at playing a different game.
I learned a whole new language playing Halo 2 on Xbox live.
And on page 1 the paper speaks about Civilization II.
Can't blame the editors this time, at least not much.
Hey don't blame me, IANAB
Shouldn't this be "computer learns game by reading manual"? The computer didn't learn to speak, it just learned to play the game...
I guess they picked a Civ manual for a reason. I don't remember the Civ II manual, but I remember the original Civ manual - that thing was a brick, a few hundred pages, with an appendix which had most of the algorithms used in the game documented. Not surprised a bot could get better at playing the game with that kind of reference material!
I wish someone was still publishing manuals like that.
Tuning pages with your right hand does not count as understanding them, any more than your hand counts as a spouse.
Just another ignorant American.
It would be interesting to feed this thing a "Java for Dummies" or "Learn C# in 21 Days" book and see if it can start writing it's own software. Maybe even throw in some books on AI and see if it can generate it's own AI software and become self aware.
Sometimes the light at the end of the tunnel is the headlight of an oncoming train.
How about a nice game of chess?
Weaseling out of things is important to learn. It's what separates us from the animals... except the weasel."
And you are too dumb to realize tits don't have an IQ. Nor do they need one.
The types of games they played is very constrained. It's only two civs on a very small map, and the only way the algorithm learns to win is a settler rush. It's not deep strategy.
----------
I am an expert in electricity. My father held the chair of applied electricity at the state prision.
For the purposes of this project, it appears that a "win" is annihilation of the opponent's civilization.
The totally random strategy never won a game. The trial-and-error based AI that started out untrained and making random decisions was almost able to match the built-in AI, after learning all it could. Upon being fed the manual, it improved significantly.
Before the AI takes over the world.
Shall we play a game?
How about Global Thermonuclear War?
insensitive clod overlords obligatory xkcd car analogy russian reversals whoosh pedant fanbois ftfy in 3...2...1..PROFIT
what side do you want
1. USA
2. EU
3. UK
4. Russia
5. Chain
6. north korea
7. iran
8. middle east
I can't believe that it had a manual that had enough information to actually boost your score. Documentation nowadays is usually pretty lame, and doesn't actually provide anything instructive.
Kudos to the writers of the Civilization II documentation ... I bet if you tried it with a modern game manual, the computer's score would go down. ;-)
This tells me they actually wrote a comprehensive guide, which was well written.
Lost at C:>. Found at C.
Have it play against me on the XBox. So it can learn profanity.
"I guess the moral of the story is, don't paint your airship with rocket fuel." -- Addison Bain
I was once slicing potatoes. I always done this by hand, but I had one of those new plastic cutting boards with the blade built in that I wanted to try. I loved the speed, but I didn't know just how fast it cut. I would do half a potato, then switch to holding down the potato with a hand held plunger thingy. The plunger was really hard to hold, so I'd go longer and longer without it. By the 5th potato (the last), I went so fast and so far that the pain of slicing a millimeter off the tip of my thumb was an instant and panic filled moment.
Immediately my mind raced...
Undo! Undo! Undo!
The doctor that used something like a needle tipped hot gun to cauterize my thumb got a kick out of the story. I've never used one of those demon machines since.
I8-D
The 46% win is after the AI implementation learns (somewhat) how to play the game. It isn't winning 46% of the time by always doing random actions.
The "totally random" is in reference to the fact that it starts random, and then learns to do the "right" (//less wrong) action based on the in-game feedback from its random action.
This is in contrast to the method the researchers were testing, which uses both in-game feedback, plus the manual's text, to refine its actions.
From the summary: "No prior knowledge of the language is used."
This may be nitpicking, but this doesn't seem to be strictly true, assuming that by "language", we mean English, since the authors used the Stanford parser to do some pre-processing of the language in the manual.
Perhaps more correct would be to say that no prior domain knowledge is used, or no prior knowledge of how the language in the manual maps to the game world is used. (Unless you want to argue that the Stanford parser is language independent, which you could, but I don't think this is fair here.)
Separately--would be interesting to run these tests with actual meaningful strategy guides (vice just the manuals) as the input, although blowing up the corpus might (speculation) require modifying their techniques, since they (appear to?) only use a singular sentence at any given point in time as input to the output action.
any more than your hand counts as a spouse.
Ohhhh...burrrrrn!!!! In his face!
the amount of NLP required for this task is almost nil. since the backend is doing massive combinatorial search anyway, all that the "reading" does is bias that search to look deeper at combinations of keywords which occur in the manual. it just so happens that game manuals are very simply written (since their point is not to be stand-alone literature).
for example, if i wrote a manual full of weird phrases like "it's pointless to consider strategies combining $foo and $bar", it would probably trip up this algo (until they patched it to look for negations) while a 5-year-old would not have a problem.
i dunno, it's cute i guess but i'd rather see, for example, an agent that learns from a blank slate to ascend in nethack. that would probably take linguistic AI deeply integrated with the game logic.
"They were pure niggers." – Noam Chomsky
A bit off topic, but I was always amused by the fact that when you play versus the computer 1v1 in Starcraft 2, the computer says "gg" when it realises that they can't possibly win.
And then they surrender.
I'm just waiting for the days when they start swearing at you and you can't tell the difference between AI and a person.
So that's how I can get better at video games! By actually reading the manual! Thankyou, science!
The obvious next step would be to feed it a fan created strategy guide and see if it gets even better.
My first impression of the linked article is one of skepticism that they are really getting out of it what they think they are... While a computer program could certainly apply word relationships from an instruction manual to its interactions with a game program, presumably it has some method of characterizing and tracking word relevance as it "learns."
That very characterization process may actually contain all the necessary "learning," and the actual text be irrelevant. The real test they need to do, is not to compare it with a program that isn't using a text as a guide, but one that is using a completely irrelevant text as a guide. I think the learning may be happening entirely in their learning mechanism such that any text would work as well-- "wrong" advice would be characterized as such by trial and error, so even bad information is useful in a system like that.
I recall back in the 1960s or early 1970s, reading a children's craft project book of some kind that had a simple AI project where you could manually train a system of matchboxes containing colored candies, what the right answer for a given input pattern was (it was either a simple letter pattern recognizer, or would learn tic-tac-toe board patterns, I forget). But what I do remember, is even the "voters" in the population who consistently voted wrong provided useful information, if you can characterize them as always doing that. I wish I could remember what book it was in, but it did make some light bulbs turn on at the time. I suspect that the word-characterization process here might work just as well given ANY piece of text fed into it, it's just a source of data to characterize, and once the data is characterized, pretty much any data would produce the proper trained result. The manual text is then just a substrate on which to hang the learning information on, such that pretty much any substrate will do.