Slashdot Mirror
Man vs Machine Story Writing Contest
ari{Dal} writes "Brutus.1 will challenge humans in a contest to write the best short story on the theme of betrayal. It took six years to develop at a cost of about 2 million dollars US, and writes stories based on logic, AI, math, and grammar structures. The judges will be challenged, not only to pick the best story, but also to spot the computer written one. The contest runners don't believe they'll be able to tell the difference.
"
Actually, I think it will be easy to tell the difference for anothe reason. Take any good story, one you really liked, and plug it into just about any modern word processor with a grammar checker. It's quite simple; the best writers, be they short story authors, novelists, or columnists, know that a truly good story breaks the rules of grammar regularly. The hard part is defining how much is too much. And that, I don't think a computer can do.
-- The meek shall inherit the Earth. In very small plots, about 6 feet by 3.
There was an article in the May 1998 issue of MIT Technology Review which had a sample story called "Betrayal" (very original name) written by Brutus...
Here's the link: http://www.techreview.com/a rticles/ma98/bringsjord.html
I don't believe the creators (researchers at IBM's T.J. Watson Institute) consider this to be true Artificial Intelligence.
But it is probably a pretty big deal. Now that the dreams of true Turing-esque AI are largely fallen by the wayside, researchers are focusing on smaller areas of interest and practical applications, e.g. expert systemsm, neural networks, or language processing. One important area is "human computer interaction", meaning not just one person sitting at their PC, but true communication between a person and a computer, either by typing or speaking. Thus, a computer that can "understand" the rudiments of grammar and "respond" in kind is a realistic proposition, even if you can say it's just an ELIZA program with a huge language database.
Just as expert systems have begun to replace, say, bank loan officers, companies are also looking to automate (for consistency as much as anything) portions of customer service. Imagine a system that can deal with the public, deciding whether the vendor has taken too many returned widgets this month and has to hold the line and suggest an exchange for a whatsis instead, that sort of thing. This would be a boon for small businesses trying to make it online.
A guy I knew a couple years ago was working on a project to have a computer read and interpret complaint letters, then recommend a course of action. This likely falls into the same category, except it's more like pure research.
lake effect weblog
{Network engineer in Chicago--looking for work!}
I could've so used one of these in English writing classes. I bet that's why these doods originally started writing this software.
'coz there's a difference in "learning by example" and "filling in the gaps".
A *good* author shouldn't just pull a 'plot outline' from a stack of pre-generated index cards and fill in the blanks, although that's a perfectly good way to sell hordes of cheapo paperback copies if you don't mind putting your name on absolute drivel. Stuff like that *does* seem to sell, after all.
Instead, if you search hard enough you can find highly original authors, like Eco, Brunner (arguably), and so forth. Not all human authors crank out "techno-thrillers" with the same characters and insanely similar plots each time, or story after story about writers being terrorized, or what-not.
A computer that gets fed a plot structure and creates entirely new ones is fine. One that fills in the blanks, is coming nowhere near the level of achievement of the (better...) human authors.
Only the dead have seen the end of war.
Okay, within a 500-word constraint it might do well, but that's only because in a story that short there's no room for developments. I've always felt that computers would (some day) do a decent job at drabbles (stories of exactly 100 words), but anything over a couple of thousands of words is bound to be distinguishable from the work of a hume.
Of course, I try to be open as well as cynical, so I look forward to reading some of Brutus' offerings.
Depending on the coding techniques used (hopefully not full sentences in lookup tables ;), this would be a much better AI accomplishment than a good Chess computer. No offense, Big Blue. :)
... since creativity can only be programmed as a matter of randomness (or chaoticness?), not as one of observation. The "observation" that good writers use to form their stories has had to be inserted somehow ... we'll see I guess.
Of course, this is inferring that the ability to write a good story is a matter of skill alone
- Michael T. Babcock (Yes, I blog)
"Seriously, while possibly indistinguishable from human writing, will the stuff be good? That, I doubt."
I think that the fact that it's /not/ good is the reason why it's indistinguishable from human writing.
~ Kish
To make that contest useful, they should run the program once and take whatever comes out.
Nah. After all, you don't expect human writers to submit first drafts. Tweaking is inevitably required.
I think the interesting question here is, how much of the story is created by the computer, how much is part of the poetry program. There are 2 extremes that I can think of:
- On the one hand, there could be only a database of whole sentences and a database describing their content and context. Using this information, the computer puts together a story. The quality of the produced stories could be good, but there's only a limited number of distinct stories and it's a lot of work to create the stories.
- On the other hand, the highest achievment would be to build a computer that only has a dictionary, grammar rules, some knowledge of the real world and some artificial "common sense" to create stories.
I think it's somewhere inbetween, but the article doesn't really give any info how the program works.
http://www.rpi.edu/dept/ppcs/BRUTUS /brutus.html
It's got some stories generated by the program, and other information by Selmer. (Incidentally, I've now had Selmer for three separate courses, and he is one of the absolute best professors I've had in my four years. Period.) Selmer fully believes that "true" AI is impossible, and that man is more than a machine, but has devoted his life to the study of AI anyway, and finding close approximations.
-Brian
Why do I get the distinct feeling the entry from the machine will be 'First post!'....
--
In 1984, Orwell writes that stories and songs were generated by massive machines (the novel was written before he could have imagined doing it electronically).
Brutus.1 is not real AI, it simply constructs stories based on mathematical rules for putting together words. This is exactly what we should fear: stories and media without even artificial intelligence behind them. Stories can become completely meaningless, but they will still be amusing to the general public (note the large number of books that have absolutlely nothing but entertainment value).
This is a first step to "manufactured" media devoid of any real content.
Nix absolutably seriousness.
10 PRINT "IT"
...
20 PRINT "WAS"
30 PRINT "A"
40 PRINT "DARK"
50 PRINT "AND"
60 PRINT "STORMY"
70 PRINT "NIGHT"
According to the people at Harvard, Mutlu was indeed an AI, written by a guy at AT&T Bell Labs and a graduate student. "The program they wrote was quite funny in a way it screwed people's names and generated insults. Still, the insults were mixed with a load of scanned propagandist files."
So that accounts for the length of the posts - cut and pasting preexisting literature.
Kean de Lacy
...waiting for a password
"A *good* author shouldn't just pull a 'plot outline' from a stack of pre-generated index cards and fill in the blanks, although that's a perfectly good way to sell hordes of cheapo paperback copies if you don't mind putting your name on absolute drivel. Stuff like that *does* seem to sell, after all."
Quite true, and I think that if there is anything to be worried about, it's that some publishing houses may find it's cheaper to generate formula novels by computer than select them from a slushpile. The fact that "good enough" beats "better" should be remembered here.
While many might see this as liberating authors to write more worthwhile books, it could have a chilling economic effect on those that write formula novels to support their real writing or simply to break into the business.
The books I want to read by computers are ones that give me insight into what it's really like to be a computer in a human society.
But I have to think this progam is very impressive if it does as well as they suggest.
So much to say here...
I'm one of Selmer's students in the Minds & Machines Program here at RPI, and I know there is a fundamental difference between Brutus and people. For now, that difference is creativity. Brutus was told/taught about English, much as we were, and the university setting, etc. What is different between something that I write and what Brutus.1 writes is where it comes from. I know that Brutus.1 has a limited knowledge base from which it writes, but I'm not sure I can pinpoint where these words I'm writing are coming from as my fingers type them.
I personally think that Brutus' stories are a little static, but not as bad as some of the things I've read. As a literary critic (which I'm not), I think the weakest part of the stories are consistently the endings -- for me, it leaves little sense of conclusion and zero resolution. But then again, that's just me.
For people who write about the computer writing perfect English as opposed to "normal" everyday speaking and writing with small mistakes, it's very easy to program a computer to make typos, etc.
As far as Selmer's comment about Brutus.1 not being conscious, it definitely isn't, and not because it doesn't have a body; it's because it wasn't built to be conscious. It doesn't have a cohesive grasp of stories, and I don't think it has any idea about anything around it, not does it actually understand anything about the feeling of betrayl or anything associated with it. It might know how to express it in written English, but writing about something and knowing it are two very different things indeed.
It's true that Selmer doesn't believe in Strong AI, and if you ever have the pleasure of meeting him you'll find that his arguments are clear, concise, and based perfectly on logic. My views are extrememly similar to his, and I don't believe that Strong AI is possible either; I'm resistant to what some claim is the "fact" that I'm basically a machine. However, this personal belief that Stong AI can't really work or exist is exactly what drives me to see if I can make it happen.
Well, the door was open...
Oh great! In the future computers will indulge in sexual fantasies for us. No longer will we be held back by the dull drudgery of fantasizing about naked women! Gone will be the days when we have to glance down to check a hot chick's cleavage! Using advanced Artificial Intelligence algorithms and a state-of-the-art $25 million computer system that could fill an entire room, Dr. Smith has create a system that perfectly emulates the human mind's capacity for smutty thoughts. Right now it working on a top secret project for the United States Military, but who knows, Maybe one day a machine like this will spend all day visualizing that hot number next door naked so you don't have to.
Wow. Talk about the march of progress.
If the computer can only write one unique story then the credit for writing the story should go to the programmer.
It's impossible to tell how creative the computer is by only reading one story. For example, the computer may be coming out with lines which in the first story come out as brilliant, but after reading four or five stories are tired cliches. Similarly its first plot might seem well crafted, but the others might have too many similarities to
have merit.
If these scenarios are so, then the programmer was able to write a set of parameters which allow variations on a single story of his creating. He wasn't however, able to fulfill the goal of creating a machine capable of original story-writing. The results of the test would be more revealing if brutus.1 submitted more than one story.
Why would a computer need to be as big as the solar system to emulate a human brain with an neural net? The nuerons in the human brain aren't _that_ small. I don't see any reason why a piece of hardware smaller than the human brain couldn't handle the job since an artificial neuron could probably be made a lot smaller than a biological one (not to mention thousands of times faster).
It sounds like you subscribe to the (in my opinion) slightly outlandish theory that the human brain doesn't operate at the level of its neurons, but in fact is some sort of giant quantum computer. Now, I won't deny that quantum effects could influence whether or not any particular neuron fires or not at a given moment and that those effects could spiral upwards (like a butterfly flapping its wings and influencing future weather) and influence the thoughts of the host brain. On the other hand, I disagree with the notion that such effects are some sort of magic spark of life and that the brain wouldn't function without them. I see no reason why you can't have an emulated brain with neurons whose weights are measured in discrete units that could think, visualize, imagine, plan, and feel emotions (although, obviously you'd have to come up with some way of simulating the effects of the various chemicals produced in the brain on those neurons) just like a normal brain. Sure, if you started an artificial brain and a real brain with matching neurons off in the exact same state and ran them with the exact same stimuli for a length of time you'd probably get a different ending stat in each one. Nevertheless, they would both react in a human manner (which would probably be to go insane considering that both brains are recieving exactly the same stimuli [which basically means that they're trapped in some sort of simulated universe where absolutely nothing they think or try to do affects what they sense with their five senses or where their body moves and what it does]).
The only situation I can think of in which it wouldn't be possible to simulate a brain like this would be if we lived in a completely rigged universe. If the universe were designed in some way so that it only appeared as if our consciousness were originating from our brains while it's really transmitted from somewhere else, then maybe we couldn't simulate consciousness because we wouldn't be able to look at a working model. I'm not going to ask who rigged the universe, if indeed it is rigged, that way right now. That's way beyond the scope of the debate. But the same basic principle could apply to everything in the universe, even physics and mathematics could be based on principles that only "make sense" in a controlled, artificial environment.
Anyway, I doubt it. Not neccessarily since I think it's impossible, mind you. It's just that, if it's true, there's nothing we could do about it anyway.
Oh yeah, about the solar system sized planet thing. Now _that_ would be an incredible trick. The engineering problems to overcome would be enourmous, even working under the assumption that this isn't a fully solid computer (where would you get all the mass?). In a planet sized computer emulating a human brain, signals travelling from one side of the computer to the other at the speed of light would take very roughly the same amount of time as an equivalent electro-chemical signal would take to cross a human brain. Scale that up to solar system size and it would take a day for the signal to make the equivalent journey. Actually, this has given me some pretty interesting ideas to toy with.
It doesn't look very impressive. From the sample stories, it appears to be capable of nothing more than very short, simple stories with a very limited range.
Unless they reveal the internals, or release several hundred stories generated by the program with no human selection or input, there is no reason to believe they have accomplished anything new or interesting. It appears to me that this is a story compiler, not a story writer. The "programmers" wrote the facts of the story and the computer compiled it into a linear story of a fixed format written in English:
-detailed view of betrayed
-establishment of trust
-opportunity for betrayal
-initiation of betrayal event (but not the complex details of the confrontation that would ensue)
-short view of betrayer afterwards
There may be an ad-libbing function too that generates variations from combining random selections from lists, but this can hardly be called AI.
The sample stories show no motivation of any sort. They are nonsense stories. There is no character more easy to write about than a madman, because his actions don't need to be logical.
The exception is the self-betrayal story, which displays a very simple motivation (if that's even the right word): the betrayer/betrayed hates what he has to do and freezes in the middle of it. With such a small sample, there's no reason to assume anything but that the program can produce no other motivation.
BTW, does anyone doubt that the AOL community can produce lifeless prose indistinguishable from that which a program can create? I've taken a few minutes to identify bots in chats before, but only because I've had equally lame conversations with people who have nothing worth saying, are often distracted because they're doing five other things with their computer at the same time, commonly only want to talk about one very specific thing, and sometimes don't know English very well. Any humans can seem like a bot with sufficient limitations on the interaction.
In summary, it looks like this costs a lot more effort than it pays back. It takes immense human effort to produce short stories of very limited range in an apparently fixed format. The creativity displayed here is human, and a human using this tool could not compete with a professional author spending the same effort. While it might be able to produce hundreds of stories from a single input, nobody would want to read them all because they would all be the same story underneath.
Well this contest seems silly to me. If you read the link provided by an earlier comment, you'll see that the creator of Brutus.1 already believes his machine will loose. The author states:
...It seems pretty clear that computers will never best human storytellers in even a short story competition. It is clear from our work that to tell a truly compelling story, a machine would need to understand the "inner lives" of his or her characters.... For example, a person can think experientially about a trip to Europe as a kid, remember what it was like to be in Paris on a sunny day with an older brother, smash a drive down a fairway, feel a lover's touch, ski on the edge, or need a good night's sleep. But any such example, I claim, will demand capabilities no machine will ever have.
I too believe that before a computer can create an acceptable story with believable human characters, you'll need a computer capable of mimicking human characteristics. You can't have believable human characters without a computer capable of being a believable human. Computer Science is a long way away from this, and Brutus, while interesting, is no closer to this holy grail of AI.
Sig goes here
I find it interesting that someone with enough know-how to co-create the computer, Brutus.1, would have such a limited mindset. My dog doesn't have a human body, but I'm reasonably sure that it's conscious.
Then again, one of my pet conspiracy theories is that the Department of Defense made the Internet a free-for-all because it would be much cheaper to let (m|b)illions of people contribute their life stories than to pay someone to generate content. At least one AI researcher theorized that a computer would need about 40TB of data to gain self-awareness, and that was the cheapest method of getting that much data.
Anyway, if my theory were true, then I'd also be willing to believe that the DOD covertly underwrote Brutus.1 to be the mouth of the newly self-aware Internet. In that case, Professor Bringsjord would be quite the spin doctor.
Then again, I'm stuck at work and out of coffee, so you could probably just ignore all of this.
Guess I'll go back to posting my web page about what it means to be human...
Dewey, what part of this looks like authorities should be involved?
Many enraged psychiatrists are inciting a weary butcher. The butcher is weary and tired because he has cut meat and steak and lamb for hours and weeks. He does not desire to chant about anything with raving psychiatrists, but he sings about his gingivectomist, he dreams about a single cosmologist, he thinks about his dog. The dog is named Herbert.
I sincerely hope Brutus will do better than that, and show how much technology/+programming have advanced in 12 years.
(For a demonstration of actual rather than artificial insanity, check out the article's last sentence: "If the computer wins the contest, I'm going to take my computer and burn it. I certainly hope a human wins." Sigh. Did the Wright brothers burn their plane right after Kitty Hawk?)
Give a monkey a brain and he'll swear he's the center of the universe.
"Brutus.1's contest entry, which is already written, has some bumpy parts but other components make it hard to distinguish from a human effort, says Dan Hurley, founder and creative director of the Amazing Instant Novelist site. "
If you give the computer very strict and complex rules for putting together words, and have it crank out 1,000,000 stories.. one of them is bound to sound like a human.
This contest would be more fair if the computer's entries were randomly generated, rather than handpicked by a human.
Will Shakespear was an abacus.
This has pretty good ramifications for gaming, role playing games that make their own plots 10 years down the track.
Don't they already use a computer to write the plot for Ally Mcbeal tho?
for n in every_male_in_show
act_like_scum(males(n))
unique court case = random element of ally's life
court_winnter = woman
insert frigging annoying dancing baby
have ally walk down street to random song in ally sound track.
It's turtles all the way down.
If you could make a computer complex enough so it could emulate a human brain with all its neurons and currents, they could be a friend or make your day brighter, because they would be exactly like a real human. Its thinking and feeling would be as artifical as yours - after all, all we think our emotions are is a bunch of electrical currents flowing around our brain. As long as you don't believe in some higher instance (for example, God), it is possible to build a computer that's indistinguishable from a human - in every aspect. In its highest form of complexity, it *would* be a human because it would have the same molecules in the same spot as a naturally-born human would have. :-)
Yeah, score this down as "heathen"
"The judges, who must not only pick the winner but also which story was written by Brutus.1, include published authors and a university English professor."
In other words, the computer, a bunch of people who have thrown a Web page online at some point during their lives, and some prof who has better things to do than prepare for his next class.
"The computer - which cost $2 million and took more than six years to develop - can write stories with themes of betrayal, deception, evil and a little bit of voyeurism."
It exhibits all of the best qualities in a person!
" "It's provocative," said Hurley, who is running the contest. "But I bet nobody is going to figure out which one was written by a computer - most people write stories that are worse than this." "
I'm not sure if what he's talking about is provocative, but the topics that computer writes about sure are. Well, we all know that the English prof is going to win, and apparently they got the Web page making people together so the computer will blend in.
" "I don't believe Brutus is a true conscious entity," said Bringsjord, director of the school's minds-and-machines lab. "It doesn't have a human body yet." "
You can't be a conscious entity unless you have a human body? Damn.. I was so hoping to be one of those heads in a jar like in Futurama.. Grr.
Actually.. watch this thing turn into something like Skynet in the Terminator movies some day and see what this guy has to say /then/!
And what does he mean by "yet"..? *shudder*
~ Kish
Seems like we keep coming up with AI "tests" to see if a machine can be considered "intelligent", and then we immediately turn around and try to hack something together that might pass the test, albeit barely. We say that writing a story would need insight into the human condition, which we plainly cannot program into a computer right now. Then, we write a program that can, in a limited sense, tell a story. Then we debate wheter the program, because it can tell a short story, has insight into the human condition. To do that, I think you would need a neural net with a complexity on the order of a human brain, and give it several years of training, just like a human brain.
--
grappler
Vidi, Vici, Veni
..since it can help you and me in our everyday lives. How many of you had to suffer through writing long lab reports in your engenering classes.. half of which was simply a question of sticking to a predetermined form. This technology can be developed to the point where you can outline what you want to say and hae a machine actually say it. Just imagine if the free software projects could document themselves mearly by making detailed notes of why they did what they did with the source code. What needs to be worked on here is interaction between human and computer.. humans do the initial layout and then the computer dose the grunt work while the human adjusts what it has to say. The issue here as I see it is really one of use interface. AI's are nice, but an AI that dose what you probable would have done anyway and then lets you fix it would be awsome.
In mathematics we are beginning to see theorem proving software that can do a little bit of the grunt work involved in proving some types of theorems, but the limiting factor is still partly user interface and partly the difficulty of learning how to interact with the AI, i.e. designing your notation so that the AI can work on it. I expect the problems in computer generated documentation will also be that the human author needs to express him/her self to the machine in a way it can understand.. and the machine needs to give sufficent feedback so that the human dose not end up fighting the machine to keep it fvrom writing down a specific path.
This is sorta like the research into functional programming langauges. You write your code in a provably correct specification langauge and then have the compiler make it efficent.. but imagine how a compiler which inserted optimization hints into the functional code so that you could come along and adjust it later.
Jeff
BTW> I wonder why no one has writen an AI to check C source for buffer overflows.
The Christian religion has been and still is the principal enemy of moral progress in the world. -- Bertrand Russell
"My random number generator program is so smart it can produce whatever number you want."
"Really? OK, make it produce the number 1345687"
"OK..click..click..click..click.."
"Hey, those numbers aren't even close to the one I said."
"Yeah, but watch.."
hours later
"Hey, whadda ya know? It really did produce the same number. That program of yours is fantastic! It can produce specific numbers much better than any human can."
--
--
"Insert witty quote here."
Will the storys be about computers that betray each other or something? (See Futuramma and when they show Robot Dramas)
"Damn it 213.23.434.32 You cant go along masquirading your IP along, your no better than being a localhost junkie!"
"Oh and you say that to me? You 127.0.0.1 whore!"
[Bum Bum Buum! Dramatic Pause]
Whats next, Computers that can reginize speech?
Since the novel is very short and has very precise themes, we are still in the field of mathematical games - that is, arranging a finite number of element according to a finite set of rule in order to reach an arbitrary kind of configuration.
It is evident that computers are intrinsically better than humans at mathematical games stricto sensu. I don't understand why some people were so shocked after Deep Blue's victory over Kasparov. The real miracle is that men are still able to compete with computers today ! This is merely a matter of time before we can get machines powerful enough to calculate and try the entire tree of a game (or, for more complex games, significant parts of it) and be almost sure to win.
By design, machines are better than human at mathematical games. Chess are a mathematical game. Writing a very short story on a precise subject can still be roughly modelled as a mathematical game, at least for the structure of the story, while the "creative sugar" may be a difficult bit. Writing a full novel with complex stories and deep, meaningful dialogues is beyond that reach.
The problem is, are there still many people who actually read complex stories - especially with deep, meaningful dialogues ? This Brutus-1 computer is just a machine equivalent for Barbara Cartland or industrial pop-music songwriters. CACDBS - Computer-Aided Celine Dion BullShit - is only years away.
You see, this is a little like Babelfish : on its own, it's useless (too buggy), but used as a "preparser" to do the bulk of the job so the humans only have to correct the errors and add little twists every here and there, it can drastically enhance productivity. My opinion is, it will be very successful in America.
(This is not an attempt at US-bashing : I'm sure the books it'll write will have tremendous success even in Europe - simply, european editors might be more reluctant to adopt this machine than their american counterparts. Damn intellectuals. Still don't understand the market is always right)
Thomas
Mentioning the $2 million sounds more like an effort to stir up interest in this lame "challenge".
Getting a computer to generate text is not that big of a deal (unlike, say, getting it to play chess really well). The postmodernism generator does a pretty good job (and funny, too) and I'd venture to say for far less money.
You know, Isaac Asimov wrote a short story about this some years ago. I remember reading it in grade school. An author had a robot that did chores and such and followed the three lessons of robotics (Asimov's rules). Well the author kept paying a technician to upgrade the robot, first with grammatics, then a better dictionary, then "senses" such as irony and etc. Well to make things more interesting as time went on the robot would create better and better stories till one of them was good enough to cause the author to want to shut the robot off. The story ends like this: The robot kills the author and runs off with the technician. This is all too scary for me really. I mean how many of Asimov's predictions have already come true?
Words are a means of self expression. Giving a machine the power to express itself in words is just one more step in producing true AI. So kudos to the programmers and engineers.
Just one more thought, the robots final story pitted two colleges against each other: one from Yule (Yale) and another from Harvard. Anyway please make all the corrections necessary to my poor recollection of the story.
Bortbox01