Natural Language CLIs?

snuf23 asks: "Altavista has a report on the future of Windows as presented by Bill Gates at Microsoft Professional Developers Conference. Curiously, one of the touted features is called "type in-line." Essentially, it's a text based interface to the computer which uses a natural language interface. Having worked at a translation software company for three years, I am familiar with the complications of parsing meaning between human languages. It seems that in computer to human you would have somewhat less complexity, at least in terms of general use. Have any natural language interface CLIs been built? Voice recognition software comes to mind ("Open the file, HAL") but what attempts have there been to replace shell interfaces with natural language interpreters?" While I'm all for making computers easier to use, would typing "move all files beginning with the letter a to the directory called 'foo'" be any improvement over "mv a* foo" (or "move a* foo" for that matter)?

Comments

MUD, MOO,and MUSHes Interface?

[echo]

I think a handy interface might be that of the MU* style games of the past...

"look"

You are standing in your office, a filing cabinent is to your right, and your desk is in front of you.

"sit at desk"

You are now sitting at your desk. On your desk is your check book register, a typewriter and a notepad.

"use typewriter"

(at this point, a typewriter program would open (i.e. word processor).

It would also accept text input at the bottom in a sort of "chat window"

"Insert new document"

"Throw document away"

"Get document"

Say you "get document"

You take the document. You should write a name on it.

"name document My Post to Slashdot"

Done.

"stand"

You stand up from the desk.

"look at filecabinet"

It's an ordinary 5 drawer filing cabinet. The drawers are labeled "Bills", "Letters", "Charts" , "Graphics", "MP3s"

open letters

I don't know what you mean.

Open drawer named Letters

You open up the drawer name "Letters"

File "My Document to Slashdot"

You store the file under the folder marked "M"

File "My Document to Slashdot" under "S"

You move the file to the folder named "S"

---------

Anyway, you get the point. Is anyone working on something like this?

Consider the audience.

[generic-man]

In my spare time, I help some people in the area use their computers. One of my clients is an elderly woman with some limitations on the use of her hand. Selecting anything, especially the tiny text links and widgets used in many situations today, can be quite difficult. I didn't even try to do tasks using the command line, since she wouldn't be able to recreate them without calling me up and asking for assistance.

Sure, hardcore *nix hackers will never need a natural language anything -- in fact, one might argue that the standard suite of commands and GNU utilities is their natural language. But for people who don't work with computers all day long, saying something like "Enter a check," "Open a new document," or "Send an e-mail" will do just fine. Each of those documents would require several windows and mouse clicks even with the most intuitive of GUI's.

Re:Consider the audience.

[generic-man]

Although it doesn't necessarily replace a CLI, the nice folks at MIT's Lab for Computer Science have set up Jupiter, a voice interface for weather information. Give it a call at 1-888-573-TALK if you're in the US and have nothing better to do on a Saturday afternoon. You can ask it simple questions like "What is the weather in Seattle today?" and "Will it rain tomorrow in New York?" and it will respond after a couple of seconds with the answer. Usually it's pretty good, if you're used to the back-and-forth interface. You have to know how it works, though -- you can only speak after the tone it gives, you can't cut it off, and its English isn't that good.

Here's what I heard when I handed the phone to a relative of mine who is hardly computer savvy:

"What's the weather in Baltimore today?" (pause, gets answer) "What about in New York?" (short pause) "New York. NO, not Utica, just New York City. Hello? I don't want to hear the weather in Utica, I don't LIVE in Utica! Just give me the weather for New York City! Oh, I give up." (click)

With all due respect, Artificial Intelligence has a long way to go before it catches up with natural stupidity. :)

Re:Consider the audience.

[munch117]

But for people who don't work with computers all day long, saying something like "Enter a check," "Open a new document," or "Send an e-mail" will do just fine.

"Enter a check", huh?

The chess 'bot would wake up and tell you

Sorry Dave, I can't let you do that. Chess rules specifically forbid entering a check.

Seriously, ambiguity is a big problem with natural-language interfaces. How is the computer to know if you wanted to do something involving your checking account or if you want to add an integrity constraint to a computer program you are writing? Or if you are playing a chess game and trying to make an illegal move ...

Natural language comprehension is an AI-complete problem. That doesn't mean useful approximations can't be done. But they only work if you keep commands simple and adhere to a computer-friendly style of expression. With requests of any complexety the risk of a misunderstanding is too great to trust interface to do anything that can't easily be undone. In this sense it is similar to DWIM interfaces (follow the link for a good anecdote).

No doubt natural language interfaces will find its niches, and someday many people may even be using natural language interfaces exclusively. But when these people need to do something more complex or risky, they will need to turn to a hacker who masters some other arcane but concise command language.

/A

this is pointless in the general case

[Aardappel]

I studied "computational linguistics" for 5 years, and if there's one thing that I got out of it is that this whole endeavour (NL parsing + understanding) is hopeless.

Natural language is ambiguous way beyond peoples imagination, and if there's anything we don't need it is ambiguity in giving commands to a computer. NL doesn't _seem_ ambiguous because we are so good at disambiguating it (most of the time, anyway) using our own extensive knowledge base, about what is "reasonable". For a computer to have access to a similar knowledge base (simulating a brain, in short), is a pretty impossible task at this point in time.

Yes, we can get away with simple hacks and partial functionality, but what good is that? it will still be ambiguous. If you want a CLI interface and you want to move away from programming language style stuff, the least you have to do is define a language that can only be parsed and interpreted in one way. This won't be natural language, so user will have to learn its peculiarities. It's a shame, but deal with it.

If I can generalise for a moment, this whole idea that we need an UI that is closer to people to make computers easier to use for computer illiterates is very shaky. The fact that we can talk to it (through the proposed CLI, or speech) isn't making the computer any easier! Do you think that because I can now say "view attachment" instead of clicking on a button, that this will help Joe AverageUser understand any better why part of his HD was wiped, and why all his email contacts got spammed with a virus? Does it help him understand where his file is stored after he uttered "write file to disk"?

There are tons of (relatively easy) things we can do to make a computer easier to use, but this particular one won't bridge the gap once single bit.

This is NOT pointless in the general case

[adubey]

I've been working in "computational linguistics" for the last two years.

I guess there were a few things you didn't study :)

The past 10 years or so a new field - statistical natural language processing (SNLP) has shown a _lot_ of promise.

Right now, if you throw a SNLP system a bunch of parsetrees, it's able to induce a grammar - even in sufficiently complicated languages. (For simple languages, you can even induce a reasonable grammar just by giving syntactically correct string. Impressive!)

The next stage after inducing syntax from training examples with tagged syntax is to induce semantics from training examples with tagges semantics.

Yes, this is still a research topic, but it is by _no_ means pointless. One day computers will be able to do anything humans can, and more.

Natural Language needs a new model

[c_monster]

While I'm all for making computers easier to use, would typing "move all files beginning with the letter a to the directory called 'foo'" be any improvement over "mv a* foo" (or "move a* foo" for that matter)?

Sure, if you stick with the existing model of computer interaction and wedge a natural language interface on it.

However, that would be as useless as having a present-day CLI to low-level system functions only. "read disk sector 1023 from disk 4 via scsi interface 0 number 0 into memory address 1e75FOO" is a waste of even the simple CLIs we have now.

We need to go in the other direction. "Open my wedding photos." "Spell-check the latest draft of my current novel." "When did I receive an e-mail from my publisher with 'foo' in the subject?"

Yes, this means even more work than just parsing a natural language and would require a pretty sophisticated model of interaction, but isn't that the kind of challenge that produces revolutionary advances in computing?

Maybe...

[Tom7]

Slashdot asks:

'' would typing "move all files beginning with the letter a to the directory called 'foo'" be any
improvement over "mv a* foo" (or "move a* foo" for that matter)? ''

Probably not, but how about "Move all files I changed in the last week onto my zip disk."? I know I'd need to check the 'find' man page for that.

Damn, MS beat me to it

[Frymaster]

I've been working off-and-on for the last year on an Eliza-type interface for Xf86config. It still has a few bugs, but the gist is there. It's basic interface is:

XEliza What sort of video card do you have?
user it's a rage 128
XEliza Tell me more about this rage 128
user Well, uh, it's got 8 megs of ram
XEliza How does 8 megs of ram make you feel?
user okay, I guess. I wish I had more of course.
XEliza Does it bother you that you wish you had more?
user Well, I don't really play a lot of games, so maybe I'm just being greedy.
XEliza Do people often say you are greedy.
user Not really, I mean, they say I spend too much time and money on my computer
XEliza Who says you spend too much time and money on your computer?
user Well, my mom does.
XEliza Tell me about your mother
error: cannot parse "mother" in module "freud":
no output written