'Tower of Babel' Translator Under Development

monopole writes "The BBC is reporting on a bilingual translator under development by Carnegie Mellon University which senses sub-vocalized speech, recognizes it, translates it and then synthesizes the translation. The overall effect would be to dub the speech of the speaker."

Comments

subvocalization

[TubeSteak]

http://en.wikipedia.org/wiki/Subvocalization

Subvocalization is basically micro-movements of the muscles associated with speech. The Wikipedia article mostly focuses on reading & subvocalization, so I wonder, do you have to be trained to do it consciously?

http://en.wikipedia.org/wiki/Subvocal_speech_recog nition

This wikipedia article says that recognition is hard.

Re:I don't get this

[thewils]

Well, I was trying to give a simple example. It can get quite convoluted. Check out Mark Twain's essay on the Awful German Language.

"The Germans have another kind of parenthesis, which they make by splitting a verb in two and putting half of it at the beginning of an exciting chapter and the OTHER HALF at the end of it."

Here's what you'll look like

[rhysjj]

Some photos of the electrode arrangement needed on the face:

http://www.flickr.com/photos/rhys/260069248/
http://www.flickr.com/photos/stasarama/245979951/
It's still a lab prototype of course, but a massively impressive one. I'm very pleased to see articulatory speech recognition (that's the main research area in this particular project, rather than the translation itself) get written up by the BBC.

Re:Here's what you'll look like

[rhysjj]

It's true that articulatory speech recognition should be easier than automatic speech recognition (ASR) based on waveform analysis alone. It's massively unfortunate that ASR research has, at least for the past 20 years, concentrated mostly on the latter and not the former. Janet Baker, whose MIT PhD introduced Hidden Markov Model (HMM)-based ASR, and opened the door to companies such as Dragon (which she and her husband founded), is herself now saying that HMMs are rubbish for speech recognition. I desperately hope that through this CMU project, and others, that people will start to take note of this.

I think you're entirely correct that the machine translation (MT) stage is a bolt-on in this particular project. This project is I think a vehicle for articulatory ASR rather than MT. But I wouldn't be so keen to dismiss MT efforts altogether. It's true that in some ways the current deployable systems make gross assumptions about language, which may be even worse than the assumptions ASR systems make about speech (that's particularly true of purely statistical MT systems). But Google and others have apparently shown that with a large enough corpus, you can get results that extend beyond simple phrase-book look-up quality.

There's one main question facing the researchers at CMU, I think. That's whether people will be happy to stick a dozen electrodes on their face in order to achieve speech-to-speech translation, or whether they'll prefer to speak into a microphone and have a speech synthesiser (e.g. the open-source Festival, partly developed at CMU) speak the result. I'm not entirely convinced they will, but I'd be absolutely delighted to be proved wrong.

Re:The science behind it is fascinating

[WoLpH]

We already have something like that, right?
It's called a universal translator

Re:Question of the Millenium

[AhtirTano]

Sorry to say, English is not as unusual as you would like to believe. (I am a linguist.)

In many ways, English is quite simple. For example, our word order is very straightforward. I work with a language were the following is a normal sentence: "This is city New called York here." (This city here is called New York.) In fact, almost every permutation of those words would be valid without a change in the basic meaning (as long as "is" is the second word). This is a so-called non-configurational language. Parsing English is easy by comparison.

I work with another language were there is a slight stress difference between the sentences "That might be true" and "He's honestly picking his butt." The words "soup" and "shit" are differentiated by a 40-50% increase in the length of the last vowel. There is one word for both "blue" and "green", and another word for "yellow", "orange", and "brown".

As to the likelihood of this project succeeding anytime soon: Languages are often not directly translatable into each other. One language I work with has an entire part of speech I cannot adequately translate into English. I have to wave my hands and point to convey the same information in English.

Killer App

[jshazen]

I can't believe nobody's posted this yet. This would be *really* useful as a *mono*-lingual translator! Build one of these into every cell phone, and suddenly I don't have to hear your inane conversation just because you happen to be sitting next to me in the plane.

This should be *much* easier to do that the version that actually translates, and it would add nearly as much to quality of life of the user and everyone else in his environs.

Re:The science behind it is fascinating

[ch0knuti]

The point is not in the height of the tower. It's the symbolism that man was defying God and trying to get "there" by his own means.

Re:Other Languages

[Shaper_pmp]

You can't fully understand what someone's saying unless you understand their cultural context, and how it differs from your own.

For example, what's the difference in (UK) English between: "I couldn't care less" and "I could care less"? In US English they're used interchangably, but in UK English they're opposites. There are many such words or phrases in the English language alone where the precise word chosen (or connotations of a word) totally changes the meaning of the entire phrase, even reversing it's meaning.

Another example would be a simple phrase in US English like "he was pissed"? US meaning is "he was angry". In UK English it means "he was drunk", and a word-for-word translation into greek it would be meaningless (the equivalent idiom in Greek would be something like "he took it on the skull").

Seriously - if you ever want to understand the drawback to automatic translation, try getting two Greek friends to talk colloquially to you, but translating each individual word into English - it's completely unintelligible.

Re:The science behind it is fascinating

[Morphine007]

I'ma go out on a limb here, but do ya think that perhaps the bible is mostly allegorical? I mean, I'm no theologist, but it would seem to me that the tower of babel story is more of a warning to the masons of the time that if they try to build too high, they'll be fucked.

I would think that the "message" from "god" is closer to "you don't understand my creation well enough to build this yet... in time you will..."

Then again, maybe I'm just a heathen...