'Tower of Babel' Translator Under Development

monopole writes "The BBC is reporting on a bilingual translator under development by Carnegie Mellon University which senses sub-vocalized speech, recognizes it, translates it and then synthesizes the translation. The overall effect would be to dub the speech of the speaker."

Comments

The science behind it is fascinating

[chowdy]

The Tower of Babel Translator is small, yellow and leechlike, and probably the oddest thing in the Universe. It feeds on brainwave energy received not from its own carrier but from those around it. It absorbs all unconscious mental frequencies from this brainwave energy to nourish itself with. It then excretes into the mind of its carrier a telepathic matrix formed by combining the conscious thought frequencies with nerve signals picked up from the speech centres of the brain which has supplied them. The practical upshot of all this is that if you stick a Tower of Babel Translator in your ear you can instantly understand anything said to you in any form of language. The speech patterns you actually hear decode the brainwave matrix which has been fed into your mind by your Tower of Babel Translator.

Re:The science behind it is fascinating

[WoLpH]

We already have something like that, right?
It's called a universal translator

Re:The science behind it is fascinating

[ch0knuti]

The point is not in the height of the tower. It's the symbolism that man was defying God and trying to get "there" by his own means.

Re:The science behind it is fascinating

[blippy]

The practical upshot of all this is that if you stick a Tower of Babel Translator in your ear

No, no, you're thinking of a babelfish. The Tower has to be inserted up a completely different orifice.

Re:The science behind it is fascinating

[mikael]

so God hasn't in fact stopped anyone trying to do anything, he has just stopped them from succeeding which would suggest that it was the success of such an enterprise he was worried about rather than the idea behind its inception.

He should have filed patents and brought patent infringement lawsuit like everyone else.

Planet Earth 1
God, et al. Year 0

Abstract

A tower being is provided for the means of reaching heaven constituted by a plurality of stone columns and support beams positions on top of each other. The tower comprises of an architectural assembly omposed of a large number of levels constructed from stone columns reinforced by stone beams. Each floor may be augmented by mud bricks to provide additional reinforcement against strong winds. Access to each level is provided through
either a wooden ladder or a mud bricks staircase.

Re:The science behind it is fascinating

[Morphine007]

I'ma go out on a limb here, but do ya think that perhaps the bible is mostly allegorical? I mean, I'm no theologist, but it would seem to me that the tower of babel story is more of a warning to the masons of the time that if they try to build too high, they'll be fucked.

I would think that the "message" from "god" is closer to "you don't understand my creation well enough to build this yet... in time you will..."

Then again, maybe I'm just a heathen...

Re:The science behind it is fascinating

[COMON$]

How you got modded 5 informative is beyond me. Insightful maybe, but informative no.

While an intelligent assumption for a person who has not delved into the history of the bible, it is however mostly wrong.

The allegory belief usually comes when one distances himself from the Bible. The less you read it (be you agnostic or Christian) the more likely you are to believe that it is allegorical and disjointed.

The bible is at best a historical account of a group of people through their eyes, thus needs to be viewed as such, and the writing is protected by God. At worst it is an accurate mythological history book.

I am not a theologian or historian, just a geek but I am fascinated with the bible's accuracy Historically, they may embellish stories to include God, but the time line is accurate and the Historical places and people are real. Which is fantastic when you consider the age of the book.

In conclusion I would say that the tower of babel story is not allegorical, but rather God chose that junction to disrupt the human race long enough for them to define themselves rather than playing silly games that he told them not to. Whether we like Mosaic law or not, even as secularists we have to admit, there is a bit of unnatural wisdom in the protection offered by Mosaic law.

Other Languages

[Longfinger]

If this technology gets good enough, none of us would ever need to learn a second language. That would be a bad thing, right?

Re:Other Languages

[Anonymous Coward]

If this technology gets good enough, none of us would ever need to learn a second language. That would be a bad thing, right?

I think that's kind of like saying that if calculators get good enough, no one needs to know math anymore.

In fact, this will probably be used in many of the same places - anywhere you'd find a cash register, you'll probably find automated translaters. You won't see them used in academia or in diplomacy, though.

Re:Other Languages

[Anonymous Coward]

This could be a terrible thing. Learning another language teaches you a lot about a way a culture thinks.

Re:Other Languages

[kfg]

Language is not composed of words. It is composed of idiomatic phrases (idiomatic phrases do not mean what the words mean) only understandable in context. True automatic translation is not possible.

As an example, I was once called upon to translate the simple advertising slogan "Si Misura" from Italian to English. This had already been translated as "Made to Measure."

Quick, without thinking, tell me what the product was?

If you're a native English speaker you probably think of a suit or dress. Maybe a kitchen cabinet. Some tool with human ergonomic requirements.

The product was a liquid chemical compound, so I translated it into the correct English idiom for such; "Custom Blended."

And with that simple example we haven't even touched on issues of syntax yet; or more complicated issues of social usage (say formal vs. informal forms).

KFG

Re:Other Languages

[antifoidulus]

But doesn't the language itself play a part in the culture? Almost any language you look at there are bound to be words that don't translate well because the object or action or emotion in question is so innately bound to the culture that they made a word for it, but to other cultures the concept isn't all that common so they never made a word for it.

Also, even translation by the best humans still destroys a lot of the subtlety and beauty in a language. It's a best a piecemeal game. Hell, most novels/tv shows are not even translated literally, some artistic liberty is usually taken to make the work "flow" in the language it is being translated into. Translation is great for contracts or technical documents, but if you really want to understand a culture then you need to learn its language.

Re:Other Languages

[JuzzFunky]

Second Language? You might not even have to larn a first language! Just grunt. Mm.

Re:Other Languages

[brian.glanz]

There are practical advantages in problem solving which have been tied to the language used in mental formulation, for example the development of what is metaphorically called "logical circuitry" has been shown to diverge between native English and Mandarin Chinese speakers.

My expectation is that spoken language will eventually go the way of handwriting: creature comfort, dying art, what once defined the best of us but becomes in many cases an indulgent inefficiency. How?

Anybody who dares to at this point, has realized they can jam wires into the human brain and let it learn to control machines on the other end. It's already beyond that in fact, with embedded communication devices being the next step, stepping shoe now currently in air: you'll see in a few days in Nature how real the "Neurochip" already is.

People should stop pretending this is about helping paraplegics by playing Space Invaders or moving a cursor with mind control, or that we're only trying to help brain injury, stroke, or paralysis patients. This is about construction workers with better than human strength in their better than human limbs. We drive vehicles through obstacles on land at 10 times the speed human beings can run, and we fly vehicles at 800 times the speed we can biologically move ourselves. We are mentally capable of managing bodily abilities far beyond those with which we are born.

This is not only about helping the disabled, and it's not only about incredible speeds or strengths. It's also about perfectly able people who would rather control personal electronics with their thoughts than search for or decipher other remote control electronics. Personal electronics are going to be a lot more personal, too; these people will eventually prefer to have personal electronics embedded in their bodies and networked with their minds.

Don't worry about losing human language: we will only lose it when we'll be better off for it, when we communicate and think better without it. The translator here, with IBM and elsewhere is of course more narrowly focused, but with this we are converging on technological telepathy and obsoleting human language.

Human logic and good intentions have come at it from a more traditional, less technological direction, giving us Esperanto, Loglan, Lojban, etc. You've probably heard of only one of these, which you probably laughed at somebody for being Geek enough to know any of. Most of them have been great ideas and well executed, but despite inherent gains in efficiency or intellectual force they are nowhere near the markets and their returns depend on mass adoption. Technology is different, it's tied directly to markets and to private profiteering with immediate amplification of wealth among the wealthy. Human beings are not going to create a better enough language, soon enough, before we create a technology which in itself superior to all human language. BG

Re:Other Languages

[AhtirTano]

Language is not composed of words. It is composed of idiomatic phrases (idiomatic phrases do not mean what the words mean) only understandable in context.

That's like saying humans aren't composed of cells. We are composed of organs who's functions are not useful in isolation. Idioms are composed of words, and the words are vital. Just like an organ will cease to work if you change the component cells, the idiom will cease to mean what you want it to if you change the component words.

True automatic translation is not possible.

True automatic translation will be possible when we have true artificial intelligence. And not a day sooner. (But maybe a day or to later.)

Re:Other Languages

[rHBa]

I won't have to larn a frist linguage wunce I instorl firfox 2

Re:Other Languages

[Fred_A]

What is the English word that equals the Greek word "logos"?

Duh, that's easy, it's "small plastic brick" even kids know that !

Re:Other Languages

[Explodicle]

Keep in mind there's only one math, and it makes sense. There are thousands of langages, and none of them make any damn sense.

Re:Other Languages

[Shaper_pmp]

You can't fully understand what someone's saying unless you understand their cultural context, and how it differs from your own.

For example, what's the difference in (UK) English between: "I couldn't care less" and "I could care less"? In US English they're used interchangably, but in UK English they're opposites. There are many such words or phrases in the English language alone where the precise word chosen (or connotations of a word) totally changes the meaning of the entire phrase, even reversing it's meaning.

Another example would be a simple phrase in US English like "he was pissed"? US meaning is "he was angry". In UK English it means "he was drunk", and a word-for-word translation into greek it would be meaningless (the equivalent idiom in Greek would be something like "he took it on the skull").

Seriously - if you ever want to understand the drawback to automatic translation, try getting two Greek friends to talk colloquially to you, but translating each individual word into English - it's completely unintelligible.

Re:Other Languages

[ashooner]

What about the fundamental issue of grammar? How is a (subject)(verb)(object) language going o be translated live into a (subject)(object)(verb) language? Or the old "The man bites the dog" example from introductory german class. Perhaps this can be done, but there is going to have to be some temporal caching...

that's awesome

[Anonymous Coward]

chinese people can now speak like poorly dubbed kung-fu movies in real life!

subvocalization

[TubeSteak]

http://en.wikipedia.org/wiki/Subvocalization

Subvocalization is basically micro-movements of the muscles associated with speech. The Wikipedia article mostly focuses on reading & subvocalization, so I wonder, do you have to be trained to do it consciously?

http://en.wikipedia.org/wiki/Subvocal_speech_recog nition

This wikipedia article says that recognition is hard.

Re:subvocalization

[Frogbert]

Well luckily I've changed the article, so now sub vocal recognition is easy

Remind me to turn this off at work

[Joebert]

Electrodes are attached to the neck and face to detect the movements that occur as the person silently mouths words and phrases.

It's only a matter of time before this thing gets me fired.

Question of the Millenium

[richdun]

So let's say this works - which language will we use as a primary one now that it doesn't matter, since everyone can understand everyone else easily?

Anyone who has studied languages knows (not "no"s or "nose") that English absolutely sucks (as in is bad, not as in pulls air into itself), but we use it widely (as in across a large range of people and places, not as in having a large girth) in large part (as in a significant reason, not as in being a big piece of something) due to the primary sources of finance and technology being in English-speaking countries (not literally the countries, but their people).

I like the idea, and see the huge, positive social impact it could have, but I feel sorry for the guy/gal responsible for it to test its ability to translate into/out of English.

Re:Question of the Millenium

[davidsyes]

Aw (not as in awe-struck...) shucks (not as in stripping corn stalks) you're such cunning linguist....

Re:Question of the Millenium

[AhtirTano]

Sorry to say, English is not as unusual as you would like to believe. (I am a linguist.)

In many ways, English is quite simple. For example, our word order is very straightforward. I work with a language were the following is a normal sentence: "This is city New called York here." (This city here is called New York.) In fact, almost every permutation of those words would be valid without a change in the basic meaning (as long as "is" is the second word). This is a so-called non-configurational language. Parsing English is easy by comparison.

I work with another language were there is a slight stress difference between the sentences "That might be true" and "He's honestly picking his butt." The words "soup" and "shit" are differentiated by a 40-50% increase in the length of the last vowel. There is one word for both "blue" and "green", and another word for "yellow", "orange", and "brown".

As to the likelihood of this project succeeding anytime soon: Languages are often not directly translatable into each other. One language I work with has an entire part of speech I cannot adequately translate into English. I have to wave my hands and point to convey the same information in English.

Re:Question of the Millenium

[ObsessiveMathsFreak]

I have to wave my hands and point to convey the same information in English.

How is this substantially different to the way the majority of native english speakers convey information?

Re:Question of the Millenium

[awol]

I am a native English speaker (well Australian anyway). Members of my familiy have taught ESL (English as a Second Language) to adults and children. Friends of mine are speech therapists. For the last 15 years I have been doing business with people who have English as their second language.

English is trivial to learn well enough to communicate. The reason? You only really need to learn vocabulary. All the points raised previously about the difficulty of automatic translation are kind of true but, for English at least (and probably Spanish), irrelevant. This is because these languages are the most accepting of inappropriate use. Actually from a linguistic perspective there is a lot of variation that can at least provide syntactic correctness and hence still convey meaning. There is very little (any?) positional context in English, ie where the meaning of a word or phrase changes according to its position in the stream. Indeed even where this might be the case it is really only in quite sophisticated use in which these things exist and that can be ruled out when you are communicating with an identified non-speaker. Also most of the speakers of English language are most accepting of inappropriate use.

Further the language positively rewards inappropriate use by the nature of English art (poetry, prose, even comedy) where so much of the beauty of are is derived from pushing the edge of meaning by interesting context. It is true that many languages are the same but perhaps none so much as English (perhaps with the exception of Arabic where it seems to me [I am not a linguist] that metaphor has a large role in even basic speech).

Having lived in London for 10 years, I regularly see non-native english speakers from completely different language families (Arabic, Chinese, Latin, Germanic, Asian, Slavic) speaking to each other in English (with varying levels of brokenness) because it is easy for them to find "common" ground to refine the synonyms they are using, even those that they do not all understand. (Sure they are in London, but even at conferences outside the UK the same is true). Part of this is because of the English as Lingua Franca but part of that is because English (as a language) is so tolerant of bad grammar. It is rare for bad grammar to fundamentally alter the meaning of a dialogue. It is possible and it is easy for confusion to arise, but then so too is it easy to clarify the actual meaning intended.

Many of the "this is impossible" comments from previous posters seem to assume that the translation described is context free. Why should it be so? The translator can just as easily be fed context about environment as well as the surrounding language of the segment being actively processed. Such information would help the translation.

So automatic translation can work (even badly) and provide a good basis for an iterative approach to conveying meaning between parties that have no common language. At a basic level even if it is just "identifiers" (nouns), "doing words" (verbs) and "describing words" (adjectives, or adverbs) it will facillitate _communication_.

Re:Question of the Millenium

[AhtirTano]

It isn't Mandarin, which I happen to be studying at the moment, since it the tone changes the meaning of individual words, not whole sentences.

The change in stress changes one word from a modal meaning "remote possibility" to a compound verb meaning "pick the butt". The adverb "honestly" and the verb "be true" are homophonous, and word order doesn't matter very much; thus, the ambiguity. This language is O'odham. The other language, the one that lets you split "compound words" is Serrano. Both are Uto-Aztecan languages spoken in the American southwest.

Oh great...

[revlayle]

the last time i heard of people constructing a Tower of Babel, the whole world got toally pwned and no one could understand each other. well, not much different than it is now is it.

/not religious

Re:Oh great...

[aussie_a]

Apparently that happened about 2,000 years after the Earth was first formed. Considering that happened 4.57 billion years ago I find it appalling that only now are they attempting to translate the numerous languages that resulted from the Tower of Babel travesty. I mean, how lazy can you get? What have they been doing all this time?

Re:Oh great...

[CmdrGravy]

The Mine Of Lebab ?

Simulated telepathy.

[EnsilZah]

I find that alot of my thought process is subvocalized.
I was wondering how hard it would be to translate that into audible words and transmit them at a volume relative to distance from the receiver.
Then you could have a social experiment where a group of people live together for a period of time while equipped with these transceivers.

Ain't Going to Happen

[eean]

Don't worry, they've been working on machine translation since the 60s and fully automatic translation still sucks. Speech to text isn't so great either.

Language is complicated!

Re:Ain't Going to Happen

[Ougarou]

Nope, ain't going to happen.

Professor Schultz said: "The idea is that you can mouth words in English and they will come out in Chinese or another language."

He must know that language is more then just words, there is grammer to. Then there is a story line, context, etc. etc. Let alone the fact that most people don't finish their sentence when they are having a conversation (tape a conversation and check it).
And for those experts who said: it showed the technology was "within reach". Stop calling yourself expert, because this is like telling people: computers can play chess now, so they will probably take over the world in a few years.

Re:Ain't Going to Happen

[schon]

language is more then just words, there is grammer to.

Ther si alos speeling (which you might want to work at improving, considering you misspelled two words under five letters each.)

Re:I don't get this

[thewils]

Well, I was trying to give a simple example. It can get quite convoluted. Check out Mark Twain's essay on the Awful German Language.

"The Germans have another kind of parenthesis, which they make by splitting a verb in two and putting half of it at the beginning of an exciting chapter and the OTHER HALF at the end of it."

Could be a huge improvement

[edwardpickman]

Any chance it could correct Bush's english? Could help the rest of us understand what's he's trying to say.

let me be the first to say:

[monopole]

"All your base are belong to us!"

Here's what you'll look like

[rhysjj]

Some photos of the electrode arrangement needed on the face:

http://www.flickr.com/photos/rhys/260069248/
http://www.flickr.com/photos/stasarama/245979951/
It's still a lab prototype of course, but a massively impressive one. I'm very pleased to see articulatory speech recognition (that's the main research area in this particular project, rather than the translation itself) get written up by the BBC.

Re:Here's what you'll look like

[rhysjj]

It's true that articulatory speech recognition should be easier than automatic speech recognition (ASR) based on waveform analysis alone. It's massively unfortunate that ASR research has, at least for the past 20 years, concentrated mostly on the latter and not the former. Janet Baker, whose MIT PhD introduced Hidden Markov Model (HMM)-based ASR, and opened the door to companies such as Dragon (which she and her husband founded), is herself now saying that HMMs are rubbish for speech recognition. I desperately hope that through this CMU project, and others, that people will start to take note of this.

I think you're entirely correct that the machine translation (MT) stage is a bolt-on in this particular project. This project is I think a vehicle for articulatory ASR rather than MT. But I wouldn't be so keen to dismiss MT efforts altogether. It's true that in some ways the current deployable systems make gross assumptions about language, which may be even worse than the assumptions ASR systems make about speech (that's particularly true of purely statistical MT systems). But Google and others have apparently shown that with a large enough corpus, you can get results that extend beyond simple phrase-book look-up quality.

There's one main question facing the researchers at CMU, I think. That's whether people will be happy to stick a dozen electrodes on their face in order to achieve speech-to-speech translation, or whether they'll prefer to speak into a microphone and have a speech synthesiser (e.g. the open-source Festival, partly developed at CMU) speak the result. I'm not entirely convinced they will, but I'd be absolutely delighted to be proved wrong.

Communication != understanding

[schwieter]

"The Babel fish, by effectively removing all barriers to communication between different races and cultures, has caused more and bloodier wars than anything else in the history of creation."

Could this allow the mute to speak?

[Allnighterking]

If it is using muscular sensors to "detect" sounds then wouldn't it be possible to create one that would allow the mute to speak? One would think that an English to English or Chinese to Chinese translation would allow then to perfect the detection process, and aid any number of people who can't for whatever reason speak but who can mouth words.

Killer App

[jshazen]

I can't believe nobody's posted this yet. This would be *really* useful as a *mono*-lingual translator! Build one of these into every cell phone, and suddenly I don't have to hear your inane conversation just because you happen to be sitting next to me in the plane.

This should be *much* easier to do that the version that actually translates, and it would add nearly as much to quality of life of the user and everyone else in his environs.

Continued..

[lobotomir]

The Tower of Babel Translator, by effectively removing all barriers to communication between different races and cultures, has caused more and bloodier wars than anything else in the history of creation.

translation, diversity, understanding

[zeromorph]

Well, the problem is that understanding someone is far more than parsing the other's sentences. It's getting to the point where you understand what meaning the other intended to convey. That's here all this machine translation still fails (and probably will fail for a long time to come). Because for that you need a lot of backround knowledge, you actually have to attune yourself to the experiences, the culture of the other. And that is a large part of what is learnt in a foreign language course.

All this automatic translation feigns that you understand the other, but actually your interpretation might be very different from the intended meaning. Sometimes a rough understanding might work, but mostly it you run into problems later. You might discover real referential differences, like you two where talking about wo very different things, but also interpretational differences or social misunderstandings which might result in severe discord.

A good way to test this are jokes, because they are such a condensed way of cultural meaning.

But this works also between varieties of one language, e.g English. Are you really sure an American fully gets what a upper middle class person from India is telling him/her about her feelings or experience, just because both of them have English as their mother tongue?

Understanding the other is an undertaking that costs a lot of effort and machine translation helps very little with that. Appreciating diversity, like appreciating everything else, demands effort and dedication and there is no short cut.

"My hovercraft is full of eels"

[shotgunefx]

Clerk: Ahh, matches!

Hungarian: Ya! Ya! Ya! Ya! Do you waaaaant...do you waaaaaant...to come back to my place, bouncy bouncy?

Clerk: Here, I don't think you're using that thing right.

Hungarian: You great
poof. Clerk: That'll be six and six, please.

Italian?

[Ruvim]

I have to wave my hands and point to convey the same information in English.

Must be Italian?

The Tower of Babel

[Jinky Williams]

Actually, it's been my understanding that it wasn't because they were somehow trying to build their way to heaven, but rather because they were going against the edict of God to go forth into the corners of the world and prosper and instead vying to stay in one place. He then frustrated all communications efforts, not just those associated with building the tower, so that people would congregate with like languages and scuttle off to their own corner.

The God of the Bible would feel not threatened in the least, I think, by humans dorking around and trying to build a heavenscraper.

Translators of Zero Wing contracted...

[ElboRuum]

A few samples from Japanese->English:

"Those walls are several feet thick and can hold back millions of gallons of water..." translated to "High columns having much fat toe plus can carry big number aqua litres"

"I'm not feeling very well, do you have some aspirin?" translated to "This day of my health is in negative. In my possession of you are pills?"

"All of your bases are in our possession." translated to "My tank is fight."

And so on.