Speech For The Deaf

I am linus's ho writes "CNN is running a story about gloves which transelate sign language into audble speach, in a stephen hawking type mannor, only, i suppose, much different. The article can be found here"

Comments

Can't work in reverse

[Ra5pu7in]

This only helps one way in the conversation. It can help the deaf person be understood by those who do not know sign language. The deaf person still has to read lips (or have a pad with speech recognition) to know what is said in response.

Why?

[El]

And this is a big win over simply typing what you want to say in on a keyboard and outputting it via a speech synthesizer? This sounds like yet another case of a solution in search of a problem to me.

Re:never gonna work

[truefluke]

Exactly. One of the things that a reputable sign language course (i.e. should be a Deaf teacher) does is get people to use a facial mirror. It would be passed around the class and people would have to 'mock' a facial expression as a reaction to a sign, for example:

How do you feel? (happy/sad face)

Boo! (surprise)

And so on. Later they would be taught how to sign an imperative statement vs. an interrogative statement. Specifically, the eyebrows.(go look it up online, it's out there)

Also of note, shaking your head 'no' negates a signed statement in American Sign, example:

q: do you have money?

a: (shaking head) have money

That would be perfectly valid ASL. Even if they can teach this thing all the dialects and regional sign languages out there, it's the biggest waste of time I've ever heard. No mainstream application value at all.

You'd be better off learning some basic ASL signs and signing them they way English speakers are used to, in English word order.

This project is more difficult than it appears

[blonde rser]

Upon first look these 95% does not seem that impressive. 1 in 20 words wrong or every third sentence gives across the wrong meaning. This would not be useful for effective communication.

But to give proper credit to the reseachers, understanding signs is very difficult: atleast, if not more, difficult than comprehending speach. Signs are not just a shape of the hand. Words and phrases are distinguished by differences in shape, location, and motion. Plus there are other subtle differences that usually cannot be picked upon by non-signers. Much in the same way there are subtle differences that speakers can pick up between similar sounding words but cannot hear with out the context of language.

The only reason that speakers are able to comprehand speach in real time is because they have an area of the speciallized for comprehension. Without this area - known as Wernicke's area - the mind is simply not quick enough to understand speach as quickly and fluently as it does. This is why, despite a computer's much faster processing ability, it is unable to comprehend speach as well as we do.

The amazing thing about Wernicke's Area is that in the case of signers it is able to leap from auditory comprehension to visual comprehension. So all the power that goes into speach comprehension is also used in signing comprehension. In this respect a computer picking up 95% is an impressive accomplishment.

Re:This project is more difficult than it appears

[anthony_dipierro]

Upon first look these 95% does not seem that impressive. 1 in 20 words wrong or every third sentence gives across the wrong meaning. This would not be useful for effective communication.

I think you're missing the point. 95% is certainly good enough for effective communication from one human to another. Alligator though one in twenty words is screwed up, the human on the other end is still going to understand water the signing human is saying.

Think Typewriter Keyboard

[Louis Savain]

While I think you give valid examples of sign language that a glove-based translator will be hard pressed to recognize and translate, I think you may be overlooking the utility of this glove.

There is no reason to suppose that new signs cannot be created specifically for the glove so that almost anything that can be expressed in English can be expressed with the glove. A deaf person will simply have to learn the new gestures to fully utilize the glove, in the same way that a typist has to learn a QWERTY or Dvorak keyboard.

Once this has been properly tested and developed with the help and participation of the international deaf community, I see no reason that a deaf person, regardless of nationality cannot use the glove to communicate with a hearing person in his/her native language. At least, one-way communication will possible in multiple languages. Add a speech to glove-language translator and you have a two-way system.

Heck, with this glove, a deaf person will have an advantage over hearing people. He or she may make him/herself understood in several languages. Just a thought.

Re:Something else (researcher's response)

[waleedk]

Actually, I do feel I have a clue. Why? Because I've talked (or more accurately signed) to Deaf people. They seem enthusiastic generally -- but have concerns over practicalities like cost and reliability.

I know of the regional difference (even in Australia, there's a northern and southern dialects and both dialects are dialects of British Sign Language) but (a) they affect less common signs (b) our system learns, so all it takes is someone to demonstrate a new sign and/or variant -- much like speech recognition systems today can cope with different accents.

Our initial attempt would be to go for Signed English to begin with (there's no translation necessary with signed english; imagine English, except the words are spoken in French, but without the grammar of French), then move on to full ASL.

I'm also aware of the cochlear implant debate. And you've missed the most important differences: (a) this solution embraces sign, rather trying to supersede it; (b) this technology can be used by those who are born deaf as well as those who are postlingually deaf; (c) it's a non-intrusive and low-risk technology - you can take off the gloves whenever you like.