Speech For The Deaf

I am linus's ho writes "CNN is running a story about gloves which transelate sign language into audble speach, in a stephen hawking type mannor, only, i suppose, much different. The article can be found here"

Comments

You say Tomato, I say Tom Ate All

[SEWilco]

"...transelate...audble speach, in a stephen hawking type mannor..."

A well-mannered spokesman for Stephen Hawking assured the public today that he will continue to live in his manor. He intends to use his existing traslator to audibly speak as he does now, and will be no meaner.

Can't work in reverse

[Ra5pu7in]

This only helps one way in the conversation. It can help the deaf person be understood by those who do not know sign language. The deaf person still has to read lips (or have a pad with speech recognition) to know what is said in response.

never gonna work

[elmegil]

My wife is a sign language interpreter, so I have some experience from which to speak.

Anyone who's spent any time around those who speak ASL or any other sign language as their primary language know that there's a hell of a lot more to sign language than the hands. It's also body posture, relative positions of the hands, and especially facial expressions. The main distinguishment between a question and a statement, for example, is all in head posture and facial expression. Another thing that this doesn't address is what's known as classifiers, where the signing person identifies some hand shape and/or position in space as a particular object/person and then uses that same shape and space in the way we would use a pronoun. This is not something I can see software picking up from mere gloves. (BTW all of this is hyper-simplified both by the fact that I myself do not know ASL and by the fact that I'm not discussing this in an ASL-technical forum).

Bottom line: it may have some limited utility in some very special conditions, but it will not simply allow a deaf person to put on a pair of gloves and have an instant voice.

Re:never gonna work

[truefluke]

Exactly. One of the things that a reputable sign language course (i.e. should be a Deaf teacher) does is get people to use a facial mirror. It would be passed around the class and people would have to 'mock' a facial expression as a reaction to a sign, for example:

How do you feel? (happy/sad face)

Boo! (surprise)

And so on. Later they would be taught how to sign an imperative statement vs. an interrogative statement. Specifically, the eyebrows.(go look it up online, it's out there)

Also of note, shaking your head 'no' negates a signed statement in American Sign, example:

q: do you have money?

a: (shaking head) have money

That would be perfectly valid ASL. Even if they can teach this thing all the dialects and regional sign languages out there, it's the biggest waste of time I've ever heard. No mainstream application value at all.

You'd be better off learning some basic ASL signs and signing them they way English speakers are used to, in English word order.

Why?

[El]

And this is a big win over simply typing what you want to say in on a keyboard and outputting it via a speech synthesizer? This sounds like yet another case of a solution in search of a problem to me.

Re:Congo

[!splut]

Amy the gorilla runs around gesticulating idiotically

"Peter! Peter! Amy can't get stupid glove off! Peter! Glove stupid idea! Give banana! I kill you!"

Re:Gloves that improve spelling?

[colmore]

Commenting on posters' spelling = pompous.

Commenting on editors' spelling = legitimate expectation of quality from a service that claims to be "news."

It's silly to expect random people commenting as fast as they can in order to deal with a stupid default "oldest post first" system to be paragons of grammar and spelling. You're lucky to understand them at all.

I don't think it's too much to ask that Slashdot editors, or people taking the time to post stories, run submissions at least through a half-decent spell check.

RIght.

[truefluke]

I sign. And I'm hearing impaired. And there is no possible way that this thing would be able to keep up with my signing speed once I'm in the 'zone', or in deep, conceptual sign.

If it is a word for word thing, then it will sound like broken English, and then when I do idiomatic sign, just like speaking idiomatic english, you have to infer the context to get the meaning.

Heh. this is ridculous. Why people just don't learn sign is beyond me.

I am linus' karma ho

[philam3nt]

A much more informative page that explains the technology (called GRASP - Glove-based Recognition of Auslan using Simple Processing) briefed in the CNN article can be found on Waleed Kadous's website:

GRASP Site:
http://www.cse.unsw.edu.au/~waleed/thesis.htm l

More generalized Gesture & Sign Language Recognition Research:
http://www.cse.unsw.edu.au/~waleed/gsl-rec/

Also see the self-proclaimed Gesture Recognition Home Page (good resource, tons of links)
http://www.cybernet.com/~ccohen/

Or just search google like I did for 'Machine Gesture Sign Language' and get a wealth of links.
</whoring>

This project is more difficult than it appears

[blonde rser]

Upon first look these 95% does not seem that impressive. 1 in 20 words wrong or every third sentence gives across the wrong meaning. This would not be useful for effective communication.

But to give proper credit to the reseachers, understanding signs is very difficult: atleast, if not more, difficult than comprehending speach. Signs are not just a shape of the hand. Words and phrases are distinguished by differences in shape, location, and motion. Plus there are other subtle differences that usually cannot be picked upon by non-signers. Much in the same way there are subtle differences that speakers can pick up between similar sounding words but cannot hear with out the context of language.

The only reason that speakers are able to comprehand speach in real time is because they have an area of the speciallized for comprehension. Without this area - known as Wernicke's area - the mind is simply not quick enough to understand speach as quickly and fluently as it does. This is why, despite a computer's much faster processing ability, it is unable to comprehend speach as well as we do.

The amazing thing about Wernicke's Area is that in the case of signers it is able to leap from auditory comprehension to visual comprehension. So all the power that goes into speach comprehension is also used in signing comprehension. In this respect a computer picking up 95% is an impressive accomplishment.

Re:This project is more difficult than it appears

[anthony_dipierro]

Upon first look these 95% does not seem that impressive. 1 in 20 words wrong or every third sentence gives across the wrong meaning. This would not be useful for effective communication.

I think you're missing the point. 95% is certainly good enough for effective communication from one human to another. Alligator though one in twenty words is screwed up, the human on the other end is still going to understand water the signing human is saying.

Think Typewriter Keyboard

[Louis Savain]

While I think you give valid examples of sign language that a glove-based translator will be hard pressed to recognize and translate, I think you may be overlooking the utility of this glove.

There is no reason to suppose that new signs cannot be created specifically for the glove so that almost anything that can be expressed in English can be expressed with the glove. A deaf person will simply have to learn the new gestures to fully utilize the glove, in the same way that a typist has to learn a QWERTY or Dvorak keyboard.

Once this has been properly tested and developed with the help and participation of the international deaf community, I see no reason that a deaf person, regardless of nationality cannot use the glove to communicate with a hearing person in his/her native language. At least, one-way communication will possible in multiple languages. Add a speech to glove-language translator and you have a two-way system.

Heck, with this glove, a deaf person will have an advantage over hearing people. He or she may make him/herself understood in several languages. Just a thought.

Re:The Deaf should learn to speak with their mouth

[leob]

There is no reason, with lots of hard work and patience that a deaf people (especially starting as young childern) cannot learn to speak properly using their voice and mouth.

There is no reason gay people cannot learn (with lots of hard work, especially starting as young children) to pretend to enjoy copulating with the opposite sex.

The whole "Deaf Culture" movement that encourages deaf people to make themselves distinct (by not speaking or reading lips) from the "Hearing" people is terrible.

The whole "gay culture" movement that encourages gay people to make themselves distinct (by not copulating with the opposite sex) from the "straight" people is terrible.

If a deaf person can learn to speak and read lips they can get along extremely well, almost unnoticed in everyday life.

If a gay person can learn to pretend to enjoy copulating with the opposite sex they can get along extremely well, almost unnoticed in everyday life.

Actually, there is a reason for neither of the two happening: the gains (of being almost unnoticed in everyday life) do not outweigh the sacrifices (hard work and patience learning how to do something extremely discomforting, especially starting as young children).

Re:Isn't this old? (researcher's response)

[waleedk]

Actually, I did a bit more research on Ryan's work, and it turns out what he built was a system that converted fingerspelling, not sign language into speech.

Sign languages typically have a finger-spelling system as a fallback. Say you want to specify a name, like McGill. There's no sign for it, so you finger-spell it. Each letter has a corresponding sign. ASL has single-handed fingerspelling, while Auslan has two-handed fingerspelling.

But it's not sign language. A typical sign might be something like "thank" where you touch the chin and move the hand forward (at least in Auslan anyway).

Recognising sign is much harder. Fingerspelling is pretty much position-independent. For sign recognition, you need to track the person's position and motion.

This is not to cast any aspersions on Ryan's work, of course -- especially that he made it so cheaply. I think that's a major accomplishment. But designs for such gloves date back to Grimes' work in 1983 (patent 4,414,537), or James Kramer's work at Stanford in the early 90's (patent 5,047,952).

Where this will come in handy...

[Ironica]

Everyone's going on about all the times this won't be useful... but think about when it will be.

- Asking for directions on a streetcorner
- Finding out bus fare
- Ordering dinner in a restaurant
- Picking up the phone when no one else is around

And so on. There are a ton of mundane things that will be very hard for a deaf person to do without the assistance of a hearing person, but might be pretty important. A device like this might make it much, much easier and safer for a deaf person to travel alone... like I did, traveling around Europe with my crappy little Radio Shack five-language translator. Yeah, I couldn't have deep philosophical conversations in French, German or Italian (Spanish I do ok in on my own), but I could get food and a hotel room.