Slashdot Mirror
Speech For The Deaf
I am linus's ho writes "CNN is running a story about gloves which transelate sign language into audble speach, in a stephen hawking type mannor, only, i suppose, much different. The article can be found here"
← Back to Stories (view on slashdot.org)
I can mount my expedition into the jungles of Africa and search for King Solomon's mines...
If your bitterest enemies are people who hack the heads off civilians, then I would say you're doing something right.
A well-mannered spokesman for Stephen Hawking assured the public today that he will continue to live in his manor. He intends to use his existing traslator to audibly speak as he does now, and will be no meaner.
This only helps one way in the conversation. It can help the deaf person be understood by those who do not know sign language. The deaf person still has to read lips (or have a pad with speech recognition) to know what is said in response.
I was taking one day at a time, but then several days got together and ambushed me. (from a Rhymes with Orange comic)
"CNN is running a story about gloves which transelate sign language into audble speach, in a stephen hawking type mannor, only, i suppose, much different. The article can be found here" "transelate"? "audble"? It seems to me that we need gloves that translate badly spelt words into the correct spelling as one types! :-)
save the GNUs!
Hmm, that sounds too complicated. I think I'll just use my eyes...
I'll finally be able to understand what gang-bangers are saying when they make those hand signals.
For hizzle my schnizzle...
Finally, math books without any of that base 6 crap in them.
Anyone who's spent any time around those who speak ASL or any other sign language as their primary language know that there's a hell of a lot more to sign language than the hands. It's also body posture, relative positions of the hands, and especially facial expressions. The main distinguishment between a question and a statement, for example, is all in head posture and facial expression. Another thing that this doesn't address is what's known as classifiers, where the signing person identifies some hand shape and/or position in space as a particular object/person and then uses that same shape and space in the way we would use a pronoun. This is not something I can see software picking up from mere gloves. (BTW all of this is hyper-simplified both by the fact that I myself do not know ASL and by the fact that I'm not discussing this in an ASL-technical forum).
Bottom line: it may have some limited utility in some very special conditions, but it will not simply allow a deaf person to put on a pair of gloves and have an instant voice.
7 November 2006: The day Americans realized corruption and incompetence weren't addressing 11 September 2001
"All art is quite useless" -- Oscar Wilde
"All art is quite useless." -- Oscar Wilde
And this is a big win over simply typing what you want to say in on a keyboard and outputting it via a speech synthesizer? This sounds like yet another case of a solution in search of a problem to me.
"Freedom means freedom for everybody" -- Dick Cheney
I wonder if they have a special macro for rapid repetitive back and forth motion to be translated into a sultry "Oh yeah, right there baby! Right there! That's the stuff!"
Karma: Marginal (mostly due to the border around the website)
Amy the gorilla runs around gesticulating idiotically
"Peter! Peter! Amy can't get stupid glove off! Peter! Glove stupid idea! Give banana! I kill you!"
The angel in the oatmeal.
Now your kung fu can speak for itself.
What we call folk wisdom is often no more than a kind of expedient stupidity.-Edward Abbey
... I don't typically flip people off until they're facing away.
"Derp de derp."
they did this with the signing gorilla 5 years ago. ./ editors: just because the website says it was posted today doesn't mean it is fresh news.
also, just because the website says it was posted last week doesn't mean you shouldn't post it front page just because you missed it.
MARIJUANA, SHROOMS, X: ONLINE?! - E
If it is a word for word thing, then it will sound like broken English, and then when I do idiomatic sign, just like speaking idiomatic english, you have to infer the context to get the meaning.
Heh. this is ridculous. Why people just don't learn sign is beyond me.
spam, spam, spam, spam, e-mail, news and spam.
A much more informative page that explains the technology (called GRASP - Glove-based Recognition of Auslan using Simple Processing) briefed in the CNN article can be found on Waleed Kadous's website:
GRASP Site:
http://www.cse.unsw.edu.au/~waleed/thesis.htm l
More generalized Gesture & Sign Language Recognition Research:
http://www.cse.unsw.edu.au/~waleed/gsl-rec/
Also see the self-proclaimed Gesture Recognition Home Page (good resource, tons of links)
http://www.cybernet.com/~ccohen/
Or just search google like I did for 'Machine Gesture Sign Language' and get a wealth of links.
</whoring>
If I had a sig, this is where it would be.
The awful 1995 movie Congo featured a gorilla wearing gloves that, in the movie's world, translated sign language into speech.
Upon first look these 95% does not seem that impressive. 1 in 20 words wrong or every third sentence gives across the wrong meaning. This would not be useful for effective communication.
But to give proper credit to the reseachers, understanding signs is very difficult: atleast, if not more, difficult than comprehending speach. Signs are not just a shape of the hand. Words and phrases are distinguished by differences in shape, location, and motion. Plus there are other subtle differences that usually cannot be picked upon by non-signers. Much in the same way there are subtle differences that speakers can pick up between similar sounding words but cannot hear with out the context of language.
The only reason that speakers are able to comprehand speach in real time is because they have an area of the speciallized for comprehension. Without this area - known as Wernicke's area - the mind is simply not quick enough to understand speach as quickly and fluently as it does. This is why, despite a computer's much faster processing ability, it is unable to comprehend speach as well as we do.
The amazing thing about Wernicke's Area is that in the case of signers it is able to leap from auditory comprehension to visual comprehension. So all the power that goes into speach comprehension is also used in signing comprehension. In this respect a computer picking up 95% is an impressive accomplishment.
1. Are there different versions of the middle finger gesture? One 'adult', and another 'radio edit'?
:P
2. When will we see a mod of this thing to a baseball glove so that we can get a translation of what the manager converse to the players on the field, live via TV?
I'm only half joking.
geek page at KY speaks
It is good that people are thinking about how to help the deaf, I just don't see this as all that useful. I am amazed that they have gotten up to 95% accuracy...though I suspect that this is with someone who went through a long training with the device, and they do mention that it has to be calibrated...a lot like the speech to text programs that never really caught on.
I think the annoyances would outweigh the benefits. They would be a pain to deal with...you would constantly have to turn them on and off so they don't start speaking when you are simply using your hands to say, pick something up. They would likely be very easy to break...your hands put things through a lot of stress.
Plus, a hearing person who is motivated can communicate with a deaf person if they put forth any effort. My high school had a program for deaf people, so I saw many of them around school...and never had a problem communicating with them. We did not use proper sign language like they used with each other, but a mix between gestures that we could all figure out, and lip reading. I have run into several of them since then and did not have any trouble communicating without pen or paper. My guess is many of them would rather communicate in this way, than deal with a flawed system that is a pain to use.
That being said...it is good that research is being done on the topic, because it not only opens people's eyes to the hassles that the deaf must go through...but it opens up a possibility where maybe in the future a system like this could be worked out that is more effective...though I think it would be very hard to do, perhaps it would not be impossible.
While I think you give valid examples of sign language that a glove-based translator will be hard pressed to recognize and translate, I think you may be overlooking the utility of this glove.
There is no reason to suppose that new signs cannot be created specifically for the glove so that almost anything that can be expressed in English can be expressed with the glove. A deaf person will simply have to learn the new gestures to fully utilize the glove, in the same way that a typist has to learn a QWERTY or Dvorak keyboard.
Once this has been properly tested and developed with the help and participation of the international deaf community, I see no reason that a deaf person, regardless of nationality cannot use the glove to communicate with a hearing person in his/her native language. At least, one-way communication will possible in multiple languages. Add a speech to glove-language translator and you have a two-way system.
Heck, with this glove, a deaf person will have an advantage over hearing people. He or she may make him/herself understood in several languages. Just a thought.
Okay, folks, can we PLEASE quit it with the "This is totally useless! Understanding sign language requires blah blah blah etc." comments? Just because YOU can't think of any reason to use it, doesn't mean there isn't valuable scientific knowledge being gained here. Maybe this will lead to a breakthrough in visual recognition systems. Who knows? Crapping on other peoples' work before much is even knows about it... great. What have YOU contributed to human progress?
"Destroy science and religion. Science would re-emerge exactly the same; but not religion." - Penn Jillette, paraphrased
Besides, there is no reason to suppose that new signs cannot be designed specifcically for this glove so that it can be used to say anything. And not just in English, but in any natural language. The deaf will just have to learn the new signs, a small effort compared to how useful it can be in the long run.
Secondly, I feel it does offer an advantage -- it uses the medium of communication that the Deaf themselves prefer to use. Sign languages are not a word-for-word translation of English. No offence to Deaf people intended, but if you've ever read any unedited text by Deaf (say, e-mail), their grammar is sometimes poor. It's not because Deaf are stupid, it's just it's not their language, and the language that they do use -- ASL or Auslan or whichever sign language -- is so totally foreign to spoken language that it is hard for them to come to grips with the grammar. So forcing them to use English is already forcing them to do something they'd prefer not to do. It's like saying why bother with Babelfish when anyone could learn Spanish?
There is no reason, with lots of hard work and patience that a deaf people (especially starting as young childern) cannot learn to speak properly using their voice and mouth.
There is no reason gay people cannot learn (with lots of hard work, especially starting as young children) to pretend to enjoy copulating with the opposite sex.
The whole "Deaf Culture" movement that encourages deaf people to make themselves distinct (by not speaking or reading lips) from the "Hearing" people is terrible.
The whole "gay culture" movement that encourages gay people to make themselves distinct (by not copulating with the opposite sex) from the "straight" people is terrible.
If a deaf person can learn to speak and read lips they can get along extremely well, almost unnoticed in everyday life.
If a gay person can learn to pretend to enjoy copulating with the opposite sex they can get along extremely well, almost unnoticed in everyday life.
Actually, there is a reason for neither of the two happening: the gains (of being almost unnoticed in everyday life) do not outweigh the sacrifices (hard work and patience learning how to do something extremely discomforting, especially starting as young children).
Sign languages typically have a finger-spelling system as a fallback. Say you want to specify a name, like McGill. There's no sign for it, so you finger-spell it. Each letter has a corresponding sign. ASL has single-handed fingerspelling, while Auslan has two-handed fingerspelling.
But it's not sign language. A typical sign might be something like "thank" where you touch the chin and move the hand forward (at least in Auslan anyway).
Recognising sign is much harder. Fingerspelling is pretty much position-independent. For sign recognition, you need to track the person's position and motion.
This is not to cast any aspersions on Ryan's work, of course -- especially that he made it so cheaply. I think that's a major accomplishment. But designs for such gloves date back to Grimes' work in 1983 (patent 4,414,537), or James Kramer's work at Stanford in the early 90's (patent 5,047,952).
Everyone's going on about all the times this won't be useful... but think about when it will be.
- Asking for directions on a streetcorner
- Finding out bus fare
- Ordering dinner in a restaurant
- Picking up the phone when no one else is around
And so on. There are a ton of mundane things that will be very hard for a deaf person to do without the assistance of a hearing person, but might be pretty important. A device like this might make it much, much easier and safer for a deaf person to travel alone... like I did, traveling around Europe with my crappy little Radio Shack five-language translator. Yeah, I couldn't have deep philosophical conversations in French, German or Italian (Spanish I do ok in on my own), but I could get food and a hotel room.
Don't you wish your girlfriend was a geek like me?
Virtual reality? Eh, I don't know about that. But I'd sure be interested to know what that thing would have to say if someone who was wearing one of those gave you the finger. :)
Surfing autopr0n?
Fwap. Fwap. Fwap. does not compute.
Speech recognition is not a dead concept. You can read all about the C-Print system that is being developed at the National Technical Institute for the Deaf (NTID) (And RIT. Shameless plug, since I'm working there temporarily). The C-Print system uses both a typing and voice recognition system to create the captioning. The captioning is then displayed in realtime on a notebook computer screen or VGA monitor just below the line of view of a student. (There's also a webcam feed so the students can use that to watch both the captioning and the lecturer.)
Advantages of the system: Creates a transcript of the class. This is automatically saved and is avaliable for the student to review later. (Remember, deaf students can't exactly take notes while they're watching continuous action). Don't nessecairly have to pay for a notetaker and interpreter for those classes.
Disadvantages: Cam is pretty stationary, and is only really useful for lecture classes. Cost is a bit higher, in equipment and captionist fees.
--You will rephrase your request for me to go to hell. Goto statements are not acceptable programming constructs
Not exactly. His translates fingerspelling into letters. I still give him major props, it's still damn good work for a kid of his age, but it's still easier for a deaf person to pull out a pen and paper than to use his tool.
Sign language is a much bigger concept than fingerspelling. Other people have given much better explanations than I have.
--You will rephrase your request for me to go to hell. Goto statements are not acceptable programming constructs
I have a great fear that we're progressing into a world where English is the norm, and that bothers me quite a bit. Sign language is by all accounts a valid and full featured language, just not a spoken one. I don't believe that people should be required or even expected to communicate in a non native language if they don't want to...and especially if they can't. For chrissake, would it kill people to learn a little bit of A/ISL? And similarly would it be that much of a problem for people in countries like the US to catch up with the much of the west and expect fluency in multiple tongues? This thing comes across as a sort of disability device and that sickens me. Sign Language is not a disability. Nor Spanish, nor Hebrew, Pashto or anything else. And despite claims by the right, English is not the official language here in America. Whew. Breathe.
It makes more sense now than it did then. Back then, it required more hardware than you could conveniently lug around. Now, you could just wire the glove up to a Palm Pilot or something comparable.
Pull through to the window. Then, break out the pen and paper and hope the person at the window can read. Or, find hearing friends to go with you.
--You will rephrase your request for me to go to hell. Goto statements are not acceptable programming constructs
I know of the regional difference (even in Australia, there's a northern and southern dialects and both dialects are dialects of British Sign Language) but (a) they affect less common signs (b) our system learns, so all it takes is someone to demonstrate a new sign and/or variant -- much like speech recognition systems today can cope with different accents.
Our initial attempt would be to go for Signed English to begin with (there's no translation necessary with signed english; imagine English, except the words are spoken in French, but without the grammar of French), then move on to full ASL.
I'm also aware of the cochlear implant debate. And you've missed the most important differences: (a) this solution embraces sign, rather trying to supersede it; (b) this technology can be used by those who are born deaf as well as those who are postlingually deaf; (c) it's a non-intrusive and low-risk technology - you can take off the gloves whenever you like.
And there's no reason that hearing people should learn how to sign is there? If you're going to suggest that more deaf people learn how to become more oral, then you should also be acceptive of the fact that more hearing people should know how to sign. If both sides did that, then there would be a more fluid method of communication rather than people going, You're imperfect because you cant do what I do!
spelt {Pronunciation Key (splt)} v. A past tense and a past participle of spell.
Spelt is as correct as spelled, even if it is the less common variant these days. Spell is an irregular verb. English has many irregular verbs. This is no more ironic than a black fly in your Chardonnay.
However, your use of "I" rather than "It" in a post criticizing a non-existent spelling error in someone else's post criticizing the spelling errors in a Slashdot article header partakes somewhat of ironicalness... ;-)
hearing person example: my wife knows a couple of German dialects after spending one of her high school years in Germany, an exchange program. She and I have compared notes on this phenomena and it's weird/freaky/cool.
And, oh yeah, 'will it learn'. OK, it has the capability to learn? That's good. But I don't see it being practical for mainstream use. It would just be another device to draw attention to myself. I get enough 'attention' or people looking at me warily because of their noticing my hearing aid when I go to McDonalds and place an order at the counter.
Don't let me discourage you tho. I'll go on record and say it can serve an application space in certain situations. But nothing can replace the communication and human bond, the discovery process, the mind-blowing experience of understanding that the person signing to you isn't that much different from you, once you get over the (rather small) learning hump (ASL can also easily incorporate mimetic actions...it's fun, it's playful, it gets the point across). The glove can serve as tool for initiation into their world (the Deaf), I'll give you that. But would the person who only relied on communication with the Deaf with that device, would they not feel that it felt artificial? They would chance to discard it at some point, I would like to believe. I think that's a sad thought that some would only ever be willing to communicate with Deaf folk if they had to wear THE THING.
And you don't have to sign completely pure American/British/Auslan Sign Language, lots of Deaf folks use English or have an understanding of the basics of said language in said regional area. You can 'borrow' the signs and sign in the spoken language word order. It's a bridge. It's a pidgin. Both sides can reach an intermediary form that's efficient enough for both. Then it will evolve. Give it a chance, don't be afraid.
I've always been willing to meet hearing folks halfway by looking at them directly to their face, talking as clearly as I can and asking them to repeat when I don't undertand something.
Half-way. Evem I can do it. It ain't that hard I like to quote Larry: there's more than one way to do it. :)
spam, spam, spam, spam, e-mail, news and spam.
Anway, I should have trotted this out earlier. Some fair example here, a little static and some of the signs are outdated, however:
http://dww.deafworldweb.org/asl/
The best place you can learn? 1) community college introductory course to ASL (again, american, make sure you try to pick up on the real language, not some manually coded spoken sign) ... and 2) Deaf clubs. Look in the phone book or call someone in the United Way in your area, they might have someone on staff who knows these things. Go to the club after you have about 6 months under your belt :)
spam, spam, spam, spam, e-mail, news and spam.
Everyone seems to be focussing on the limitations of this thing and saying it will only be useful in a limited set of circumstances. Heck, EVERYTHING is only useful in a limited set of circumstances.
However, this will be VERY useful in a large number of circumstances. Think about how many people you interact with in your day to day business, and how often you use speech to do this, I'm sure only about 1%, if that, of people can understand the ASL for "cream and sugar please" or whatnot. This glove will allow deaf people to make themselves heard to those around them, possibly not accurately, certainly not poetically, but heard nonetheless.
Its a better idea than esperanto and vanilla coke.
Spelling errors usually grate on me, though if they're occasional and minor I don't bother complaining. We all make mistakes and our fingers are all just slightly too thick for the keys sometimes. I've forced myself to add common misspellings to my visual dictionary, like "theif" vs. "thief."
But for some reason misspelled words stand out from the page as if they're bold, italic, and <blink>ing, and when I'm reading or skimming I almost always come to a dead stop when I hit one. It's kind of a sudden "something here isn't right" feeling. Severely mangled grammar gets me too, though less reliably.
And yes, sometimes when I'm reading newsgroups or message boards it's a brain feature I really wish I could switch off!
It's not all bad, though: when I was in school my friends used me as a high-speed term-paper proofreading machine. Five seconds of scanning a typical typewritten page and I'd've spotted most of the spelling errors and a lot of the grammar errors. And I don't think I ever once had to have one of my papers checked for spelling and grammar, a big time-saver in the days before decent automated spell checking.
Slashdot could use a few editors with this skill/curse, I think.
To bring this back on-topic... I wonder how strict the gloves in question will be about spelling (of words built out of signs for letters) or exact finger and hand positions. Seems like you'd want them to be as generous in that regard as possible, which probably makes the problem that much harder to solve.
Gay people will at least communicate with straight people. Your analogy sucks.
Heh, yeah there is a reason. 99 times out of 100 the signed language of a region can be traced back to the deaf kids. Deaf kids invented sign. They wanted a way to communicate with their friends, and family, so they would agree on common meanings and build the signs from there. Usually the deaf kids are on their own, they have hearing parents, and so on. With the residential style school setting, it give them more time to refine it and make it something that was 'their own', usually because the schools promoted the 'Oral' tradition of teaching children (voice only).
A lot of the information I have is mixed with a little european (deaf) history and mostly american (deaf) history, but from what I understand, the main reason why there's regional differences is the same as spoken languages: they evolved. The languages reflect the needs of the people at the time. Language is a tool. Sure, the language is more refined today, but it had to start somewhere, and it is the deaf kids who started it.
Hee, that's my really simple explanation. I could sit and type out a longer, more accurate answer, but this is Slashdot, and I'm sure somebody else will put in their 2 cents or say 'no that's wrong, blah blah'.
I just woke up and haven't even had my first coffee yet, heh.
Hope that helped.
spam, spam, spam, spam, e-mail, news and spam.