Slashdot Mirror
Gloves Translate Sign Language Into Auditory Speech
Zothecula writes about some pretty cool sensor gloves. From the article: "Since beginning in 2003, the Microsoft Imagine Cup has tasked students the world over with developing technology aimed at solving real-world problems. In this, its 10th year, students were asked to build their project around a specific Millennium Development Goal ... The winners have just been announced ... [and winning] first place (and US$25,000) in the Software Design category was the Ukraine's quadSquad with their EnableTalk gloves that translate sign language into speech in real time."
Another $20 an hour job gone...
Until I see a gorilla using them to talk about a secret diamond mine, I'm unimpressed.
"In God We Trust, All Others Pay Cash"
This is a cool idea. But it kind of only goes half way. Now they need a Thing that can translate spoken speech into glove movements. Oh, yea, baby!
"Although the software was developed under Windows Phone 7, the team was forced to turn to the older Windows Mobile platform for their entry because Windows Phone 7 doesn’t provide developers access to the Bluetooth stack, which is how the gloves communicate wirelessly with a mobile device running the translation software."
And that tells me all I need to know about WP7!
After watching the video, it seems that what they've done is create gloves which recognize the various fingerspelling signs. If somebody wants to sign "I need to withdraw money" (like, at a bank), what this allows them to do is to make the sign for "I", then "N, E, E, D", then "T,O", and so forth. Then the gloves feed that output into a TTS system. This works (because ASL users and English speakers share a writing system), but is horribly inefficient, and would be equivalent to a translation module that makes you speak every letter of the written words individually before putting the words into Spanish.
This is fundamentally different from "translating sign language", where the gloves would recognize the (much more complex and spatially oriented) sign for "I", for "need", for "withdraw" and for "money", and then translate that into "I need to withdraw money" and speak it aloud. Adding in the fact that ASL syntax is fundamentally different than in English, it's quite a tall order. Interpreters need not fear.
This is cool, nobody's denying that, and for some jobs, this might be great, but at the moment, I don't see it working much faster than taking out the requisite smartphone and writing down what you're trying to get across.
With a text to speech app?
Slashdot's rate-of-post filter: Preventing you from posting too many great ideas at once.
The gloves translate dactylology (finger spelling). That's fairly easy, but it's horribly inefficient.
'Real' sign languages (like ASL) are much harder to translate because they are somewhat non-linear. A single gesture can describe several things at once: size, direction, emotional state, etc. There's no way you can translate it without fully understanding the context of the speech. And we all know how good computers are at such tasks...
Only if they spell out each word, which is not generally what ASL users do. Even many phrases, my limited knowledge of includes "Thank You" which is one motion, not spelling out the words.
A grad student at Stanford at the Center for Design Research did this in the mid-1980s. It had to be connected to a workstation back then, of course.
Mod parent up! Finger Spelling is *not* Sign Language. If all this does is translate finger spelling into synthesized speech, the same thing could be done much faster and cheaper by just typing the words on a standard smartphone device.
This is not even cool. It is just, plain, wrong in so many ways. All of the money and hype spent making and marketing this device would reap 10X as much benefit if the same money were spent educating people about the real nature of deafness and sign language. The developers of this waste of time could start by taking a class about deafness themselves.
The fact that Slashdot perpetuates the inaccurate headline equating finger spelling with sign language just demonstrates how ignorant we all are.
^ This is exactly the kind of post /. will be filled with the day transporters or flying cars are actually built.
Close palm, raise middle finger. Universal language at it's best.
Then I guess it translates sign language. Finger-spelling is just an alphabet. I don't know about other sign languages, but ASL is a full language with its own grammar and conventions, and it would take a lot more than a glove to interpret it. Positions of hands with respect to the body are important, as are facial expressions, and ASL's pronoun system is largely spatial with the handshape only indicating the type of pronoun (e.g. personal vs. possessive). Even if a piece of technology could reliably capture and interpret all that information, you're still back at the problem of natural language translation into the destination language.
Still, a cool hack.
This actually does solve a real world problem
Use case?
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
It very much DOES, just not in the most efficient way.
while this may have limited to no REAL value at the moment, there were a LOT of saying the EXACT same thing about the internet in 92-94.
You naysayers might want to think about that.
I just want to know the use case. What if it were perfect and could precisely translate sign language into spoken or printed English - what advantage would it hold over a keyboard or smartphone or even a pen and paper? Plus, many (most?) deaf people can speak.
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
While I suspect that was a joke, in theory Kinect would actually be a better platform upon which to build an ASL-to-English translator, if not for the fact that the skeletal data it provides does not include fingers.
And the fact that you'd need to wear some kind of exoskeleton that mounts the sensor far forward of your body, so it can see what your hands and arms and face are doing.
Mobility is the point here. If you can sit at a computer, you can just type instead.
Of course you can just type on a smartphone too, without wearing a freaky bulky obtrusive glove. But others have already mentioned that.
Maybe your mute? you can't speak. sign language (real sign language) is much faster then typing or writing from what I understand. Wear the gloves leave your phone in your pocket, wear a small speaker around your neck. You look fairly normal standing at the bank instead of like a freak waving a clip board.
keep in mind to do sign language, you don't need to be able to see your hands like with typing. you don't have to fish a device out your pocket every time you want to talk...
It would be perfect for any situation where speaking isn't possible. and maybe even typing
Loud machine rooms? built in speakers in the sound protection gear.
scuba diving?
someone on a respirator?
I'm not saying it would be a airplane or internet, level game changer but I could seriously see folks coming up with some nifty uses for this.
Actually more I sit thinking about it more game changing I think this COULD be... ( if perfected of course)
You are right I suppose - if they could get it to do more than just simple letter signs (which are slower than typing). And if they could get it to translate sign to English. And if you could still use your hands while wearing the gloves.
But I gotta tellya, when stuff like this is happening, the uses for something that depends on exaggerated hand motions seems to diminish :)
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
Maybe your mute? you can't speak. sign language (real sign language) is much faster then typing or writing from what I understand.
If true, this is the really interesting part. If signs could be interpreted like typing, but faster, the gloves would replace keyboards in a way that voice-to-text can never do, even if done perfectly.
Can you be Even More Awesome?!
Olfactory Smelling.
I am anosmic, you insensitive clod!
(btw, I really am anosmic, a smelling device would be awesome :p)
I am an ACCA student. Got a query on Accountancy/Finance? Maybe I can help!