Slashdot Mirror
Robotic Hand Translates Speech into Sign Language
usermilk writes "Robot educators Keita Matsuo and Hirotsugu Sakai have created a robot hand that translate the spoken word into sign language for the deaf. From the article: 'A microchip in the robot recognizes the 50-character hiragana syllabary and about 10 simple phrases such as "ohayo" (good morning) and sends the information to a central computer, which sends commands to 18 micromotors in the joints of the robotic hand, translating the sound it hears into sign language.'"
Good lord! I imagine the Japanese language with its 1945+ character alphabet is hard enough to learn; learning Japanese sign language must really suck.
You know what would really spoil those deaf kids is, instead of a robot doing sign language, a robot that shows images or words based on what a speaker says. I know, I know; creating a robot to do this is a feat within itself and impressive in its own right, but perhaps there are better ways of communicating with a robot if it can already perform more than adequate speech recognition.
Falun Dafa is good!
They only need to put it on wheels and it can become a scutter.
Additional warning:
Do not let this robot pat you on the back whilst near the top of the stairs.
liqbase
Call me culturally insensitive but, why not simply translate speech to text?
... Does it also distinguish between different 'dialects' in sign language?
I seem to recall that sign languages differ between countries, same as 'natural' language.
However this is really great for deaf people.
80 CC D8 AF AE D3 AB 54 B7 2E CE 67 C7
signing "I'll be back"
Would this not be more useful as software, able to render simple 3d hands with low microprocessor overheads, and preferably available for mobile phones and PDAs?
Deaf people could carry a PDA, and when they need to find out what someone is saying, they can hold the PDA up like a microphone, and watch the screen, assuming the translation is at least reasonable accurate...
Of course they could lipread too but some find that harder than others, and this could also be used eventually to cross language barriers?
I imagine it's extremely hard to lipread a foreign language.
Tickle... Amy... Tickle
http://www.imdb.com/title/tt0112715/
So it's not just a hand, but a hand with two legs!
This gives a whole new meaning to the phrase "talk to the hand"
Yes but not nearly as intimidating. Who's going to get their lunch money taken -- deaf kid with a PDA, or deaf kid with a giant robot hand?
Since sign languages are different all over the world, I don't know if there is the same problem in Japan, but:
American Sign Language is not English (American or other).
Thus, translating speech to ASL would reach people that that understand ASL but don't read Englih.
Exam 4/C again. Maybe I'll do better this time.
After reading a couple of replies to my parent post, I was thinking about people that might understand signing, but not read or hear.
It is my understanding that children can learn to sign before they can learn to read. (In fact hearing children can learn to sign before learning to speak.)
Similarly, developmentally challenged people, such as certain people with Down's Syndrome, never learn to read, but can sign just fine.
Reading takes certain specific brain functions, and it is not inconceivable that there are people who have had head trauma that damaged the reading part of their brain, and the hearing part, but can still understand sign language.
These are just quick thoughts and may have lots of holes, and little sense. Please feel free to expand/correct/flame/whatever.
Exam 4/C again. Maybe I'll do better this time.
There's a researcher at Gallaudet working on the other side of this equation with a system designed to recognize sign langauge, which seems like a much harder problem.
ASL isn't like English in that there are always specific words- a lot of it has to do with spacial context (where in the signing space the sign was made) and a whole class of signs that don't translate directly into words (they're hand shapes which can translate into an event or a description of an object or set of objects).
And, as the research page shows, facial expressions and even facial movements can be part of a sign.
Of course, this is American Sign Language, Japanese Sign Language may be very different.
In the early 90's I worked with the robotic finger spelling hands called "Dexter" & "Ralph". Those devices were intended for individuals who are both deaf and blind. An individual with this kind of disability must rest their hand on the back of someone's hand (or on the back of the robotic hand) and feel letters as they are signed by the hand/fingers.
Bah, Joel invented this on MST3000 years ago. Where's my edible sneakers?
I've had enough abrasive sigs. Kittens are cute and fuzzy.
Don't you guys ever consider the fact that some of these breakthroughs are not built for commercial applications?
Instead of trying to analyze these achievements in the rather constricted mould of "Why not 3D graphics" or "Why not text on a screen", consider the use of this technology in the future - when say, the robots to help disabled people finally get off the assembly lines. By then, this process would've been refined to the point of being able to do an excellent job in communications.
As a researcher in the field of robotics, a lot of work which I do or goes on around me, has definite implications - if not now, atleast in the next decade or so. And don't we owe to ourselves to look at developments such as this just for the *sake of the development itself*?
If Bill Gates had a dime for every time a Windows box crashed...oh, wait a minute - he already does.
From my observation, much of the "color" or entertainment value in signed conversations comes not from the movement of the hands but the expressions on the face etc. combined with the movements. Clearly, this robot is still nowhere near being capable of the same range of expression as a human being. As a simple test, I'd like to see the robot tell a joke and get the same laughs as a proficient human signer telling the same joke...
I've abandoned my search for truth; now I'm just looking for some useful delusions.
But kids don't spontaneously pick up on writing. What you are asking these deaf children to do is learn ASL (which is basically not written), and the written form of a foreign language. That's like only hearing English (no writing) and only seeing written Chinese (no speaking). That's tricky to learn, and almost impossible to do without extensive training.