Slashdot Mirror


New Wearable Tech Translates Sign Language Into Text (thestack.com)

An anonymous reader writes: A new wearable technology developed by a team of biomedical engineers at Texas A&M University seeks to aid seamless communication between deaf people who use sign language and those who do not understand it. The arm device contains a network of sensors which track hand movements, as well as the electromyography (EMG) signals generated by the muscles in the wrist, and process and translate the different signals into text in real-time.The prototype currently uses Bluetooth to translate the sign language to a computer or smartphone.

32 comments

  1. huh ? by Anonymous Coward · · Score: 3, Insightful

    why cant deaf people just type their words into a TTS app running on their smartphones.

    1. Re:huh ? by Anubis+IV · · Score: 1

      why cant deaf people just type their words into a TTS app running on their smartphones.

      Maybe they're wanting to engage in public speaking with a mixed crowd of hearing and deaf individuals but are either self-conscious of how well they can speak or never developed the skill to begin with?

      Maybe they're online with their gaming console and want a way to communicate quickly to teammates without interrupting their gameplay with on-screen keyboards that take forever to type into?

      Maybe they want something that's faster than typing on a smartphone?

      There are plenty of reasons for this sort of thing, just like there are for any other sort of translation service.

    2. Re:huh ? by Anonymous Coward · · Score: 0

      You could do it that way, but...

      - This plus speech to text plus google glasses. Communications between deaf/hearing people are now trivial. No need for typing.
      - Throw in a text reader to an earpiece. Communications between any combination of deaf/hearing/blind people are now trivial. No need for typing.
      - Link it all up to google translate in real-time. Communications between any combination of deaf/hearing/blind people speaking any language they like are now trivial. No need for typing.
      - Bonus points if you can put the subtitles right next to the person speaking/gesturing - hard, but if you can correlate the mouth/hand movements with appropriately blind-source-separated voice then potentially doable (you could also watch where the glasses wearer is looking and just translate that voice). Congrats, now anyone can wander into a party full of people speaking any old language and trivially converse with whoever they want, even if they or the other person lack hearing or sight. And... no need for typing.

      I'd give it maybe 5 years before we have the complete system. The first point or three we could probably do right now, give or take some accuracy (though I suspect this is an application where near enough is very close to good enough - after all, a non-signing person trying to communicate with a profoundly deaf person right now is pretty much a non-starter). The last bit is hard, but not fundamentally impossible - like I said, maybe 5 years.

    3. Re:huh ? by wonkey_monkey · · Score: 1

      Why can't people who speak Dutch just speak English?

      --
      systemd is Roko's Basilisk.
    4. Re:huh ? by tgeller · · Score: 1

      The same reason you don't type everything -- signing is (much) faster, easier, requires less education/skill, much more accessible to people with disabilities...

      Having said that, it's interesting to see how *everybody*, including the hearing/speaking, are typing more and more to communicate -- even in person. As hearing society changes, so does deaf society.

      --
      Tom Geller
  2. Sign language is for cows. by Anonymous Coward · · Score: 0

    You are all cows. Cows say moo. MOOOO! MOOOOO! Moo cows MOOOO! Moo say the cows. Learn how to say moo in sign language.

  3. It uses the integrated face system by Anonymous Coward · · Score: 0

    The wearable tech uses the integrated face system to translate text.

  4. Cool! by Anonymous Coward · · Score: 0

    Cool!

  5. Good gorilla! by SoundGuyNoise · · Score: 1

    Put it on a gorilla and then we'll see if it's any good.

    --
    You never expect irony, do you?
    Want to be a professional wrestler? Visit www.iyfwrestling.com
    @iyfwrestling
    1. Re:Good gorilla! by Anonymous Coward · · Score: 0

      Ugly gorillas. Go away.

      [Cue laser burning through gorillas.]

    2. Re:Good gorilla! by Anonymous Coward · · Score: 0

      The last message translated reads:

      Don't run! We are your friends!

  6. Option 2: Pen + Paper by Anonymous Coward · · Score: 0

    There, I just saved you thousands of dollars!

  7. Linguistically unsound by mikaere · · Score: 1

    My wife is a New Zealand Sign Language interpreter (it's our third official language here). The problem with sign language is that it doesn't just use hand gestures, but also requires facial expression to convey meaning. So while they might be able to interpret gestures of the hands, they will miss a significant proportion of the meaning.

    The end result will be something that might be able to translate finger spelling, but not much else. Also, sign language uses relative location to the speaker in order to convey meaning such as tense (in front of you is the future, behind you is the past), so they had better put on tech to track position relative to the speaker.

    --
    It's good luck to be superstitious
    1. Re:Linguistically unsound by ToThoseOfUs · · Score: 1

      I've learnt a little Auslan (Australian sign language) and it's not only the above points, but also context. Completely different english words may use the same sign depending on context.

    2. Re:Linguistically unsound by mcswell · · Score: 1

      American Sign Language (ASL) has a syntax and morphology as different from English as that of Chinese or Arabic. There are some examples here: http://files.start-american-si.... (I'm sure that's true of other sign languages as well.) It may be that these researchers do some kind of grammatical analysis (the links don't say), but it's highly unlikely that it uses the same statistically based MT approach that Google MT and other modern MT systems use, for a simple reason: statistical MT works off of bilingual parallel corpora (texts written in the two languages), but there's next to no written corpus of ASL. So they're probably translating word-for-word.

  8. Damn by Anonymous Coward · · Score: 0

    I was just playing with this idea with a Project Soli. Glad it's getting done at least.

  9. Wonderful, but... by Anonymous Coward · · Score: 0

    Why wouldn't they just type it into the smartphone?

  10. Would be cool if.. by Anonymous Coward · · Score: 0

    You could then send it to a nearby wireless device.

  11. Deaf, not blind by Anonymous Coward · · Score: 0

    I don't understand the point of this device. If all it does is translate sign language to text, then why don't the deaf people just type out the text themselves on their cellphone or something and show it to whoever they're trying to talk to? They're not blind so they can type just fine. This seems needlessly complicated and very prone to error.

    1. Re:Deaf, not blind by Anonymous Coward · · Score: 0

      Like I can understand emojis of hands in different positions.
      ðY'âoeOEïðY'âoeðY'ðY'ZâoeSðY'OEðY'SðY'ðY'ðY'ðY'ðY'ðY(TM)ðY(TM)OEâïðY'ðY'ðY'...ðY(TM)

  12. They missed something crucial by MasseKid · · Score: 1

    There's something they've missed here that is crucial, and it's called user acceptance. The user of the device (the deaf person in this case) must want to use the technology. You would think that this would be wildly popular with the users from the outside looking in. However, there is a very real deaf culture: https://en.wikipedia.org/wiki/... . Many deaf people do not believe there is anything wrong with being deaf and actively reject those who seek treatment to cure their deafness. I suspect this will be viewed as an attack on their culture.

    1. Re:They missed something crucial by Anonymous Coward · · Score: 0

      > Many deaf people do not believe there is anything wrong with being deaf

      One must wonder how many deaf people were there in the Stone Age? Probably enough to feed saber-tooth tigers and giant snakes... I can't hear you la-la-la, ouch... Nowadays they are just voluntary human sacrifice on the altar of wheeled transportation.

      As a hungarian I can't stop being angry over the stupidity of those deafs who do not accept cochlear implants, even though they are suitable and doctors are chasing them left and right with the good news. That tech is dear to me, as it grew out of the theoretical work of two Nobel awardees, the ethnic hungarian dr. Robert Barany and the hungarian prof. Gyorgy Bekesy.

      Nowadays the up-to-date 32 electrode cochlear implants even afford spendid music listening experience and participation in open air, multi-speaker discussions is trivial. In richer countries toddlers are now being operated on both ears, beause the cochlear tech is already so good that waiting for even better gear cannot makes any sense, compared to the large, immediate benefits of stereophony.

      Sure, cochlear kits are expensive, the price of a fancy new compact car, but it is magnitudes cheaper for the guv'mint than paying disability benefits for life to those deafs who cannot work due to their hands being occupied with handwaving. Cochlear wearers are useful for the society and their families and also themselves, they are full, not handicapped. Not since Jesus's time and the works of the Apostles have we seen such wonderful healings! Thus, deaf parents wanting to produce deaf kids and keep them staying deafs should be caned in public and their kids taken away by the state and given cochlear implants.

  13. YOU COWS!! by Anonymous Coward · · Score: 0

    YOu COWS!!"#"€%&//

    cows!##

    YOU COW!!#%&€&%

    YOU ARE ALL COWSSSSS!€%&/

    COWS UNTO THE ABYSS!€#%€&%/&/%

    THe cows are calling me now. I must go unto them.

    The cows.

    THe cows.

    The............... coooooooooooowwwwws.

  14. Why bother? by Anonymous Coward · · Score: 0

    Deaf culture is insular. I have a high risk of going deaf in my mid-50s (runs in the family- almost all elderly on my maternal line are stone deaf even with no occupational risks / noise exposure). I won't be learning ASL. Nor will my child if she loses hearing due to a viral infection or whatever. Hearing aids, cochlear implants, 8x11 whiteboard, text-to-speech technology or whatever. ASL is an anachronism.

  15. Communicating in one direction? by hackwrench · · Score: 1

    This may help deaf people send a message. but it's one way and it seems to me that the options already available may be slower but both sides can use it. Both sides can learn how to type.

  16. Niche market by Anonymous Coward · · Score: 0

    Too complex to use in everyday life. However it could be use as some form of steno.

  17. FU IOT by Anonymous Coward · · Score: 0

    Does it translate the middle finger correctly?

  18. Texas A&M ? by Anonymous Coward · · Score: 0

    Texas A&M P uke

  19. Intimate Gestures by Anonymous Coward · · Score: 0

    When you masturbate does it automatically translate to "Oh Yes" "Oh Yeah" "Oh my god" "Yes".

  20. Dominatrix by Anonymous Coward · · Score: 0

    Finally I'll be able to understand my deaf dominatrix who keeps hitting me for no reason - I can never understand her.

  21. Down with the deaf-lobby, Nobel for cochlear tech! by Anonymous Coward · · Score: 0

    1. There is no sing language. In almost each and every country of the world, the deaf use totally different sets of hand singnals, mutually unintellegible. That is so stupid, as even on the high seas everybody understands the same handwaving (flag signals that is).

    2. Deafs shall stop being a luddite and get cochlear implants, so they could learn to hear, speak, converse and now even enjoy music. In normal countries social security covers it 100% for at least one ear. (No USA is not a normal country...)

    3. Deaf people are really dumb, superstitious and believe in silly conspiracies. They are actually trying to ban cochlear implants and consider themselves kind of a 21st century redskins, whose "unique culture" is being destroyed by the white hearers and their evil cochlear concoction. They are refusing to accept that deafness is a disability, which makes them guilty of blasphemy, because YHWH created mankind specifically with two ears to hear.

    Regrettably there are a lot of born-deaf american people in secluded jewish and radical protestant cults, due to gene deterioration that occurs from massive in-breeding. These communities hold disproportionally large financial and media-political influence in the USA and they fight against cochlear technology worldwide. By way of lobbying and discrimination complaint propaganda, they have prevented cochlear implant inventors from receiving the medical Nobel Prize on at least 3 occasions, despite overwhelming nominations. (Sweden is a very "politically correct" country they are subservient to delinquent minorities, be them somalians or deafs.)

    4. Some countries are now considering to legally classify cochlear implants as "anti-muteness vaccination", so they can compel deaf babies receive the artificial hearing early in life and learn to hear-speak normally, no matter what the deaf parents say. (Deafs are very selfish people, they want their kids to stay deaf and handwave all their life. Some even contract genetical research companies to sort out their semen and eggs, so they can get their very own 100% guaranteed deaf babies via in-vitro fertilization. No kidding.)

    5. Sometimes I wish cochlear implants ceased to exist and artificial eyes were granted to mankind instead. There is not a blind person who claims to be un-handicapped. They are not antagonistic towards the seers and don't want their kids to be born blind for forever generations. They are humble and kind people, who are generally liked by all and it is such a misery that science cannot yet help them. They are the ones who deserve all the benefits science has heapen on the ungrateful deafs.

  22. More details, please? by Anonymous Coward · · Score: 0

    We had a group of students working on a very similar product just a few months ago. They based their system ( http://sign2speech.com/ ) on the Myo armband. Did anybody find more details/paper/webpage about this very project?