Slashdot Mirror
Microsoft Research Uses Kinect To Translate Between Spoken and Sign Languages
An anonymous reader writes in with a neat project Microsoft is working on to translate sign language with a Kinect. "Microsoft Research is now using the Kinect to bridge the gap between folks who don't speak the same language, whether they can hear or not. The Kinect Sign Language Translator is a research prototype that can translate sign language into spoken language and vice versa. The best part? It does it all in real time."
Kinect translation with new autocorrect-for-ASL: "I really like your tits"
Thanks Microsoft!
Sent from my ENIAC
We were doing real-time ASL translation to text using the webcam on the Indy2 workstation back in 1997, success rate was about 85% and most of the misses were from hidden object problems which the Kinect does nothing to help with.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
You clearly haven't seen two or more deaf people in public.
Plenty of people sign. Heck, plenty of hearing people sign.
Conversations (deaf-deaf, hearing-hearing, hearing-deaf) are all much more natural face-to-face where you can interrupt, show expressions, assign nuanced gestures or tones to words -- all things that are not well suited for text.
never underestimate what people can do
http://appleinsider.com/articles/11/09/15/stevie_wonder_thanks_steve_jobs_for_making_ios_devices_fully_accessible
I'd love to see a blind person use sign language.
"" How about taking the safety labels off everything, and let the stupidity-problem solve itself? """
They do. They use iPhones (and iPads). iOS has an accessibility feature called VoiceOver which changes the input paradigm from a touch initiating a 'click' action to one where a touch reads out the description of the UI element with text-to-speech. Two taps on the item will send a tap to the UI, and a three-finger swipe will initiate scrolling. So you can basically drag your finger across the screen and find things with your ears instead of your eyes, then navigate and interact as a sighted person would once you get your bearings.
It's a shame Obamacare doesn't address hearing aids and glasses, it would have likely gotten a lot more support.
That's funny. Not only is the deaf community a tiny minority (unlikely to have any impact at all), the issue of hearing aids is enough to divide them! If they found out you could get a cochlear implant with insurance purchased through an exchange, you'd see little other than opposition. (Yeah, they're that crazy.)
Not that they're likely to be aware of the issue, as illiteracy is so prevalent. Still, if they found out, they'd oppose it.
Required reading for internet skeptics
This is one of the stupidest things I've read all day. You think that pupils at a school for the deaf are sending each other text messages as they stand next to each other? Signing is still very popular among the deaf, and is even a part of the distinct culture that has arisen in deaf communities.
Your comment shows the sign of ignorance. Plenty of deaf AND hearing people sign anywhere. It's easily 10 times faster than texting. Even faster than speech itself.
Yeah, trippy that the Deaf want to protect being Deaf so much that they don't want other deaf people to hear!
Spot on. Also, sign language is quite important culturally as the native language of deaf people. All cultures view their native language as essential to their cultural identity. The deaf are no different (or so I hear.)
I have two deaf cousins who would beg to differ, and all of us in the family who learned to sign can communicate much quicker by signing than grabbing a smart phone. And my hands do not need electricity. Fucking douchebag.
The Revolution Will Not Be Televised
I'd love to see a blind person try to use touch screen phone.
Touch screen phones may well be the best tech to come along for helping blind people ever.
http://bits.blogs.nytimes.com/2013/09/29/disruptions-guided-by-touch-screens-blind-turn-to-smartphones-for-sight
Since that might be pay-walled, here's a copy:
Disruptions: Visually Impaired Turn to Smartphones to See Their World
September 29, 2013, 11:00 am
In recent years, many smartphone apps that are aimed at blind people have appeared.
Luis Perez loves taking photographs. He shoots mostly on an iPhone, snapping gorgeous pictures of sunsets, vintage cars, old buildings and cute puppies. But when he arrives at a photo shoot, people are often startled when he pulls out a long white cane.
In addition to being a professional photographer, Mr. Perez is almost blind.
"With the iPhone I am able to use the same technology as everyone else, and having a product that doesn't have a stigma that other technologies do has been really important to me," said Mr. Perez, who is also an advocate for blind people and speaks regularly at conferences about the benefits of technology for people who cannot see. "Now, even if you're blind, you can still take a photo."
Smartphones and tablets, with their flat glass touch screens and nary a texture anywhere, may not seem like the best technological innovation for people who cannot see. But advocates for the blind say the devices could be the biggest assistive aid to come along since Braille was invented in the 1820s.
Counterintuitive? You bet. People with vision problems can use a smartphone's voice commands to read or write. They can determine denominations of money using a camera app, figure out where they are using GPS and compass applications, and, like Mr. Perez, take photos.
Google's latest releases of its Android operating systems have increased its assistive technologies, specifically with updates to TalkBack, a Google-made application that adds spoken, audible and vibration feedback to a smartphone. Windows phones also offer some voice commands, but they are fewer than either Google's or Apple's.
Among Apple's features are ones that help people with vision problems take pictures. In assistive mode, for example, the phone can say how many heads are in a picture and where they are in the frame, so someone who is blind knows if the family photo she is about to take includes everyone.
All this has come as a delightful shock to most people with vision problems.
"We were sort of conditioned to believe that you can't use a touch screen because you can't see it," said Dorrie Rush, the marketing director of accessible technology at Lighthouse International, a nonprofit vision education and rehabilitation center. "The belief was the tools for the visually impaired must have a tactile screen, which, it turns out, is completely untrue."
Ms. Rush, who has a retinal disorder, said that before the smartphone, people who were visually impaired could use a flip-phone to make calls, but they could not read on the tiny two-inch screens. While the first version of the iPhone allowed people who were losing their vision to enlarge text, it wasn't until 2009, when the company introduced accessibility features, that the device became a benefit to blind people.
While some companies might have altruistic goals in building products and services for people who have lost their sight, the number of people who need these products is growing.
About 10 million people in the United States are blind or partly blind, according to statistics from the American Foundation for the Blind. And some estimates predict that over the next 30 years, as the vast baby boomer generation ages, the number of adults with vision impairments could double.
Apple's assistive technologies also include VoiceOver, which t
When information is power, privacy is freedom.
it is quicker and more convenient for the disabled to send a text message.
I would have thought it would be more convenient for someone to speak/be spoken to in their native language - sign - rather than send/receive a text in a foreign language - English or whatever.
Esli epei etot cumprenan, shris soa Sfaha.
No, and neither has she.
Disclaimer: yes I work for Microsoft. No not on these projects.
This was demo'd live in front of 30K MSFT employees at our annual company meeting. It nearly brought me to tears. Yes, I can see through demoware and and yes it's highly imperfect, but honestly it was the single most impressive use of technology I've ever seen. It was both novel and simple. It combined hardware, algorithms, user experience, and cloud scale. I don't know if it will ever go anywhere though I expect that it will. The key point here is that these are off the shelf components. Kinect and gesture APIs combined with machine translation and text to speech. It's important that these are, all or nearly all public production APIs. Such a system 10 years ago even if possible, would never make it to market because of the tiny user base. Today we can build such apps for the 0.01% of the population that need Mandarin Sign Language translated to English. And it can be cost effectively. That is the point. Technology being used to address real problems for under served communities. So yes, maybe people researched automated sign language recognition years ago, but bringing it to market and enabling a scenario for real people is a wholly different beast