Cell Phones for the Deaf

nitzan writes "Quoting from the article: 'the software translates the voice on the other side of the line into a three dimensional animated face on the computer, whose lips move in real time synch with the voice allowing the receiver to lip read.' Unfortunately this only works with laptops, but a pda version is in the works." The company website has a demonstration.

Comments

Good idea and good start but....

[BWJones]

This is a fantastic idea which will enable communication for the vast numbers of hearing impaired, however if the web-site is any indication, the technology needs improvement. I'm pretty good at reading lips and I was working pretty hard to figure out what was being said with the sound off.

This doesn't really do much...

[Cyclopedian]

Lip reading is only half the whole "info-stream" that comes out of peoples mouths. I know this. I'm deaf (severe to profound sensori-neural hearing loss, since birth) and I'll tell you one thing: lip-reading can give ambiguous results.

Someone can say "Pot" and yet with the same lip movement, can also say "My". Men with bushy mustaches are a lip-reading disaster.

For me, I've adapted in my own way: I rely heavily on my hearing aids. That combination of both lip-reading and hearing the audio stream from your mouth enables me to achieve at least a 70% success rate (under ideal conditions, if it's a party atomosphere, fudgeddaboutit). I've had hearing aids since I was 1 1/2, and only with extensive speech therapy can I speak well. I'm one of the few deaf-from-birth people that can do it this well. So, from that perspective, I can speak on a phone (as long as I can understand that mangled audio coming out the receiver, which is 0%).

Why don't they just focus on speech recognition? A great speech recognition phone would enable deaf people that speak to use phones for near real-time conversations. In addition, such technology can also be (easily?) adapted to foreign language translators for tourists.

However, until such technology is available at the consumer level, I'm stuck with two-way text messaging devices like the T-Mobile SideKick.

-Cyc

The reasons this is better than speech-to-text

[zipwow]

Partly, because speech to text isn't very good.

Speech to text isn't very good because its very hard to turn phonetics into words. Our ability to understand people is very reliant on context. Knowing what's been said helps you understand what's being said.

Some will say that speech to text is getting fairly good in English, which is somewhat true. Obviously, though, there are bigger markets in other languages.

So how does this thing work, if it doesn't do speech to text? It does speech to phonetics, and phonetics to lips.

For example, its relatively easy to understand when someone has said "h -ee- r", but knowing if that's supposed to be "here" or "hear" is quite difficult.

This is why the same software works across languages. "Th" is "Th" in any language, and your single algorithm doesn't have to care.

-Zipwow