Slashdot Mirror
Researchers Convert Mouth Movements Into Speech
andylim writes "According to Cellular News, researchers at Germany's Karlsruhe Institute of Technology have developed a method for mobile phones to convert silent mouth movements into speech. As recombu.com points out, the 'potential for secret conversations just got huge.' You could pass the time by making phone calls from the cinema without disturbing anyone. In noisy places like bars and clubs you could make yourself heard without having to shout."
According to my ASL instructor, lip readers are rarely more than 50% accurate. Which makes me wonder about the alleged capabilities of this software, honestly.
Any serious geek has one of these.
"A government is a body of people usually -- notably -- ungoverned." -Shepherd Book
I have some hearing loss, and went to a seminar at the VA once about adapting. I don't know how good lip readers get, but for me, at least, it's mostly useful if I have an idea what's being said and just need to fill in bits that I didn't quite catch. I suspect that this will need at least some training with the user, just like voice recognition software does, and that it's going to be a long time before it's good with anything but a very limited vocabulary.
Good, inexpensive web hosting
You might want to look at this.
I don't think this technology is THAT new... or that it's that inaccurate.
On a side note, I'm hearing impaired (car engine exploded a bit too close to my head). I *CAN* hear -- and that supplements the lip reading I *DO* do... and asking my friend who is totally deaf (and on AIM as I type this), I think that 50% estimate is way low...
Using a mobile phone in a cinema is one of the least considerate things anyone can do, they create light pollution
One could make such a phone with a 'dark mode' and equip it with IR illuminators and camera.
Have gnu, will travel.
Try it with older people from the bush. They speak without opening their mouth to keep the flies out. Some move the lips but keep the teeth together.
Fifteen (!) years ago, I took a UC Extension class on Neural Networks taught by Stanford professor David Stork. He had developed a lip-reading system for communication in noisy environments, such as an airplane-repair facility. If you could do it 15 years ago with workstation-class desktops, I suppose you could do it with a smartphone today.
The best they can hope to reconstruct from mouth movements alone is the formant frequencies (think of them as high and low band pass filters that shape the characteristics of the pure tones generated by the vocal folds). This means that any information encoded in pitch will be totally lost.
In a relatively non-tonal language like English, you still might be able to make sense of the speech. It will just sound like a Vocoder or Peter Frampton's talk box.
Good luck understanding anything in Chinese, though.
"In noisy places like bars and clubs you could make yourself heard without having to shout."
Or more likely, used by men in conjunction with Babel Fish to chat-up women who don't speak English.
If the pattern goes 9am, 10am, 11am, why isn't noon 12am?
I had to lip-read for 13 years before I got my cochlear implant. If we were talking about such things as vacuums, I'd probably understand what you were saying. If it were just out of the blue, though, yeah, I might misunderstand and punch you for it. In fact, it was considered great fun to say things to me and have me repeat them as I understood them. "Pear pimples for hairy fishnuts?" Stuff like that.
Truth, Justice. Or the American Way.