Voice Recognition and Programming?

dsaxena asks: "I'm suffering from repetitive-stress injuries to my wrists/foreamrs from typing too much. Luckilly it hasn't progressed into carpal tunnel syndrome yet, but it will if I'm not careful about it. I've been looking at alternative input methods and was wondering how many people have tried using speech recognition for programming? I've seen demos of speech recongition software (Dragon and ViaVoice), but it's awlays been in the context of writing letters or reports. Would it work as an efficient input method when trying to code in C?"

"It's all in the grammar."

Voice recognition still has a ways to mature before it is robust enough for coding since a lot of programming involves the use of symbols, not words. Sure, you can assign "names" to these symbols and start speaking away, but it would be cleaner if voice recognition engines are made aware of the syntax of specific languages and would do this for you (many modern editors have this feature). Is voice recognition moving in this direction, or do coders have a while to wait before they can start using their keyboards as paperweights?

Comments

Not appropriate for vocal input

[Tet]

With a bit of work, voice recognition systems may be taught to recognize the syntax for some programming languages. However, all this means is that you can dictate a program to the machine. You can't actually program it because programming is not linear in the same way as writing prose. You'd need to be able to tell the editor to go up 3 lines, or to copy the next 7 lines into the do_more_stuff function. Voice recognition just isn't an efficient interface for doing that sort of stuff. The keyboard will be with us for quite some time to come (even if it may evolve into more efficient forms, such as a stenograph machines -- unlikely, but possible).

how about handwriting?

[josepha48]

It would be possible to write your program with a stylus and touchpad. It would need to be able to recgonize your hadnwriting too. I am wondering why systems like these are not making there way to the pc. I mean if you can do it on a palm, you can do it on a pc as well. This would make more sense if you ask me. It would not have to deal with all the symbols differently it would be another character. You can usually write pretty fast and once you get used to it this may work. You may even be able to get rid of the keyboard as well.
~~~~~~~~~~~~~~~~~~~~
I don't want a lot, I just want it all ;-)
Flame away, I have a hose!

Function names?

[DeHackEd]

I think the biggest obstacle to overcome would be the funny names programmers assign for their variables and functions. While I use dragon naturallyspeaking (which come with Corel WP Suite 8 Win9x) and it does support saying things like "OPEN BRACKET", a function name like vsprintf would be difficult to choose how to put that into normal words. And then you have to type in the name of the word into the add to dictonary feature and then say the word three times to program it. Seems like a lot for work for the whole LIBC.

Also, I feel that programming languages were really meant to be written by keyboard. Saying int i equals sizeof open bracket int close bracket times four semicolon just seems like too much work compared to entering it on a keyboard. Even then i will most likely be misinterpreted as eye.

I think its possible to write code in such a way, but perhaps too impractical. It would either involve creating a whole new, large dictonary for your programming needs or maybe your very own #include file to convert things like equals to =.

Sore throats, and how do you pronounce '}'

[human bean]

When I am programming, I often crank ten screens worth per day. By the time I finish editing these, I have probably typed ten times that. I would hate to speak all of this, especially if working in the classic "open cube farm".

This could be worse than those annoying jackals from the marketing department, who when cubed near us insist on monitoring their voicemail by speakerphone, and always at top volume. Grrhh! Or the admin from hell who insists that everyone benefits from decibelic exposure to retro radio. "All disco, all the time..." Smash! Whump! Mrphf! (sound of small boom-box radio receiver being forced up human orifice)

Maybe you could get a better keyboard, and some macros for it?

Hybrid speech/stylus interfaces

[hatless]

Man, the signal-to-noise ratio here just keeps getting worse. Dvorak keyboards and "good" keyboards ain't gonna stop RSI. Dictation software alone isn't well suited to coding, what with the massive amounts of critical punctuation and formatting. And handwriting and gesture recognition is just plain slow compared to typing or speech.

What I'm surprised by is how little imagination the handwriting recognition and speech-recognition developers have. Dictation software is usually used by people who *do* have the use of at least one hand. So why not have hybrid interfaces?

The tedious process of editing material in a dictation environment could be simplified and sped up greatly if you could simultaneously use a stylus to jump the cursor around and scriblle out or mark errors--or make corrections. You could also be presented with floating popups showing choices for ambiguous words and phrases the dictation engine is having trouble with even as you continue to dictate.

I think a hybrid interface like this--using handwriting, gestures and speech together would allow for quicker and more precise and efficient dictation, and would make dictating things like source code or scientific data and equations at least as quick and fluid as keyboarding.

It really is about time the keyboard and that other vile RSI torture device, the mouse, started to go away for most applications. Shaking developers out of the misguided focus on 100% hands-free dictation and all-or-nothing handwriting recognition would help immensely.

Just one question

[jaa]

since I can't really answer yours:

are you getting this (early ct syndrome) professionally treated? In the words of jwz:

"Do not fuck around. If you are experiencing any kind of pain, get to a doctor and get it diagnosed."

Looking for alternative solutions is great, but don't do it to the exclusion of professional help.

Best of luck.

alternative Keyboard layouts

[Jordan Block]

You cold awways try out a different keyboard layout. The Dvorak layout is supposed to minimise repeditive strain injuries.

I had a mild bout with the same thing not too long ago. I went through some Psyiotherapy and stuff to deflame (is that a word??) the nerves in my elbows that were causing my hands to go numb.

Now I'm just more careful about how I rest my elbows on my desk, and I'm doing much better.

Been there, done that, gave up

[mouseman]

When I was finishing up grad school, I had to deal with RSI. I tried physical therapy, funky keyboards, dvorak, and, yes, dictation software. I wrote much of my dissertation using dragon dictate, and did some programming with it as well. I had voice macros for common symbols, keywords and emacs commands.

Did it work? Yeah, sort of. For entering normal text, dictation software has gotten to the point that it is definitely usable -- by someone who absolutely can't get by without it (95% accuracy seems really good until you realize it's getting one in 20 words wrong. and your spellchecker won't be any help). But editing programs is harder -- you spend a lot of time making changes to existing text, which is hard to do by voice. After leaving grad school, I gave NaturallySpeaking(TM) a try. I found it better in some ways, worse in others. I think it is far worse for programming than dragon dictate. at least the version i used was (maybe it's been improved), because you could only dictate into a dedicated window, and there was no support for voice macros (IIRC).

So where am I now? Pain-free most of the time. I have flare-ups from time to time and fully accept that this is a lifetime condition. I still use dvorak; it *feels* better, but that's about all I can say. Since leaving grad school, I spend less time in front of a keyboard, by virtue of spending more time in meetings. :-( That, plus (somewhat) better habits has made a world of difference. I think of RSI as a physical symptom of a psychological condition: compulsiveness. A normal, healthy person does not keep doing something that causes pain. I think the key to dealing with RSI (note: not recovering from -- it's too late for that) is to listen to your body and stop doing things that cause pain. Dictation software might work as a stopgap, but if you don't make a fundamental change in your habits, you'll just end up with a strained voice in addition to your strained wrists.

good luck!