Slashdot Mirror
Voice Recognition for a Techie?
kaybee asks: "I am a long-time developer, sysadmin, and general computer junkie (for fun and for work) who needs to seriously curb the usage of his hands. I'm curious as to the current voice recognition options, preferably usable on Linux and Windows. I prefer the command-line to a GUI, I prefer Vim to anything else, and I still read my email with Pine. I'd like to hear options for sending email via voice, which I hope is easy, and I'd love to hear of any solutions that allow effective coding via voice, which seems much more difficult."
Oh, I'm sorry. I thought you said voice recognition for a Trekkie....
Tibbon
tibbon.com
Write it yourself. Grab the Microsoft Speech SDK and WINE or some suitable interoperatibility layer and you should be good for Windows and Linux. The Microsoft Speech SDK doesn't require oodles of code to make it work, so you should be able to get a working sample under Windows in about half an hour. It comes with some rudimentry samples as well, and since it's not released under any particularly binding license you can just build your code around it.
'Course you could go the other way with some Open Source speech recognition and cygwin or similiar.
I'll subscribe to Slashdot when I see a month without a dupe, a typo, or an article the "editors" didn't read.
No, computer, I said, "awk single quote left curly print dollar one right curly single quote file dot txt pipe sort pipe uniq dash see greater than a dot out"
shudder
[...]who needs to seriously curb the usage of his hands.
Lest they... *ahem* wander.
I've actually used some vioce recognition over the years and it's got a lot better than it used to be. last time I used to use a voice recognition software on my computer even though I did loads of the training it just didn't seem to get it; eventually I had to give up... it wasn't cheap either. Whilst I think it will have potential to do a lot in the future I'm just not sure that it's really at the stage where it can be considered a full time replacement; especially for technical jobs
*''I can't believe it's not a hyperlink.''
The main issue I see with coding by voice is that each character needs to be said by a word. We only have 26 single sounds we can make (at least us english speakers) and so pretty much everything besides the basic sounds have to be the result of multiple letters strung together. Here's some math:
Lets say you type at about 40wpm, or about 160characters per minute (this is a low estimate of 4 chars per word), or about 2.5 characters per second.
To be as productive speaking, you'd probabily have to speak about the same number of words per second as you type characters, or 2.5 words. That's really fast.
Sorry bub, doesn't look speech is a very good alternative. Hell, Brain Implants on the other hand...
Voice recognition is good for letter-writing but bad for overall computer usage, especially in UNIX shell (incl vi and especially Emacs). Picking programs that don't require jumping all over the keyboard for basic tasks can reduce the strain. Same goes for programming syntax: Python is a lot more RSI-friendly than Perl, for example. (IMHO) Write scripts that automate routine tasks, even if it's just one line with lots of regex.
Eye won stride to yews voice wreck iginition soft wear tomb ache a slash dot post. Eye was knot imp pressed. It was sofa king we todd did
Seriously, if you're suffering hand or arm pain, you should think about the way you're doing things now. Speech recognition is unlikely to replace your current coding practices, although it might help with writing reports.
Instead, try using the keyboard break feature in gnome. To start with, have it kick you off your computer every 30 mins for a 3 min break, and don't allow yourself to postpone breaks. Get some equivalent software for windows too. Use your 3 min breaks to walk around and stretch. Within a week, you won't be a lot less productive, but your arms will feel a lot better. Then you can maybe up it to 40 mins. In the short term, a course of anti-inflams might help (ask your doctor).
Also, don't come home in the evening and play games on your computer, or do more work. Your arms probably can't take it. Equivalently, inform your employer of your condition and subsequent inability to work reckless overtime hours.
These two things should get you started for long-term sustainable maintenance of your arms.
It's been a while since I've had to look into speech recognition for linux, but this link should help you get started: Linux Accessibility Resource Site
Read down to the section about speech recognition. I hope that helps.
My inner self is ineffable, so don't eff with me.
For gaming on WinXP, I use an app called Shoot!. While playing Falcon, I use it for fairly simple (press T, wait 5 seconds, press 1) macros. I was dicking around and decided to set up a profile for some simple stuff in Cygwin. If I say "list", the program returns "ls". "List all" will return "ls -a". "List all long" will return "ls -la".
You can, with some tweaking, even get it to understand complicated stuff. If I say "manual g r u b", I can get "man grub". "Vi save quit" could be mapped to ":wq" without too much trouble.
Anything you can type, it can do.
I don't think it works under Linux. I don't know of anything like it under linux. It does, however, work quite well inside PuTTY.
I'd rather you do it wrong, than for me to have to do it at all.
"I'd like to hear options for sending email via voice, which I hope is easy, and I'd love to hear of any solutions that allow effective coding via voice, which seems much more difficult."
I've wondered about this myself. I tend to use my computer with the headphones on. Often, I'm listening to music or.. well just plain silence, just the standard dings of Windows. I do pay attention, though, to the sounds coming from the computer. (i.e. the traditional hoo-hoo of recieving an email.) I've always wondered about what more could be done with sound to make the user more aware of the goings on with their computer, especially when a number of apps are actively working. I think I was inspired by an episode of Futurama I caught. One of the character's personalities was in the Pilot's body. The Pilot, whose personality was in yet another body was trying to describe how to interact with the ship. I remember him saying "Can you hear that faint little tone? That's the status of..".. or something or other.
In any event, it's fun to imagine. I wouldn't mind if a soft low-volume voice were to say "You have recieved an email from: John Smith." I had a job a few years ago where that would have been a nice little feature since messages would come in that required urgent attention. My solution to the problem at the time was to use a custom filter that would specficially notify me of important messages by bringing a little window up to the surface. That was fairly annoying, though, when the computer was busy and it was slow as molasses to get the window to go away.
"I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)
I ran into this problem while working (coding) & trying to do grad school (in Comp Sci). The first point I'd make is, take a rest break (no computer use) for a while if you can. ASR isn't really there yet, & it won't help you with other things you might want to be pain-free for... seriously. That said, there is a group called "Open Source Speech Recognition Initiative" whose mailing list I'm on, but they don't have any product yet. Might get a better answer posting there, though. Or not. There's also a group on Yahoo (I think) called VoiceCoder. That's your best bet right now, although it's all about Dragon Naturally Speaking & various hacks & kludges to be able to do coding, use Dragon for Linux, etc. Dragon has been reported to run under WINE, but of course YMMV depending on your hardware, versions, etc., etc. Finally, whatever approach you try, expect it to take a good long while before you begin to approach your hand-using productivity. The technology isn't there yet, and even though I know how to improve it, I have no Ph.D. so no one would give me the $$ to do the research that could back up my claim.
nificant.
http://perlbox.sourceforge.net/
http://cmusphinx.sourceforge.net/
Command and control is a lot easier to do with voice recognition since the dictionary the engine has to choose from is so much smaller. Having voice recognition engines understand arbitrary words well is still a bit difficult.
I was looking to make a headless system that I could bark commands at, and I was quite sucessful at developing my own actually. I used IBM's ViaVoice SDK and modified a few of the sample programs they had that were written in C. It took a little work getting the system running, it being a tad old and all, but eventually got it down to where it was completely useable and could make requests like "new mail" and "talk to me dirty". Oh and yes, it was a Linux system it was running on (Slackware 8).
Google it.
First, find a solution that makes it easy to enter text into a GUI (gnome accessibility, WINE w/dragon natural speaking, whatever).
Find a subset of words that are short, easy to remember, easy to say, and above all -- accurately translated by the chosen voice recognition software.
Then create a small perl script that can take this coded input and convert it into a nicely formatted chunk of code.
You can have different translators for different target languages... for example
In shell programming, you might have the following:
hash -> #
bang -> !
pipe -> |
test -> [
end test -> ]
mark -> '
quote -> "
end mark/quote (keeps them balanced for shell scripts)
for identifiers... don't name them. For example, lets' say you wanted to do this:
#!/bin/bash
function hello_lcase {
HELLO = $1
if [ -z $HELLO ] ; then
echo "Hello world"
else
echo -n "Hello from "
echo $HELLO | sed -e 's/.*/\L\0/'
fi
}
you would say:
hash bang slash bin slash bash
new function 1
set local 1 ref in 1
if test empty ref local 1 end test
then
echo string 1
else
echo option n string 2
echo ref local 1 pipe program s e d option e space
mark s slash dot star slash back upper l back 0 slash end mark
end if
end function 1
you'd run the perl script and it'd ask you:
what do you want to call function 1: foo
what do you want to call local variable 1 in function 1: HELLO
what do you want to use for string resource 1: Hello World
what do you want to use for string resource 2: Hello from
and it'd output the script (maybe after running through indent)
You could substitute "1" for any easily recalled mnemonic or symbol the text->speech translator is unlikely to mistranslate (in this case "foo" and "hello" would probably be fine as is)
Then you'd get a chance to globally "refactor" your symbols and give them nice-looking names, only having to type them once.
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
I just did a presentation on speech recognition software for the Office of Disabilities Services at my school, and since I see that you have a lot of response on the technical aspect of it, I'd like to bring up something else: how speaking to the computer affects *you*. One of the things that most surprised me about using speech recognition is how speaking comes from a different part of the brain than typing. Composition through speech is *very* difficult to start; don't think you're going to just dive in and compose an essay or report right off the top: even if the computer can understand you, you won't be able to coherently phrase your thoughts in a truly professional manner. Speech recognition is at its best when used for email and (ironically, at least I thought) instant messaging, because these forms of communication most accurately mimic speech. I don't know how it's going to affect coding, though, 'cause I wasn't brave enough to try that (but I can only imagine it would be difficult). I just wanted to offer a slightly differerent perspective on it. It certainly seemed like I was using different neural pathways or *something*, so just remember: as much as you're going to be training the speech recognition profile, you're going to be training yourself, as well!
Don't get too discouraged by the large number of commenters who haven't used speech recognition or who don't understand why someone might need to lay off the keyboard for a while. I wrote 100k lines of C++ code hands-free for my astronomy thesis over the course of two years, using with speech recognition software that is now about 10 years out-of-date. There have been significant improvements in both the speech recognition technology and tools for coding by voice since then. For coding, take a look at the VoiceCode project at http://voicecode.iit.nrc.ca/VoiceCode/public/ywiki .cgi
For other tools/approaches to coding by voice, see also the VoiceCoder group at yahoo groups:
http://groups.yahoo.com/group/VoiceCoder/
I don't know of any open-source or non-commercial dictation software which matches the accuracy and ease-of-use of the Dragon NaturallySpeaking (fair warning - I work for Nuance, which makes Natspeak, though I was a user long before I became an employee). Natspeak is only available for MS Windows, but you can always put a Windows box on your desk and connect to a unix host via an X server (exceed, xwin32). That generally works well for command-line stuff, not so great for GUIs (but you say you prefer command-lines anyway).
For something that runs on Linux directly, you might have a look at the Accessible Speech Recognition Technology software. It's a research project, not a polished system, but you might be able to hack it to do what you need.
xvoice is a gtk1 X application which uses IBM's ViaVoice engine to provide voice control and dictation support to arbitrary X applications. xvoice.sf.net is the url. The mailing list mainly covers issues of getting the ViaVoice libs working on modern distributions. The last release of VV was around the glibc2.0/2.1 era and most new ld.so's will struggle to execute the libraries and java dependancies. It's also fairly hard to buy a copy of VV 2nd hand anywhere and IBM appear to ignore any request to release it.
However once you get past all of these issues (actually even running the old gtk1 xvoice becomes hard on modern dists), it works a charm. As it's X clean, you can X to any X server, be it one run under OSX or Windows, or a Sun SPARC box. You just need the mic connected to the x86 Linux box the client runs on.
This meets your requirement for editing in vim etc. The accuracy, I found was fantastic.
It's possible to recognize speech pretty well (and no, the ridiculous examples of "I'll Ike's peach recognition all hot" don't really happen for any reasonable engine that uses language models, and most of them do these days).
... delete ... delete ... replace Yesterday with Monday" (while it's possible to recognize "replace X with Y", you still have to be pretty focused not to say anything else).
The main problem is that no one actually speaks or writes as eloquently as people present speech recognition.
Try this experiment: map backspace, delete and arrow keys to @ and try to write a letter or some code. You'll quickly give up. When you see demos of speech recognition, you never hear someone saying "Yesterday I went to the cinema. umm Monday actually. Ha, look the computer is still writing. Oh boy... delete delete delete delete
The missing bit is the intelligent dialogue that is redundant when you type. When you type, you have arrow keys, control keys and backspace. When you talk, these things are part of the communication, and writing an intelligent dialogue system is not trivial. If you want another experiment for the limits of speech recognition, just try whatever you want a computer to do with a real person. Try to dictate code to someone, and you'll soon find that it's not that simple. A person can even ask at the right time "empty brackets?" after you say a function name followed by a semi-colon, yet it's still very difficult to dictate code (or even a letter without any corrections).
There is another problem: Imagine that you type away, and suddenly you see that you've forgotten a semi-colon. But as you're writing a game, you have the constants UP, DOWN, LEFT and RIGHT. Hmmmmm.... Now you have to change your code (or the code you've downloaded) to suit the interface. Not good. Another option would be "missing semi-colon at the end of the line beginning with strcpy", but you need a very intelligent dialogue system for that.
Note: I've assumed that the recognition is perfect (and the problem is with our brains), but of course it isn't.
http://freshmeat.net/projects/speechlion