How One Programmer Is Coding Faster By Voice Than Keyboard

mikejuk writes "Is it possible that we have been wasting our time typing programs. Could voice recognition, with a little help from an invented spoken language, be the solution we didn't know we needed? About two years ago Tavis Rudd, developed a bad case of RSI caused by typing lots of code using Emacs. It was so severe that he couldn't code. As he puts it: 'Desperate, I tried voice recognition'. The Dragon Naturally Speaking system used by Rudd supported standard language quite well, but it wasn't adapted to program editing commands. The solution was to use a Python speech extension, DragonFly, to program custom commands. OK, so far so good, but ... the commands weren't quite what you might have expected. Instead of English words for commands he used short vocalizations — you have to hear it to believe it. Now programming sounds like a conversation with R2D2. The advantage is that it is faster and the recognition is easier — it also sounds very cool and very techie. it is claimed that the system is faster than typing. So much so that it is still in use after the RSI cleared up."

Comments

Probably not faster than auto complete

[ModernGeek]

The first thing that came to mind was how much useless and repetitive things that a programmer has to churn out to make things work, however I'd have a hard time believing that this could be faster than someone using something like autocomplete as done in .NET.

Re: Probably not faster than auto complete

[alen]

Yeah, but Microsoft is evil and everyone knows you are dumb as door nails if you cheat with autocomplete and code suggestions in a modern IDE rather than type everything yourself

Re:Probably not faster than auto complete

[complete loony]

I haven't RTFA, but I'd guess his biggest gains came from scripting things that were annoying to say. If you invested the same amount of effort in automating things that you would normally type, you can get very big productivity gains.

My father is a (retired) software developer. When he was a kid he lost both of his hands in an accident. So how does he type? By holding a pen between his arms. How does he code efficiently? Lots and lots of domain specific vi macros. For example, with less than 10 key presses he can insert a well formatted SQL Insert statement, with all columns from the requested table pre-filled.

Re:Probably not faster than auto complete

[exploder]

Imagine if your .o object files were more malleable than text, now realize that they are, but you haven't developed the tools to do so.

I'm on a horse.

Re:Probably not faster than auto complete

[complete loony]

He's been in the local paper a few times as a human interest story during his life, but there's nothing available online that I can find. Though he has told me a fair number of anecdotes from work over the years.

Once he turned up to a job interview after being referred by an old colleague. The interviewer hastily excused himself to call the guy; "You didn't tell me he didn't have any hands". "I didn't think it was relevant to his ability".

After discovering that a Y2K compliance project was starting (ie using 4 digit dates everywhere), with an estimate of 8 people for 2 months. He came back after about 2 hours with all the changes checked in.

Re: Probably not faster than auto complete

[Yvanhoe]

Or, you know, install it for the editor you use. Dude, this is 2013, there are autocomplete packages for vim, for god's sake.

Re: Probably not faster than auto complete

[Anonymous Coward]

Maybe you were too young, but they pretty much created the concept of autocomplete for programming with Intellisense.

You might have had a point about my youth and inexperience if MS had developed autocomplete for punchcards, 'cos that's what I learned on, but in reality it's been around longer than Windows has been.

I remember using autocomplete coding in some IDE on a Vax box in the mid '80s, and later using it on Suns and Amigas in the late '80s and early '90s (Pascal variant, maybe?). I know Delphi had good autocompletion.

So no, Microsoft didn't invent it, and no amount of revisionist history here will change that....

Re: Probably not faster than auto complete

[PybusJ]

Maybe I'm too old, but programmers editors have been supporting completion using ctags and its ilk since Microsoft were dabling in DOS. Intellisense didn't appear until the late 90s. Sure MS's focus on tools for developers helped to cement their OS monopoly at that time, but there was plenty of completion support, and a lot of research in context sensitive completion long before.

Re:Probably not faster than auto complete

[X0563511]

Why would you (routinely) want to directly edit your object files? What exactly are you trying to do that a non-experimental compiler can't do better?

Emacs

[captain_dope_pants]

Really ? He'd have been better off with VI - everyone knows it stands for Voice Input :p

Re:Emacs

[Immerman]

> while (true != false) process_more_stupid_code();

Ah, I see you've selected "a lawyer enters the room" as your stopping condition. Should I be suspicious of the code in that harmlessly named function?

Re:Emacs

[newcastlejon]

Should I be suspicious of the code in that harmlessly named function?

Not really. The whole scope ceases to be a problem when vanish(puff_of_logic) gets called at the end.

Re:Emacs

[doti]

VI VI VI

the editor of the beast

Re:Emacs

[hackula]

"Computer! Colon Q! Return!"

You can't win..

[Jah-Wren Ryel]

So how long until he gets laryngitis and has to start typing again?

Re:You can't win..

[Seumas]

If you watch the video, he discusses that. He does about 40-60% of his coding with this system and he does keep voice-strain in mind (in fact, he was sucking on a hard candy during the demonstration to keep his voice from drying out). You may not do 100% of your work in it, but just imagine if you could cut the amount of typing you do down to about half of normal? Suddenly, you're spreading some of the load to your voice, keeping either from being excessively stressed.

Re:You can't win..

[Tablizer]

It's best to spread out the load: 1/3 with your fingers, 1/3 with your voice, and 1/3 with your dick.

Re:You can't win..

[Jah-Wren Ryel]

> Switch to a brain controlled interface.

And then when your brain wears out you are totally screwed!

Coding != Typing

[JaredOfEuropa]

It's an impressive demonstration of voice recognition, and rather useful for people suffering from RSI, but to suggest that we may all benefit from this? Besides the fact that speech-to-text is a decidedly crappy input method in open plan offices (especially with the extra noises added on), it is also questionable if this will make us code faster. In my experience, typing speed is not really a major limiting factor in coding speed, when taking problem solving and debugging into account. When coding, I do not spend that much time ting, actually.

Re:Coding != Typing

[oodaloop]

I do not spend that much time ting, actually.

You don't say?

Re:Coding != Typing

[Krishnoid]

It's an impressive demonstration of voice recognition

Is it really? Since English is a non-phonetic language, I wonder if there's a parallel between coming up with consistently pronounced individual phonemes for special characters, and PalmOS Graffiti. In both cases:

• the computer gets help in keeping input tokens distinct by restricting the breadth of expressivity of the input space,
• the human has to alter their input to improve recognition for cases the computer has difficulty with.

It is definitely impressive from a practical perspective in that tweaking the voice recognition like this can produce much improved results.

Re:Coding != Typing

[buchner.johannes]

In the talk he says open source voice recognition software, e.g. Sphinx, don't work at all, and you have to go for Dragon.
But then creates writes his own command language.

So I don't understand why do we need voice recognition software that recognizes English? All we need is a open source package that translates sounds into some phonetic dictionary (e.g. IPA), and from there the second problem is to translate that into a native language.

Or even simpler, give it a dictionary of commands, and get it to find the closest match, or if all are unlikely, do nothing.

Why try to go for something so hard, but there is no software for the simpler problem that would make people very productive by just commanding the computer?

Codijng faster by voice because...

[Anonymous Coward]

His coding by hand has been crippled by RSI. That's like saying someone can get around faster in a wheelchair because they've broken their legs.

Re:Codijng faster by voice because...

[CastrTroy]

I think it means he's actually coding faster now than before he got the RSI. Although I think this works for the same reason that Palm Graffiti worked a lot better than regular handwriting recognition. Change the alphabet so there are no longer any ambiguous letters, and it's a lot easier to recognize them. In this case, he's only using a very limited vocabulary, with a very well defined grammar. This makes it a lot easier for the program to figure out what the correct interpretation is.

Re:Codijng faster by voice because...

[jeffb (2.718)]

That's like saying someone can get around faster in a wheelchair because they've broken their legs.

You might want to look up the record time for completing the Boston Marathon in a wheelchair vs. on foot.

And you thought *your* cube mates were annoying

[msobkow]

It's bad enough to hear people yelling at their phones in the cubes around. Now one can expect to hear someone yelling at the computer...

Sounds Tedious

[Polarised Bear]

I really can't imagine it being more effective than typing and I really don't want to sit and make sounds at my screen. If it works for him that's great but count me out.

Writing is easy

[rrohbeck]

90% of my work is debugging and even figuring out the failure scenario and testing against it. Writing the fix and new code is easy and quick.

Re:Writing is easy

[Tablizer]

90%? Maybe you suck :-)

Re:Writing is easy

[rrohbeck]

I'm not debugging *my* code :)

Re:Try that with LISP

[Mitchell314]

"Open parenthetheeth liphth wun too theven clothe parenthetheeth wetun"

Huh, it actually works.

This is like the corded keyboard

[Required Snark]

The mouse/keyboard combination was not the original combination envisioned by Douglas Englebart, the inventor of the mouse. He paired it with a chorded keyboard that could be operated with one hand. Clearly text input with one hand and mouse input with the other is a better input paradigm, but it is still not in use much today.

This use of speech recognition seems like a similar situation. It works for a few people, but it will not ever have a large user community. QWERTY keyboards are so dominant that their network effect makes other input modes irrelevant. Even those who adopt it will still be using conventional keyboards away from their custom environment.

Re:This is like the corded keyboard

[hibiki_r]

The problem is not with the person learning the new input UI, but with those that end up having to use your computer with zero training. I remember trying to help out a coworker that decided to buy a kinesis keyboard, and then set it up to dvorak: Not only I had to remember to switch layouts every single time I had to type something, and managed to miss 15% of my keystrokes due to its strange button placement, but then space, enter and backspace are all wrong, and right next to each other: It was just a horrible experience. I can only imagine what would happen with a chorded keyboard.

Re:Try that with LISP

[Anonymous Coward]

The first demo in the talk is editing lisp.

autocomplete as done in Borland C++ ..

[dgharmon]

> I'd have a hard time believing that this could be faster than someone using something like autocomplete as done in .NET ..

autocomplete was around long before .NET as was context-sensitive-help before Microsoft renamed it Intellisense ..

Re:autocomplete as done in Borland C++ ..

[Anubis IV]

Sure, but he was merely offering an example that most people here are likely at least somewhat familiar with. He never made any claims to its originality. I mean, if I was making a car analogy, I'd likely reference a car that people here know, rather than the earliest one that exhibited the traits I was referencing, just because the purpose of the statement is solely to supplement understanding of a separate topic.

Re:autocomplete as done in Borland C++ ..

[Anubis IV]

Auto-complete is not so common as tires on cars, however. Every car out there has tires on it, whereas I'd be willing to wager that FAR less than half of the tools used for writing code feature auto-complete (there are loads of bare-bones editors that get as far as syntax coloring but don't get around to featuring auto-complete). That the most popular ones happen to have it while hundreds of smaller products don't would lend more credence to the idea that it's useful to name a specific one, rather than referring to them as a whole.

Oompah Oompah Band

[Tablizer]

Pararenthesis parenthesis parenthesis. ... Was that one too much?

No, open "parenthesis" will be abbreviated "pah". And close parenthesis will be "ump".

Thus, coding will sound like, "Umpah lumpa, dipity doo, I have another puzzle for you..."

http://www.youtube.com/watch?v=qw0zZttfUaw

Input is not the limiter when coding

[gweihir]

Unless you are programming utterly structure starved glue-code, input is not the limiting factor, thinking about what you want to input is.

Re:Input is not the limiter when coding

[pla]

Unless you are programming utterly structure starved glue-code, input is not the limiting factor, thinking about what you want to input is.

You beat me to it.

I can type in code pretty damned fast - Fast enough that people frequently ask me how often I go through keyboards - Fast enough that I've actually had people in the room with me ask if I had just typed something meaningful or merely mashed keys for the hell of it - And, while coding, I tend to spend far, far more time thinking than coding. Someone watching me program for an hour would see 3-5 minutes at a time of complete inactivity, followed by assaulting the keyboard for a 30 second burst, rinse wash repeat.

Re:Input is not the limiter when coding

[Kjella]

You beat me to it. I can type in code pretty damned fast - Fast enough that people frequently ask me how often I go through keyboards - Fast enough that I've actually had people in the room with me ask if I had just typed something meaningful or merely mashed keys for the hell of it - And, while coding, I tend to spend far, far more time thinking than coding. Someone watching me program for an hour would see 3-5 minutes at a time of complete inactivity, followed by assaulting the keyboard for a 30 second burst, rinse wash repeat.

I think I can have considerably longer buffer/burst cycles, the challenge is keeping the big picture in your head while doing the little parts, and there I feel the duration of the bursts matter. If I've figured that to solve a business problem I need to change code sections A2, B4, C3 and D1 I'll start working on A2 and if it's quick and easy I won't forget the rest while if I struggle and need to churn out a lot of boilerplate by the time I'm done I might not remember what those other changes were. Either you then have to take notes or pseudocode the whole solution first or recreate it from memory, in those cases faster input would help keep me "in the flow", even though the input itself is only a small fraction of the wall time.

New rapping fad

[Tablizer]

You: "Biff Niff Bobblewop, Mop Top, Flip Flop Dribble Nibble Bazzle Dazzle Ropple Popple Slip Dip..."

Boss: "Get to work, no more rapping!"

You: "I'm not rapping, I'm programming via voice commands. Actually, I'm doing both."

Boss: "Yeah, right. It's called Riff Raff."

Re:The secret is hot sauce

[PolygamousRanchKid]

So... what? Do you just chug it?

I believe it's intended to be administered as an enema . . .

You might be able to convince some frat boys into trying it . . . they're already doing it with alcohol.

Not shocking

[quantaman]

Not a fan of evolutionary psychology, but I think there's a lot of reason to think we do have an aptitude for spoken language. I wouldn't be at all surprised if a well designed voice system left more mental focus available for the task of coding.

I'm not sure if the technology is there yet, and you still don't want to hear your officemates jabbering away, but I could see the theoretical usability of a spoken word interface surpassing that of typing.

Must see, VR coding in action:

[2fuf]

https://www.youtube.com/watch?v=MzJ0CytAsec

Re:Mouse

[xlsior]

Does his RSI prevent him from using a mouse? How does he cut and paste?

If you actually watched the video, you could have seen that he was using voice commands to select blocks of text and cutting/pasting them that way as well.

#cough#

[MickLinux]

#snort# #ugh-phptt# gah gak #thwip# #snrgle# #cmhmm#...

Save

Compile

There. Your routine is done.

Re:Get the computer to think instead

[Tablizer]

Welcome to Slashdot, Mr. Ballmer.

Lisp programmers need foot pedals

[ebno-10db]

What Lisp programmers really need are two foot pedals - one for left parentheses and one for right parentheses. That should cover 90% of their input requirements.

Re:The secret is hot sauce

[Anonymous Coward]

Capsaicin is used for pain relief in various cases, applied topically. Typically it involves putting some other short term, local anestetic on the area, then putting really high strength capsaicin paste on the area. It overloads the nerves, so after it is removed, perception of pain is reduced for quite a while later. Although I would be careful about using it for RSI. While capsaicin can reduce very specific kinds of inflammation that are caused due to response of nerves, if that isn't relevant to your situation, you might just be treating the symptom. You could end up continuing to damage things, just not feel it while it happens.

This is awesome and will change peoples' lives.

[students]

I am seeing lots of negative and off topic comments, many of which show people only watched part of the video. I thought it was totally amazing. When I was a teenager I had to change the course of my career away from computational science towards experimental science because of RSI-like problems. If I had his tools when I was 14 and had known I needed to use them, my career would be totally different and possibly much better since my programming talents would actually have been used.

Today, I can't really afford to spend several months learning to replicate his work, but hopefully soon it will be easier to learn. This will never be for everyone - some people can't use modal programs - and maybe it will never work in every context (Can it talk to my 20 year old Tektronix oscilloscope over GPIB?) but the video showed it can work. I hope good documentation, native linux support, and support for latex will be forthcoming so it can help me do science.

Re:He was doing it wrong

[zieroh]

I have been typing for more than 30 years and I don't have an issue.

I have been touch typing for 30 years and I don't have an issue either. If you think either of our statements actually proves anything, you're not very smart.

Re:Try that with LISP

[mwvdlee]

Spoken like somebody who has no idea what either LISP or COBOL is.

Voice computing, once again

[whitroth]

... and the response, from 20 years ago, is still the same:
        a) "Hey, Joe, are you still working on that difficult piece of code, or coming to lunch?
                  AAARRRGHGHGHHH!
        and the ever popular
        b) (as disgruntled, just fired emplyee walks out of HR) START! RUN! FORMAT C:! YES!YES!YES!

                      mark