Slashdot Mirror
Type With Your Eyes
hof writes: "Ever wanted to enter text by just looking at the screen? Take a look at Dasher. You enter text by looking or pointing to letters or words which the program thinks you are about to enter. I wonder how this can be optimized for coding -- a break for your wrists, and the code is available under GPL."
I know a guy who was born with a serious physical handicap where he has very little motor control.
He cannot write, type, or even speak. For the longest time, he actually used a board covered with the alphabet to 'talk'. He would look at the letters on the board, and you had to decifer what was being looked at. This way, he could spell out what he wanted to say. His parents were quite quick at it, and they could carry on a conversation very well.
He actually upgraded to a pair of glasses w/ a small laser on the frame a few years ago. He could then spell by looking at the keyboard, which was covered with photo-receptors. Then, the computer would talk to you Hawking-style. It was a groovy innovation. It was quite pricey, though.
Perhaps an open-source innovation such as this could open up doors for people like him. It would make equipment used for social interaction cheaper and more readily available.
Hmm, you say it would be a pain for programming.
Imagine instead a 'phrase' keyboard. for, while, {, your variable names, and object method calls all floating towards you.
Pick and assemble you algorithm.
Brilliant. The applications go way beyond simple strings of letters.
[% slash_sig_val.text %]
Did anybody else think immediately about Stephen Hawking upon seeing this? I searched the site, and it says in the history that he was originally one of the mein targets for the project. Anything that can possibly help him to communicate faster would be wonderful, as well as for all other disabled people. Nice work so far!
-
Basically they use a markov chain which has in it the probabilities that one letter will appear after another. It's very similar to the disassociated press generators you can find out there.
For example, here is one I wrote which generates new random words based on the probabilities of one pair of letters appearing after another pair. I used pairs because it generates more English-like words.
It was "taught" using the contents of /usr/dict/words and written in Perl.
Avantslash - View Slashdot cleanly on your mobile phone.
Noone seems to want to answer the second part of the post...
For programming, it would have to be integrated with your IDE in an editor which is both syntax and semanitcally sensitive. Dasher uses some sort of a dictionary - usually in English - to predict what the next letter is most likely to be. This is obviously not a traditional dictionary, because it manages to predict across words.
For coding use, the dictionary, instead of being static, should be dynamic. Instead of having all the words in the English language, it should have only words (and symbols) which are semantically valid at the point the cursor is positioned in the edit buffer. Furthermore, it should weight them. Local variables are very likely, method variables are less likely, strange packages even less likely. In fact, it might subcategorise these into pseudo-letters, so that when writing (e.g.) Java, packages would appear as a single pseudo-letter. It could also add localisation-type information - entities referred to within a few lines are more likely to be accessed again soon. This doesn't change the sorting order used by Dasher, which is always alphabetic, but it does change "visual space" allocated to each letter in the search area.
You probably want an auto-beautifier i.e. new lines and indents/outdents are added automatically as needed.
This has some interesting side-effects. For example, you can only enter semantically valid programs. It has implicit auto-completion - once the following letters are unambigous, the remaining characters occupy the whole namespace.
The brackets case is also nice. The only close bracket that is ever possible is the close of the most recently opened, and that often has a very high probablity. Which means that inserting an open bracket implicity "arms" the system with the matching close bracket. The same applies for closing strings - the close string charager is always high probability.
For case significant languages there are some interesting effects. Obviously, in principle both upper and lower case must be present, though it may well be that a small minority of letters are accessible at any given instant. At a guesss, it would be better to split into Upper case and Lower case rather than interleave upper and lower - but that is something to experiment with.
Punctuation-type characters show minor problems - we all know alphabetic order, but does ";" come before or after "+"? I don't know. But I expect we could learn this - there aren't that many symbols.
One could also add a special pseudo-section for language-defined keywords, so you just chose a single prefix zone and then go straight to the set of all known keywords. Usuallly no more than 50 or so, and not usually all semantically valid, so you might get quite quick access to them.
Of course, there is a tradeoff with all these special zones. One of the points about Dasher is that you don't need any more specailised knowledge than having learned the alphabet to operate it. Adding language-sensitive zones and so on adds extra operator learning time. But since you have to learn the language anyway, I don't think it is that much of a burden.
I would expect this sort of strategy to (at least) double the input speed for Dasher for a particular programming language. Forget using your eyes - for the able at least - but it might make mouse-driven program-writing a lot faster. In fact, it might overtake typing for the special case of program input in a "known" language. Though I think the most valuable feature would actually be the inability to input a semantically erroneous program. Which means you "only" have to worry about logic bugs and not typing bugs.
A good place to try this would be to create a jEdit plugin. JEdit already has plugin Java browsers and beautifiers, so a lot of the code ought to be there already. A Java-style Dasher window would be a very interesting project. If anybody feels upt to doing this, I would like to help (I don't think I have got the time to lead such a project).
Consciousness is an illusion caused by an excess of self consciousness.
Furthermore, ERICA is integrated with Windows, so you can use it to completely control the computer and do almost anything you need (not sure how well it would work with Quake ;-) . And just to make it more interesting, it was made by the same guy that made Stephen Hawking's system!
Anyone with a PocketPC should definitely give the demo a spin. I tried it (last time /. covered this.. hehe), and it's pretty impressive how quickly you can become not just effective, but downright fast.