Developing "Eyes-Free" Gadgets and Applications

The New York Times is running a story about Google engineer T. V. Raman, who lost his vision at age 14 but didn't let that stand in the way of his interest in technology. In addition to modifying a version of Google's search engine to give preference to pages that were more compliant with accessibility guidelines, Raman is now working on making cell phones easier to use without needing to look at them. "Since he cannot precisely hit a button on a touch screen, Mr. Raman created a dialer that works based on relative positions. It interprets any place where he first touches the screen as a 5, the center of a regular telephone dial pad. To dial any other number, he simply slides his finger in its direction — up and to the left for 1, down and to the right for 9, and so on. If he makes a mistake, he can erase a digit simply by shaking the phone, which can detect motion." Raman and a co-worker, Charles Chen, are also attempting to extend various phones' ability to read back scanned text to include signs that are anywhere in the phone's field of view.

Comments

Re:Reading Signs

[hedwards]

Or we could admit that the current crop of iPhone inspired phones are a tremendous leap backwards. Seriously, even if you're not blind, the fact that you have to pull it out of your pocket to use is a pain in the ass. Makes me wonder what the point of wireless headsets really are if you ultimately still have to look at the phone to use it.

I'm not sure any of that other stuff is going to be helpful if companies are exercising in a pathological hatred of necessary buttons.

They should follow the Screenless MP3 Player.

[MrCrassic]

I wish I could remember the name of this device, but it was essentially a MP3 player with no screen; just directional buttons and voice-based navigation. It was manufactured for blind users, and it worked wonderfully. It could even read text files and accept CF cards for expansion!

I think that a cell phone with just buttons on it and braille lettering would suffice, provided that the voice navigation is really good. An added advantage is that having no screen can make for very thin and attractive devices, if aesthetics is something of a priority for them.

Re:They should follow the Screenless MP3 Player.

[rusty0101]

One of the down sides to hardware that has no screen on it is that there is not a sufficient market for general use, and as a result the cost to the consumer is significantly higher. Braile readers for example end up being hand made because there isn't a sufficient demand to mass produce them, but the result is a display that's one or possibly 2 lines of text, that costs a couple hundred to a couple thousand dollars to produce.

Part of the idea of what Raman is doing is taking existing consumer hardware and applying software solutions to make them more usable to the blind.

A similar solution would be to use a cell phone to convert the words a person is speaking into text that a deaf person could read. You could build into it language recognition and translation and the deaf person on the bus next to you on your next tour of Europe, Japan, China, etc. may get more out of what is going on around you than you do.

A screen free MP3 player may sound like something that a blind user would appreciate, especially if the control hardware, and the audio menus were well designed. The big problem becomes one of what all the device can do. Adding a camera to a device like this seems somewhat counter intuitive from a consumer goods perspective, even if the hardware cost were just additional pennies. How about gps location hardware. Again just pennies, but to build it into that MP3 player doesn't seem to make all that much sense, does it. Add in a compass and an accelerometer, and you have a navigation aid that a blind user could use to get almost anywhere with.

The thing is that pretty much everything described is in a G1, so really adding usability for the blind is primarily a matter of plugging into the user interface with something that converts text to speech and where possible images to text. Both of those are partially solved problems. The part that's still difficult is getting it all tied together, and making available sufficient processing power to make it operate cleanly.

Re:what about speech recognition

[Anonymous Coward]

Voice recognition is annoying enough for sighted people. For the blind who place greater reliance on audio feedback from the device it's a pain in the ass. Every time you say something to the phone you'd then have to stop speaking in order to listen to it respond. Or keep talking until the end and then try to fix all the errors through voice control... fun, fun, fun.

Re:Reading Signs

[djupedal]

>Seriously, even if you're not blind, the fact that you have to pull it out of your pocket to use is a pain in the ass.

Yeah, because technology such as 'hands-free Bluetooth' hasn't been invented yet on your planet.

When I'm using an earwig or in the car, the phone stays in the pocket...everything, from connecting to the car to taking calls happens automatically or via voice control - what a country.

That's pretty cool

[geek]

I went to a college that catered heavily to people with disabilities, primarily the deaf but also a large number of the blind. It occurred to me early on that a great deal of the tech they use is developed by people without disabilities and then tested on people with them.

I think Mr. Raman is pretty unique in the sense he's able to develop like this as a blind man. That said, I'm curious why voice recognition wasn't considered the better option?

Re:SWYPE

[rusty0101]

swype is intended as a replacement for an entire keyboard, and presumes that you are reasonably close to the first letter in the word.

One way to make swype work like this would be to start with a 'common' character on the keyboard and draw from there. Preferably central to the keyboard. Another alternative would be to build a 5x5 grid of letters, doubling the c and k characters into one cell of the grid, and using that as a swype board. (It would fit on a screen better as well) That layout is what the tap codes that POWs have used for communicating were based on. A 'faster' version of that would be to build the 5x5, layout where the corners could be function keys to switch between Alphabet, numbers, symbols, and perhaps smilies or commonly used words. Set the alphabet layout to start with 'E' in the middle, 'A', 'I', 'O' and 'U' in the corners around the 'E', and fill in the rest with the most commonly used 4 consanents, then the rest of the alphabet (c/k doubled, or possibly q/z) in a ring around the middle 9.

On the 'number' pad 5 is in the middle, as described, and the outside 13 unused keys could be speed dial entries, or something like that.

Those 'doubled' c/k or q/z keys would be software selected based on context. (which makes q/z seem more intuitively a better combination, not a lot of words using both letters.)

The big difference is that the swype idea is intended to speed up entry of entire words, and presumes you can see the keyboard on the screen. This input method is intended to enter individual numbers (and later by extension characters) on the assumption that you can't 'see' the input surface.

An alternative would be to use a multi-touch interface to do 'chord' key entry. From what I recall the G1 is not able to support that in software yet, and I don't know if the hardware supports multi-touch yet either. However a bluetooth based Chording keyboard would probably work very well.