Designing an OS for Blind/Deaf Users?

Sushant Bhatia asks: "I work for a team developing technology for individuals who are blind and I have had the opportunity to use some screen reading software and while there have been leaps of progress it is still quite tedious to use, and not at all user friendly. One of my managers recently posed an interesting question for me: 'How would you design an OS from scratch that would target individuals who are blind and/or deaf?' What about inputs such as keyboards or refreshable Braille devices?"

Comments

CLI would be IDEAL

[BagOBones]

Really if you are going to take only text as input and output is going to be serial text, speech (blind but not deaf), braille CLI is the way to go.

This does not prevent you from multi tasking BTW, it simply means that you need to work within a well defined context.

Nothing new to invent.

Needs

[Oculus Habent]

I knew a pair of blind gentlemen who worked MSN Tech Support, and we set up a computer for the two of them to learn MSN Explorer with JAWS piped to speakers so they could both listen together.

The experience left me both in awe of their ability to hear all sorts of detail and in disgust at the lack of accessibility. The the custom interface was made out of poorly named images. One particularly useless one (Image 14, IIRC) was the minimize, maximize, and close buttons, all together. This brought me to my thoughts on a vector-based UI. Imagine the convenience of smooth scalability across different resolution displays...

Anyway, concerns that I can think of are as follows:

1. API
A series of abstracted interface methods should be made available. The categories are pretty simple... User Interface (menus, buttons, inputs), Text (static & editable text), Media (audio, video, pictures)... this is all off the top of my head, so feel free to improve on it. Each category simply defines a type of data, and then you can build ways to retrieve or interact with it.

2. Registration
I don't care if everyone puts their close button in the same place with the same icon. Visual users can typically locate these things. What they should do is then register that component with the UI Manager. Components could fall into multiple categories, i.e. a graphic on a web page with URLs mapped on it is both a picture and a series of links. Add a "group" indicator or hierarchy to properly collect controls and data together, and I think you have the basic needs covered.

Using these two parts, we should be able to build simple command interfaces. The ability to define the set of controls, displays, and texts for a given interface means you can see them all at once, or hear or feel them in sequence. Your interface can choose to discard or delay extra media (no sudden advert noises on audio interfaces or no need to waste processing time on decoding the video portion of a media file) through a variety of user-adjustable settings.

For visually-impaired individuals, I think the vector-based interface could make huge strides. Right now, you can buy a 21-inch monitor and set it to 800x600, or use a projector, but new laptops are still 1024x768 or higher. I listened in on a Dell Customer Service call from an older gentleman who loved the laptop he purchased, but couldn't read the high-res screen. If a vector-based interface was available that allowed his to change the point size - similar to Mozilla's Ctrl-Scroll size changes - he would have been fine.

I think the key, and the hard part, is getting buy-in on this kind of pervasive detailing of interfaces. HTML/XHTML is a great start for this, because this kind of extension is very easy based on the nesting and pre-defined components on a page.

Interfaces for the disabled or impaired could come in handy for everyone. These same advances are where the "technologies of the future" come from. Until we push the mouse away, we're stuck to the desktop metaphor.