Designing an OS for Blind/Deaf Users?

Sushant Bhatia asks: "I work for a team developing technology for individuals who are blind and I have had the opportunity to use some screen reading software and while there have been leaps of progress it is still quite tedious to use, and not at all user friendly. One of my managers recently posed an interesting question for me: 'How would you design an OS from scratch that would target individuals who are blind and/or deaf?' What about inputs such as keyboards or refreshable Braille devices?"

Comments

One way to cut costs

[bravehamster]

Well, at least for BlindOS you don't have to worry about writing video drivers...

Re:One way to cut costs

[Tebriel]

Unless your tech support/any other user of the machine isn't blind.

Re:One way to cut costs

[pizen]

And not having to power a laptop screen means you should get great battery life.

Re:One way to cut costs

[SnprBoB86]

There's always SSH, remote desktop, and equivalents...

Re:One way to cut costs

[mattspammail]

Cut costs there, sure. But other possibilities exist. Would you really need as fast a processor on a box like this? Your output needs/speeds may not be as high as the needs of a traditional user. I am not a deaf or blind user (IANABODU henceforth), but I'm assum... stating that these should be different machines. Think of the possible needs/lack thereof.

Visually impaired: video drivers possibly not needed; traditional 3D gaming not an option; output devices less resource intensive (no monitor may be needed); no traditional high-resource GUI needs (skinning, fading, etc). There may however be some needs that are not listed here. For example, if you have a visually impaired audiophile who enjoys running resource-intensive software apps, a beast of a computer might still be what the geek ordered. Again, IANABODU.

Audio-impaired users would have needs largely similar to those of traditional users, and perhaps then some. Either way, speakers may not be needed, or sound cards, etc. Since sound is included on so many mobos lately, it's probably more expensive to remove it than it is to keep it. Again, IANABODU.

Another statement that should be made is that the typical user doesn't even use a fraction of what their computer can do. How many users do you know who purchase 3.x Ghz wintel boxes, just because their 2.4 Ghz is a year and a half old, just to camp in front of it and use IE/Outlook/Solitaire?

Re:One way to cut costs

[idonthack]

Poo. My sig broke. It's supposed to say:
What subliminal message?

---
"I hate quotations." - Ralph Waldo Emerson
Generated by SlashdotRndSig via GreaseMonkey

Re:One way to cut costs

[Aeiri]

so youre saying people who are not blind would be completely incapable of using a machine that did not have a video interface?

I can't read braille, can you?

Re:One way to cut costs

[xENoLocO]

Uhh... wouldnt it be easier to just add speakers?

um...

[ed.han]

maybe i'm being obtuse, but wouldn't the sole useful input method for the blind be verbal? as for the deaf: why would you not use a GUI? this seems too simple so what am i missing?

ed

Re:um...

[ne0n]

I've seen a blind guy type over 100wpm, no errors. also, his screen reader sounds like a mad speak&spell on crack, it's so ridiculously fast. Much faster, in fact, than most sighted people can read.

Re:um...

[pizzaman100]

One incorrect assumption about the blind is that they are blind. ;) There are a large spectrum of 'legally blind' people who do in fact have some usable vision. They won't be driving cars, but they can user computers. For such people, visual aids are helpful (magnification software, or really large resolution like 320x240).

My four year old daughter is legally blind, but she is learning how to use a regular computer just fine. It's amazing to watch. In the mean time, I'm working on having the government buy a really large plasma display to 'facilitate' my daughter's learning environment. :)

re:um...

[ed.han]

uberdave: yeah, i'm sure they can, but when you come right down to it, speech is still much faster. i seem to recall 40 WPM being considered the desirable benchmark for anything other than executive assistants, etc. i type at 85-90. but that's nothing compared to the speed of speech, which IIRC can run from speeds of 110-120 WPM. why limit the user?

here's a question though: instead of the folder analogy, would everything have a flat file structure, with users performing search queries for desired files, etc.?

ed

Re:um...

[SeventyBang]

You aren't missing anything. Someone got a little giddy over thinking of something interesting and jumped the gun.

I know several deaf people who don't care about mods to a GUI, including Windows.

When it comes to the blind, however, it doesn't mean it can't be a GUI, but there still has to be the correct UI. There are already a number of products which some of the blind people I know (one owns his own recruiting and consulting firm) and have said they don't need a change for most of what they do.

So no, I don't know why the original focus was on an OS. About the only OS which really worried about the UI is BEOS as it was designed for multimedia.

Maybe we should file a "request for clarification" from the author as to why they meant OS instead of UI?

Re:um...

[Txiasaeia]

It's not the user that's limited, but the speech recognition software. When I type, I can be reasonably sure that what I'm typing will actually appear on the screen (even with my eyes closed or looking away). Voice recognition, on the other hand, isn't as accurate, and any increase in speed gained by using speech will be lost for a for a blind person when they need to get the computer to read back what you've written and manually correct it.

Re:um...

[pizzaman100]

Another note: a good Windows utility for completely blind users is Jaws. It is a speech software utility for Windows. (example: to navigate, the software verbally says: 'start', 'programs', accesories', etc. I saw a demonstraton of the software by a blind man, and he kicked ass (kinda like the hacker dude in Sneakers). Anyway, he could navigate way quicker using the keyboard than the typical sighted windows user could.

Re:um...

[Pengunea]

As the parent article states, "blind and/or deaf". That possible "and" puts an interesting twist on things that blows "sole useful input" right out of the water.

Yes, blind and visually impaired people can use vocal interfaces to use computers. Personally as a legally blind programmer I can't STAND vocal interfaces. They bloat the amount of time it takes me to tab through code by a factor of at least three. So I make do with using the PC at a lower resolution. I'm one of the lucky ones out there who can do this with no problems.

As well there are braille-based systems available for completely blind induviduals and these sometimes also work for deafblind folks. There are also ways to adapt current OS'es for deaf induviduals so that any audial cues are represented onscreen. I'm not familiar of any tools that can process an audio file and create captioning though, which is a shame as that'd rock.

My point boils down to there aren't suitable OS options available for deaf, blind, AND/or deafblind induviduals. As the BBC reported working with technology can be an excessively frustrating situation for deafblind induviduals. In addition when you want to create a solution for deaf, blind, AND/or deafblind induviduals you have the issue of no two snowflakes being the same so to speak. On top of that it has to be easy to use. For those of us who can't/don't want to spend an hour just trying to read the first email in our inbox it has to be fast to use as well. There's also considerations such as multi-format training materials, child-sized input devices, learning curves due to induviduals being used to using other devices, etc. The list goes on.

This, folks is a REAL challenge. One I hope someone can rise to and overcome.

Re:um...

[Kirth]

Why the hell would I use a window-environment at all?

80xSomething, textmode. The same as the braille-device has. Various consoles, a screenreader in the background which starts reading on a specific key on any console and in any program (be that shell, editor or webbrowser); maybe a hacked up "screen"-program could do that, that would also allow for dozens of virtual terminals without logging in dozens of times. I've seen a *real hacker* work on a vt220 with screen, slrn, bitchx, mutt, dozens of consoles, vim, ssh. Incredibly fast.

In fact, most unices would already do nicely, I suspect, given some nice text-to-speech software and good drivers for braille-hardware.

A little unclear on specifics

[winkydink]

about the OS, but for deaf/blind people, the hw platform should be a pinball machine.

Old SNL skit...

[Anonymous Coward]

Before closed captioning, sometimes on the news, they would have a guy at the bottom of the screen doing sign language during a section called "News for the Hearing Impaired". On Saturday Night Live, they made fun of this, with Garret Morris, in the role of the anchor for the "hearing impaited" would yell "OUR TOP STORY TONIGHT" when the anchor would say "Our Top Stoty Tonight"

508 compliance

[tyates]

I would start with the US government's recommendations for app development for people with disabilities. Most apps written for the Fed have to be section 508 compliant, which helps ensure that they'll work with screen readers. Keep in mind though that from my (limited) experience, 508 compliance is more than an art than a science - you know, you get something that's kinda sorta 508 compliant.
http://www.section508.gov/

Apple VoiceOver and Universal Access

[daveschroeder]

While not part of an OS designed "from scratch" for vision-impaired individuals, Apple VoiceOver is the first such functionality of its kind to be included for free with a commercial operating system. It's a fully integrated screenreader and accessibility interface for Mac OS X, and is tightly integrated with both the operating system and its APIs, and is extensively supported in several common applications.

As for hearing-impaired individuals, the task is much easier, as the primary interface to a computer is already visual. However, visual alerts and features that would correspond with otherwise audio-only events have also been integrated by Apple in Universal Access.

In addition, Universal Access includes features that assist individuals with motor impairments as well.

While it may be an interesting and informative exercise to think about the types of things you'd do if you were going to "build it from scratch", it might be more productive to think about how these capabilities could be added to existing commodity operating systems, such that the technology can continue to be affordable and easily supported.

Re:Apple VoiceOver and Universal Access

[tetsuji]

One of the most useful things about a GUI is the visual cues that it gives to jog the memory. CLI is often more powerful, but it is lacking in visual cues and so takes longer to master because you have to remember everything.

So, what about creating some sort of sonic scheme for locating the user in 2d or greater sonic "space" that's equivalent to a sighted user's peripheral vision giving information about the GUI environment. I'm thinking perhaps musical chords could represent different parts of the "sonic desktop" and that chorded input could provide navigation for sightless users. Perhaps the keyboard could even be completely replaced by something like a stenographer's keyboard, where chorded input is the norm.

My ideas

[AKAImBatman]

Blind:

1. Use a device that creates a topographical image on a pad based on the screen color values. This would allow the blind user to "feel" his way around the screen. More advanced versions could allow the user to use his hands as a mouse, by accepting a certain amount of pressure as a click.

2. Controls on the device should allow the user to "zoom in" on particular areas. This would help the user more easily find toolbar buttons and the like.

3. Replace the system fonts with braille fonts. (I'm uncertain as to how one might add bold or italics for emphasis, but I'm sure a system can be devised. Perhaps extra bumps outside the normal character area?)

4. No right clicks. Right clicking is more of functionality for advanced users anyway. Mac OS X can get along without it, so other OSes should be able to do so as well.

5. "Selected" items should actually invert in the control device. This would allow the person to easily understand what (s)he has selected at the moment.

6. Standard controls such as checkboxes, radio buttons, and the like should be skinned to be more "feel" friendly. i.e. Simple invertable boxes would work better for checkboxes and radio buttons than our current iconic forms.

7. One handed brail keyboard? It's just a thought, but if the blind could be taught to use a one hand keyboard, they could read and type at the same time.

Deaf: What are some of the actual challenges facing a deaf computer user? Computers are primarily visual, and tend to suffer little with the loss of sound. (Unless I'm listening to music, I usually keep my machines muted.) My only thought is that the standard issues of movie subtitles apply.

Re:My ideas

[Anonymous Coward]

I wonder how many of these ideas would actually work in practice. Perhaps a blind person could contribute to this discussion, and we could get a true perspective on this. There's probably a huge difference between what sighted people think blind people need and what people who are blind actually need.

Re:My ideas

[AKAImBatman]

What is the point of that? Braille is meant to be felt, and would have no purpose on a screen

Dude, read it again. I'm suggesting that they have a screen that they can *feel*. Regular characters would be very difficult to discern on such a device, so changing the fonts to braille is the best solution.

Re:My ideas

[Dun Malg]

5. "Selected" items should actually invert in the control device. This would allow the person to easily understand what (s)he has selected at the moment.

Braille consists of raised bumps. "Inverting" them, i.e. turning them into dents, renders them essentially unreadable.

Re:My ideas

[AKAImBatman]

The first thing I tend to do is turn off the damn bell. I can tell when my input is not accepted just fine, thank you. In fact, (as I stated in my post) I consistently mute my computer unless I'm listening to music or watching television. (I have a TV card.)

Considering that muting the computer would have the same effect as being deaf (rest of the environment excluded), I fail to see how the terminal bell would help.

The only case where I see an issue is with people who are both deaf and blind. Special notifications such as new email would be hard for them to detect. The only solution I can find in that situation is to add a vibrator to the touch device. If the system has an event that the user should know about, the device should vibrate litely. Info should also be added to a system log that the user can check to figure out what the latest event was. (Especially useful for when the user was away from his/her desk.)

Re:My ideas

[AKAImBatman]

Braille consists of raised bumps. "Inverting" them, i.e. turning them into dents, renders them essentially unreadable.

Sorry, I was unclear. I was referring to icons and buttons being inverted. Text should never be inverted, but should actually be raised when selected. Thanks for catching that.

Re:My ideas

[Narril Duskwalker]

1. Use a device that creates a topographical image on a pad based on the screen color values. This would allow the blind user to "feel" his way around the screen. More advanced versions could allow the user to use his hands as a mouse, by accepting a certain amount of pressure as a click.

Man i dunno about that, can you imagine the stuff they would have to feel while going through their spam?

Re:My ideas

[AKAImBatman]

Man, if they ever develop such a thing,

You mean, something like this? There are a variety of one handed keyboard designs, the most common of which are chorded keyboards. The primary issue is that they require a lot more training than a standard 104-key keyboard. Most people wouldn't bother with such training, but in the case of blind users, many already go through various special training. (e.g. Braille, sign-language, etc.)

Re:My ideas

[Anonymous Custard]

Special notifications such as new email would be hard for them to detect.

There are pressable buttons on certain "multimedia" keyboards that open your email, open your web browser, etc.

A nice way to do notification for someone who can't hear or see would be to pop-out a similar button on the keyboard (or equivalent). When the email notification would normally appear / disappear on screen, the button would pop out / pop back in off the keyboard. The button could be pressed to start the email reading function.

Recreating the screen from a visual device into a tactile (feeling) device seems to be a common idea, so far.

Re:My ideas

[Eideewt]

Applied Minds has created a topographical map that works like that. Not pressure sensitive, I think.

One answer:

[Rosco P. Coltrane]

blinux.

CLI would be IDEAL

[BagOBones]

Really if you are going to take only text as input and output is going to be serial text, speech (blind but not deaf), braille CLI is the way to go.

This does not prevent you from multi tasking BTW, it simply means that you need to work within a well defined context.

Nothing new to invent.

Re:CLI would be IDEAL

[Kirth]

The standard unix tools need to be redesigned to be as easily spoken as they are interperated by the average user.
Absolutely. There's a nice article on this. The author implemented an "ls" which outputs "644" on "ls -p". I'm all for something like that. Special switches to GNU ls or whatever to allow things like this. As long as there are enough letters left for parameters, do it.

And then, there is /bin/ed of course ;).

On the contrary

[phorm]

The OS is going to be at least somewhat compatible with other OS's to be somewhat useful. I should also have specialized input devices for people with hearing or vision deficiencies, and probably be compatible with modern X86 hardware (unless you plan on starting everything from scratch).

Since visual input plays a huge part in interfacing with a modern OS and/or software, you're probably not going to need software compatability. You would need, however, to make the document formats compatible with say office, or a standard XML format, etc.

Still, an OS redesign is a big task. It might be better to go with an existing OS.

So basically, you're not going to need "video" drivers, but you will need an interface with specific hardware for the blind, various document formats, storage/printing devices, etc etc

Re:On the contrary

[Baricom]

I agree. The challenge in developing an accessible user interface is that every disability has a different set of needs. For example, a person with a hearing impairment might be able to get along just fine on a general operating system - my home computer has a pair of headphones which I basically only use when I'm listening to music.

It seems to me that operating systems today are farthest behind in serving people with visual impairments. It's obvious that a person who is totally or partially blind isn't going to have much fun trying to use a Graphical User Interface.

I think the best solution at the moment is to write software that works within existing operating systems, using the built-in accessibility toolkits. A lot of improvement could be realized in this area alone.

Re:On the contrary

[laffer1]

You may not need video drivers, but some people are legally blind but can still see a little. I had a history professor who could not drive and had mr. magoo glasses. He had to hold up tests and quizes up to his face to read them, but he still functioned. He would have great difficulty using a screen most of the time, but it would still be useful to have video for certain things.

Also, I do a website for a blind business owner. He has his secretary look at the site, etc. Sometimes she looks at his computer to help him. In that case, basic VESA video support would be nice. Interesting thing is that he actually coded his original website before i took over. He still does updates. Not to bad at html actually. He has some trouble with css properties but i think thats understandable! In this case, he lost his sight during his teens so he has an idea what a box is or colors.

I personally think a linux or bsd would be a great base and then a totally customized X11 system with a specialized window manager would do the trick. The real time would need to be put into the navigation system, web browser, text editor and word processing software. It would be nice if someone thought outside the box and made other types of software like accounting software (quickbooks type stuff) etc. I'm amazed what a blind person can do.

The guy uses windows with some very expensive software. He told me it was 20,000 dollars for a dell customzied with the software. The reason is that the software isn't expensive, but customizing it for the setup need by that person is. If someone could design an interface that allowed the blind person to migrate settings or do it themselves it would save them a lot of money!!!!!

i worked at a blindness charity on the computers

[hilaryduff]

this was in '97. one bit of advice i can give is to make it so blind AND sighted people can use it. this is important because often the blind user will need to be taught to use it, and another blind user isnt always available/ideal for teaching another blind user (depending).

Oralux

[TheSync]

Oralux is a Knoppix live-CD to facilitate access to GNU/Linux for the visually impaired. The Oralux user interface is based on Emacspeak or Yasr, and has FLITE and EFM (Festival/MBROLA version).

Re:A small sense of advancement.

[uberdave]

What really weirds me out is braille signs in airports and subway stations. How is a blind person supposed to know that there is a sign there, feel the entire wall?

Use Linux.

[DosBubba]

Some Links:
Linux Accessibility HOWTO
The Blind Linux Project with mailing list
brltty for your braille input needs
Or, get an all in one distro

AN OS?

[Quasar1999]

What the hell does an OS have to do with who's using it?

An OS should be a standard toolset for applications to be built on. Memory management, driver management, and application management is what an OS should do... it should worry about dividing hardware resources up among it's processes, it should worry about managing drivers... it should not have anything to do with the user interface that runs on it. This question was obviously posed by someone who hasn't the first clue about what an OS actually is.

Re:AN OS?

[sixteenraisins]

it should not have anything to do with the user interface that runs on it

And are there not tools available in an OS that allow the user to choose and install applications for the OS to manage? How might such a disabled person start and/or stop any processes on your OS which, as you say, should not have anything to do with the user interface?

Wipe your hard drive and do a fresh install of your favorite OS, then blindfold yourself and install and use your apps. Go ahead, I dare you.

Re:AN OS?

[Infernal Device]

On the other hand, both your response and the one it responds to are crass and ignorant.

It is reasonable that a non-technical person would confuse the OS with the interface, since more and more, the interface is designed to shield the non-technical user from the inner workings. This is the world we live in, rather than some idealized form where everybody has a clear understanding of everything.

Perhaps the OP's real expertise is in working with the deaf and blind in a different area rather than computer interfaces. Perhaps this person has not had the exposure that you have had. Whatever the case, it doesn't matter if the person was informed or not; by both of y'all showing your asses, you've contributed a little bit more to one of the problems, rather than a sensible solution.

Re:AN OS?

[Anonymous Coward]

The phrase "Blind OS" represents muddle-headed thinking from the very start.

An operating system is an abstraction of resource managers of various hardware-level resources: CPU, disk, etc.

The challenges a blind person experiences in using a computer has nothing to do CPU scheduling, filesystems, interprocess communication, nor of anything else that relates to operating systems. The challenge of the blind is all in the user interface.

(Sigh.)

FINALLY!

[drewzhrodague]

It must not be so obvious to other people, but this is quite obvious to me. Computers don't necessarily need screens, we don't need to be hand-eye-coordinating little pointers on screens anymore. FOr those of us that have (most of) our sense intact, doesn't it make sense to not shunt them into a little tiny box?

So I'm talking about UI.

Does it make sense to use a mouse to click the start button while you're in bumper-to-bumper traffic, and driving a manual transmission? I can't imagine why this is still the case. Hay you UI and interface designers -- take a lesson from those of us who can't hear, or can't see, and imagine how they'd get things done. Same concepts can and should apply to handhelds, phones, PDAs, remote controls, etc. There are far more attractive (women) things to look at than little blinkety gadgets. Also, lots of user-wishes can be infered from small sensors (photodiode, accelerometers, available Wi-Fi networks), and the laptop doesn't really have to be open all the time in order for us to be connected and informed.

Please take a lesson from the needs of our blind and deaf brothers -- and enlighten the UIs for everyone.

Rockbox - MP3 Player OS For Blind Users

[meehawl]

The open-source mp3 player OS Rockbox includes a "Talking Menu" option that will read back commands, playlists, and song/file information. It's very useful for blind users, as well as sighted hands-free/driving use.

first person shooter for blind OS

[esobofh]

your in a room - do you;

1 go left
2 go right
3 go straight
4 reload

Re:Ok, blind/deaf are covered, what about the dumb

[PaxTech]

There already exists an OS for the dumb. It's called Windows.

*rimshot*

Deaf need visual cues

[blondieeng]

Okay, "haha" about the Deaf jokes. As a female (gasp!) Deaf computer user (abuser?) what I need is a captioning feed for videos. Popular news sites routinely feature video news clips but they are meaninless to me without knowing what is being said. How about some captioning? Too expensive? It sure seems so because I sure as hell never see it. As for Braille, not all blind people know Braille and legally blind people usually have *some* sight and prefer to utilize their sight as best they can along with screen readers.

I wouldn't

[kfg]

I think this is one of those cases where it isn't mere pedantry to point out that the OS and the shell/interface are two very different things and that what you want is an interface designed from scratch for the blind.

You can use any already existing decent OS as the base.

You do not have to reinvent the wheel to invent the wheelchair.

If it were me I think I would start out by trying to scratch build a decent IRC client. What you learn by doing that will teach you things you will need to know about such interfaces before you start out at a lower level.

KFG

Re:One way to cut costs - outsourcing!

[Tackhead]

> > Well, at least for BlindOS you don't have to worry about writing video drivers...
>
> Unless your tech support/any other user of the machine isn't blind.

Ah, this is about finding an OS for tech support folks in India. They could be blind, because all the work is done over the phone. Spending five minutes on the phone proves they're deaf. And the quality of the solutions proves that they're dumb.

Ever since I was a young boy,
I took the support call,
From Delhi down to Bangalore,
I must have played them all.
But I ain't seen nothin' like him
In any support-cube hall,
That deaf, dumb and blind tech
Always says "re-install!"

Sits there like a statue,
He's a voicemail machine,
Please to reading from scripts,
Keeps his call queue clean,
Bullshits by intuition,
Never seen him fall,
That deaf, dumb, and blind tech
Always says "re-install!"

Mac OS9 was most blind freindly OS ever

[Anonymous Coward]

Mac OS 9 was the most blind enabled ever, even for 'refreshable-braille"

The reason... for the first 10 years maybe 15 years EVERY developer followed EVERY rule and used the official GUI and official controls (with text labels in them) and the compter gui was also mode-less, as well as very intuitive.

Sadly... apple hired cretinous morons who destroyed the gui in apples own idiotic tangential offerings ruining everything.

At one point in their own written standards manual 'HUMAN INTERFACE GUIDELINE" apple proposed that COMMAND-C which means "copy" (except under a few versions of dvorak keyboard mappings) was to not mean copy, but was to mean "CANCEL" if a cancel button was visible in a dialog!!!! HAHAHAHAHAHHAHAH

The Next was idiotic in version 0.9 of NeXTStep OS and Command-2 was duplicate but they eventually changed it to Command-D for all apps.

But OS9 was flawless, and every app worked well with blind software, and people rarely double bufferred pixels, and even if they did, it was easy to track the blits because the OS bitcopy commands were always used by all programmers so text strings could still be located without ever ever using "screen scrapers" to perform OCR crud.

Blind worked great on macs.

As for blind-AND-deaf ... same thing... but the issue becomes a matter of two handed browsing on a screen. On hand moved on a tablet and the other hand received braille on fingertips

eventually i got a job at a company that sold such software (AT A FINANCIAL LOSS BUT FOR CHARITY) called Berkely systems. They even employed fulltime blind employees to test the windows-only version which was a poor substitute for perfect competing mac OS9 versions. Why? becuase no wintel idiots followed any standards ever for getting text onto a screen or following GUI guidlines for controls.

The Mac OS9 is still useful. might as well stick with it. Apple distributed LowVision screen zoomers (an optional control panel on the installer CD) for years standard free with all copies of mac OS for people with extrememly poor vision.

Re:OS design for people with disabilities

[pizen]

ding ding ding!

I think the blind and deaf would benefit from a preemptive scheduler in addition to a journaling filesystem.

Already done

[TuringTest]

It's called Archy. This OS is a redesign of the command line with a focus on habit-forming, not navigational use.

The LEAP technique for quick positioning would make it better for blind users than a traditional CLI where users can't easily scan the output of a command. In Archy, users can touch-type the destination point and have it read in loud voice, instead of having to hear the whole text.

Needs

[Oculus Habent]

I knew a pair of blind gentlemen who worked MSN Tech Support, and we set up a computer for the two of them to learn MSN Explorer with JAWS piped to speakers so they could both listen together.

The experience left me both in awe of their ability to hear all sorts of detail and in disgust at the lack of accessibility. The the custom interface was made out of poorly named images. One particularly useless one (Image 14, IIRC) was the minimize, maximize, and close buttons, all together. This brought me to my thoughts on a vector-based UI. Imagine the convenience of smooth scalability across different resolution displays...

Anyway, concerns that I can think of are as follows:

1. API
A series of abstracted interface methods should be made available. The categories are pretty simple... User Interface (menus, buttons, inputs), Text (static & editable text), Media (audio, video, pictures)... this is all off the top of my head, so feel free to improve on it. Each category simply defines a type of data, and then you can build ways to retrieve or interact with it.

2. Registration
I don't care if everyone puts their close button in the same place with the same icon. Visual users can typically locate these things. What they should do is then register that component with the UI Manager. Components could fall into multiple categories, i.e. a graphic on a web page with URLs mapped on it is both a picture and a series of links. Add a "group" indicator or hierarchy to properly collect controls and data together, and I think you have the basic needs covered.

Using these two parts, we should be able to build simple command interfaces. The ability to define the set of controls, displays, and texts for a given interface means you can see them all at once, or hear or feel them in sequence. Your interface can choose to discard or delay extra media (no sudden advert noises on audio interfaces or no need to waste processing time on decoding the video portion of a media file) through a variety of user-adjustable settings.

For visually-impaired individuals, I think the vector-based interface could make huge strides. Right now, you can buy a 21-inch monitor and set it to 800x600, or use a projector, but new laptops are still 1024x768 or higher. I listened in on a Dell Customer Service call from an older gentleman who loved the laptop he purchased, but couldn't read the high-res screen. If a vector-based interface was available that allowed his to change the point size - similar to Mozilla's Ctrl-Scroll size changes - he would have been fine.

I think the key, and the hard part, is getting buy-in on this kind of pervasive detailing of interfaces. HTML/XHTML is a great start for this, because this kind of extension is very easy based on the nesting and pre-defined components on a page.

Interfaces for the disabled or impaired could come in handy for everyone. These same advances are where the "technologies of the future" come from. Until we push the mouse away, we're stuck to the desktop metaphor.

Point one

[gmhowell]

My wife uses some of these devices (BrailleNote QT) and the biggest problem is using the wrong underlying operating system. DOS and Linux work just fine when translated to the command line; the WinCE that underlies her machine sucks. Hard. Sure, it gives a bare modicum of device and software (read: Outlook) compatibility, but at the cost of needless hardware overhead, and a special version of the three finger salute.

Asking 'what about blind and/or deaf' shows a bit of lack of understanding. The best replacement for the blind is speech. The only option for DB is refreshable braille. I suggest you and/or your employer make some contact with DB groups. At a DB camp a couple of weeks ago, some devs and sales reps from some tech outfit (forget which one) displayed their wares. The kneejerk /. response is "did they contribute for their market research?" To which I say "sod off". The market is fairly small, and should be one with a tighter feedback loop between vendors and users. My email address is non-obfuscated should you need some ideas or contact info for some groups.

Finally, expose the API and make some generic libraries available to use to people who like to program. PulseData/HumanWare wants, I belive a dev kit fee. Umm, screw that. Entry level prices on the BrailleNote are ca. $3000, IIRC, so there's plenty of money out there. Not sure about you, but Franklin Scientific, Blazie, PD, etc. are Hardware companies. Let a little bit of "Open Source"ism do some development for you. There is tremendous word of mouth, and if some third party makes some brilliant add on, it will get around the community. Depending on how you license it, you may be able to ship later. (Personally, I would go BSD or LGPL.)

In summation:

1. Don't confuse blind and deaf and deaf-blind. Each is unique with unique needs. An attempt to be all things to all people will either have stratospheric costs or poor quality or both.

2. Ask your customers instead of some random slashbots. They are out there.

3. Build it from the ground up, since you'll likely have bizarro hardware anyway.

4. Make it possible to program for it. It (the DB group) is a tight community, with lots of people looking out for others, so it's not like you'll be helping a competitor. Think of third parties as value added.

5. (Not mentioned above, but kinda goes with 2) Update your freaking website with real, up to date, and complete information. Have a company policy of a real, human reply to all correspondence within x hours, where x72. Even "I got your note and am investigating your concerns" is better than some of what I get from PD from time to time.

And a freebie. My wife never uses the voice prompts on her BN, but I activate them if I have to do tech support (needing an onsite geek is a bad thing, BTW). Try to get a speech synth chip that sounds better than the WOPR or Speak and Spell I had as a kid. Seriously. It's 2005, and every time I hear the voice, I expect to hear "Would you like to play a game?"

Re:Point one

[Anonymous Coward]

Finally, expose the API and make some generic libraries available to use to people who like to program. PulseData/HumanWare wants, I belive a dev kit fee. Umm, screw that. Entry level prices on the BrailleNote are ca. $3000, IIRC, so there's plenty of money out there. Not sure about you, but Franklin Scientific, Blazie, PD, etc. are Hardware companies. Let a little bit of "Open Source"ism do some development for you. .... (Personally, I would go BSD or LGPL.)

That is exactly what I tried to do when creating libbraille... an open source (LGPL) library which supports more than 30 types of Braille displays (PulseData, F* Scientific, Blazie, EuroBraille, you name it...) with a simple common API.

I can't really say that Braille display manufacturers are very cooperative (they are not _at all_). Which is stupid since this project has facilitated the creation of many applications using Braille displays, which in the end helps to sell more Braille displays.

Since those Braille displays are very expensive (usually >5000$), I also created a virtual graphical Braille display with Gtk+ which can be used by developers to test how there application would be rendered in Braille. So feel free to test it...

Interesting projects, but...

[Ingolfke]

I can't seem to find any screenshots. Has anyone seen the screenshots?

My experiences in this field

[rawket.scientist]

I have some undergrad computer science background, and work for a blind man. I've been trying to help make his computer more accessible to him.

By way of background, he uses his computer (a Windows box on a university network) for e-mail and writing scholarly articles. His screen reading program is JAWS. He would like to use the internet more, but he's frustrated by a combination of factors, some JAWS-related and some related to inaccessible design. The boss is highly intelligent and has been blind from birth. He's very used to other adaptive technologies like Braille or Open Book (a program that uses a scanner to translate print to speech), but he's a complete computing novice.

It's worth noting that the computing problems he has are completely different from those that I would face if I were blinded tomorrow. A lot of his trouble is just related to a poor grasp of fundamentals. It's very difficult to explain the difference between opening a folder on the desktop and browsing the contents of a folder through an application's Open dialog box. Compounding the problem is that the boss is ten years late to the party, and most of his sighted assistants take things like right clicking on objects for granted.

On the other hand, he has some advantages that a recently blinded person would not. The boss is used to taking in huge volumes of information through his ears, and can absorb synthesized speech at a dizzying rate. He also reads and types Braille grade 2, and gets a lot of use from his Braille Lite, which is like a PDA for Braille users, with a special Braille keyboard and a refreshable display of dots. It's great for him, but the chord- and abbreviation-based systems of Grade 2 Braille, take a long time to learn, and wouldn't be useful to your grandmother, who's losing her vision to diabetes.

So here are some of the problems I've encountered in my time with the boss:

• Crappy mouse-only interfaces
• Memorization. Using JAWS fluently requires that a blind person memorize dozens, if not hundreds, of keystrokes and interface layouts. Sighted users have tool tips and hot key indicators and a hundred other visual affordances to prod us along the path. If JAWS can provide an equivalent, I've yet to find it. Coming back to a program you haven't used in a year pretty much means starting from scratch with learning the interface.
• Portability. JAWS is insanely expensive and can't be installed on just any computer on the network. It tethers the user to one station.
• Compatability. The university uses Novell Netware in the boss's college. Since the log-in screen would load before JAWS, it's not accessible, which means that the boss can't use Novell. This in turn means that he can't use the public printers, the calendar tools, and other programs that are doled out through those gates. On the plus side, it means that he's the only guy in the college with administrator privileges and a more-or-less unencumbered net connection :)
• Flash and Java-based menus and advertisements online that break JAWS
• JAWS assumes a certain level of basic computer competence in its users. Its help menus will tell you everything you never wanted to know about setting the properties of a folder. But we forget that there are computing lessons even more basic like that. For example, the boss still doesn't really know why he would want to use folders, or what the difference is between the Word application and a Word document.
• Last and certainly not the least are the education gaps between the boss and his assistants. The boss's scholarly field is not technical in nature, and his students reflect that. I'm the first assistant he's had who understood keystrokes as a way of navigating interfaces; everyone else bogs down saying "Well normally, I would just click that. I don't know any way to do

Re:One way to cut costs - outsourcing!

[Klaus Obermeyer]

It's no secret that many of the Americans calling Indian support centers know more than the supposed "support" personal they are calling.

That's why it's perfect that these Americans have lost their Job, now they finally have the time to talk the Indians through their old job!

my experience

[greenrom]

About 5 years ago back when I was in school, I worked on a project sponsored by the state of washington to develop an improved text telephone for people who are both deaf and blind. To help us develop a prototype, we met with a lot of deaf-blind people and people who assisted them. I strongly encourage you to meet with people who are deaf-blind before trying to design a device that would meet their needs. Here are a few things I learned while working on the project.

* Most deaf-blind people are not born deaf and blind. Most deaf-blind people are born either deaf or blind and then lose the other sense as they get older. I don't have any numbers to back this up, but the overwhelming majority of deaf-blind people we met suffered from Usher Syndrome. Most people who have this genetic disorder are born deaf and then start going blind around 30. From my experience, most of these people only start to learn braille when they start to experience vision loss. It can be difficult for adults to learn braille. As a result, it's very useful to have a display suitable for people with reduced vision in addition to a braille display to ease the transition as these people are learning braille.

* Deaf-blind people communicate by using sign language and feeling the speaker's hand as things are signed. However, deaf-blind people often need to communicate with people who do not know sign langage (a repairman, neighbor, paramedics, etc). To do this, they often rely on a computer, text telephone, or similar device that allows them to type messages back and forth to the person they need to communicate with. It's important to keep secondary uses like this in mind when designing a product.

* For the severely vision impaired, we found that LCD and CRT based displays were not easy to read. The displays that were easiest to read had about 2-inch letters and emitted a bright light in the blue-green area of the color spectrum. Scrolling text is very difficult to read for people with severe vision impairments. Many of the people we worked with had severe tunnel vision that made multi-line displays confusing. These people would keep their head a couple of inches from the screen and move their head to scan across the line. We ended up using a large vacuum-flourescent display that would advance a line at a time at the user's control. We also found that having interchangeable color filters for the display made it useful for more people since everyone's vision loss is different.

* We found that multi-line braille displays were confusing for a lot of people and didn't serve much purpose since you can only read one line at a time. The design we used that people seemed to like the best was a 20 character braille display with a button on the left-hand side for scrolling up one line and a button on the right hand side for scrolling down one line.

* Some people we met were good typists, but most people tended to one-finger it. That's bad enough if you can see, but it's even worse for blind people. We had braille lettering on our keys, and we found that a lot of people would scan the keys with their fingers to locate the letter they wanted to type. However, they often ended up inadvertantly pressing these keys as they were scanning the braille. Using very stiff key switches greatly reduced this problem.

* Placement of keys, power switches, etc is very important. Things need to be easy to locate but difficult to inadvertantly hit. This is harder than it sounds and you probably won't know the mistakes in the layout until you ask a blind person to use it.

* Before I worked on this project, I had no idea that there are two different kinds of braille: 6-dot and 8-dot. Far more people are familiar with 6-dot since it's what's used for most books, but 6-dot has a lot of limitations. The only symbols you get are period and comma. Numbers are letters that are prefixed by the number symbol. As you can see, there can be some information l