Slashdot Mirror
Designing an OS for Blind/Deaf Users?
Sushant Bhatia asks: "I work for a team developing technology for individuals who are blind and I have had the opportunity to use some screen reading software and while there have been leaps of progress it is still quite tedious to use, and not at all user friendly. One of my managers recently posed an interesting question for me: 'How would you design an OS from scratch that would target individuals who are blind and/or deaf?' What about inputs such as keyboards or refreshable Braille devices?"
Well, at least for BlindOS you don't have to worry about writing video drivers...
---- El diablo esta en mis pantalones! Mire, mire!
about the OS, but for deaf/blind people, the hw platform should be a pinball machine.
"I'd rather be a lightning rod than a seismometer." -Ken Kesey
I would start with the US government's recommendations for app development for people with disabilities. Most apps written for the Fed have to be section 508 compliant, which helps ensure that they'll work with screen readers. Keep in mind though that from my (limited) experience, 508 compliance is more than an art than a science - you know, you get something that's kinda sorta 508 compliant.
http://www.section508.gov/
Tristan Yates
While not part of an OS designed "from scratch" for vision-impaired individuals, Apple VoiceOver is the first such functionality of its kind to be included for free with a commercial operating system. It's a fully integrated screenreader and accessibility interface for Mac OS X, and is tightly integrated with both the operating system and its APIs, and is extensively supported in several common applications.
As for hearing-impaired individuals, the task is much easier, as the primary interface to a computer is already visual. However, visual alerts and features that would correspond with otherwise audio-only events have also been integrated by Apple in Universal Access.
In addition, Universal Access includes features that assist individuals with motor impairments as well.
While it may be an interesting and informative exercise to think about the types of things you'd do if you were going to "build it from scratch", it might be more productive to think about how these capabilities could be added to existing commodity operating systems, such that the technology can continue to be affordable and easily supported.
Blind:
1. Use a device that creates a topographical image on a pad based on the screen color values. This would allow the blind user to "feel" his way around the screen. More advanced versions could allow the user to use his hands as a mouse, by accepting a certain amount of pressure as a click.
2. Controls on the device should allow the user to "zoom in" on particular areas. This would help the user more easily find toolbar buttons and the like.
3. Replace the system fonts with braille fonts. (I'm uncertain as to how one might add bold or italics for emphasis, but I'm sure a system can be devised. Perhaps extra bumps outside the normal character area?)
4. No right clicks. Right clicking is more of functionality for advanced users anyway. Mac OS X can get along without it, so other OSes should be able to do so as well.
5. "Selected" items should actually invert in the control device. This would allow the person to easily understand what (s)he has selected at the moment.
6. Standard controls such as checkboxes, radio buttons, and the like should be skinned to be more "feel" friendly. i.e. Simple invertable boxes would work better for checkboxes and radio buttons than our current iconic forms.
7. One handed brail keyboard? It's just a thought, but if the blind could be taught to use a one hand keyboard, they could read and type at the same time.
Deaf: What are some of the actual challenges facing a deaf computer user? Computers are primarily visual, and tend to suffer little with the loss of sound. (Unless I'm listening to music, I usually keep my machines muted.) My only thought is that the standard issues of movie subtitles apply.
Javascript + Nintendo DSi = DSiCade
blinux.
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
Really if you are going to take only text as input and output is going to be serial text, speech (blind but not deaf), braille CLI is the way to go.
This does not prevent you from multi tasking BTW, it simply means that you need to work within a well defined context.
Nothing new to invent.
EA David Gardner -"... but the consumers have proven that actually what they want is fun."
I've seen a blind guy type over 100wpm, no errors. also, his screen reader sounds like a mad speak&spell on crack, it's so ridiculously fast. Much faster, in fact, than most sighted people can read.
$
this was in '97. one bit of advice i can give is to make it so blind AND sighted people can use it. this is important because often the blind user will need to be taught to use it, and another blind user isnt always available/ideal for teaching another blind user (depending).
What the hell does an OS have to do with who's using it?
An OS should be a standard toolset for applications to be built on. Memory management, driver management, and application management is what an OS should do... it should worry about dividing hardware resources up among it's processes, it should worry about managing drivers... it should not have anything to do with the user interface that runs on it. This question was obviously posed by someone who hasn't the first clue about what an OS actually is.
---
Programming is like sex... Make one mistake and support it the rest of your life.
My four year old daughter is legally blind, but she is learning how to use a regular computer just fine. It's amazing to watch. In the mean time, I'm working on having the government buy a really large plasma display to 'facilitate' my daughter's learning environment. :)
There already exists an OS for the dumb. It's called Windows.
*rimshot*
All movements for social change begin as missions, evolve into businesses, and end up as rackets.
I think this is one of those cases where it isn't mere pedantry to point out that the OS and the shell/interface are two very different things and that what you want is an interface designed from scratch for the blind.
You can use any already existing decent OS as the base.
You do not have to reinvent the wheel to invent the wheelchair.
If it were me I think I would start out by trying to scratch build a decent IRC client. What you learn by doing that will teach you things you will need to know about such interfaces before you start out at a lower level.
KFG
>
> Unless your tech support/any other user of the machine isn't blind.
Ah, this is about finding an OS for tech support folks in India. They could be blind, because all the work is done over the phone. Spending five minutes on the phone proves they're deaf. And the quality of the solutions proves that they're dumb.
Ever since I was a young boy,
I took the support call,
From Delhi down to Bangalore,
I must have played them all.
But I ain't seen nothin' like him
In any support-cube hall,
That deaf, dumb and blind tech
Always says "re-install!"
Sits there like a statue,
He's a voicemail machine,
Please to reading from scripts,
Keeps his call queue clean,
Bullshits by intuition,
Never seen him fall,
That deaf, dumb, and blind tech
Always says "re-install!"
Another note: a good Windows utility for completely blind users is Jaws. It is a speech software utility for Windows. (example: to navigate, the software verbally says: 'start', 'programs', accesories', etc. I saw a demonstraton of the software by a blind man, and he kicked ass (kinda like the hacker dude in Sneakers). Anyway, he could navigate way quicker using the keyboard than the typical sighted windows user could.
As the parent article states, "blind and/or deaf". That possible "and" puts an interesting twist on things that blows "sole useful input" right out of the water.
Yes, blind and visually impaired people can use vocal interfaces to use computers. Personally as a legally blind programmer I can't STAND vocal interfaces. They bloat the amount of time it takes me to tab through code by a factor of at least three. So I make do with using the PC at a lower resolution. I'm one of the lucky ones out there who can do this with no problems.
As well there are braille-based systems available for completely blind induviduals and these sometimes also work for deafblind folks. There are also ways to adapt current OS'es for deaf induviduals so that any audial cues are represented onscreen. I'm not familiar of any tools that can process an audio file and create captioning though, which is a shame as that'd rock.
My point boils down to there aren't suitable OS options available for deaf, blind, AND/or deafblind induviduals. As the BBC reported working with technology can be an excessively frustrating situation for deafblind induviduals. In addition when you want to create a solution for deaf, blind, AND/or deafblind induviduals you have the issue of no two snowflakes being the same so to speak. On top of that it has to be easy to use. For those of us who can't/don't want to spend an hour just trying to read the first email in our inbox it has to be fast to use as well. There's also considerations such as multi-format training materials, child-sized input devices, learning curves due to induviduals being used to using other devices, etc. The list goes on.
This, folks is a REAL challenge. One I hope someone can rise to and overcome.
Starkle, starkle, little twink.
Why the hell would I use a window-environment at all?
80xSomething, textmode. The same as the braille-device has. Various consoles, a screenreader in the background which starts reading on a specific key on any console and in any program (be that shell, editor or webbrowser); maybe a hacked up "screen"-program could do that, that would also allow for dozens of virtual terminals without logging in dozens of times. I've seen a *real hacker* work on a vt220 with screen, slrn, bitchx, mutt, dozens of consoles, vim, ssh. Incredibly fast.
In fact, most unices would already do nicely, I suspect, given some nice text-to-speech software and good drivers for braille-hardware.
"The more prohibitions there are, The poorer the people will be" -- Lao Tse
I agree. The challenge in developing an accessible user interface is that every disability has a different set of needs. For example, a person with a hearing impairment might be able to get along just fine on a general operating system - my home computer has a pair of headphones which I basically only use when I'm listening to music.
It seems to me that operating systems today are farthest behind in serving people with visual impairments. It's obvious that a person who is totally or partially blind isn't going to have much fun trying to use a Graphical User Interface.
I think the best solution at the moment is to write software that works within existing operating systems, using the built-in accessibility toolkits. A lot of improvement could be realized in this area alone.
I knew a pair of blind gentlemen who worked MSN Tech Support, and we set up a computer for the two of them to learn MSN Explorer with JAWS piped to speakers so they could both listen together.
The experience left me both in awe of their ability to hear all sorts of detail and in disgust at the lack of accessibility. The the custom interface was made out of poorly named images. One particularly useless one (Image 14, IIRC) was the minimize, maximize, and close buttons, all together. This brought me to my thoughts on a vector-based UI. Imagine the convenience of smooth scalability across different resolution displays...
Anyway, concerns that I can think of are as follows:
1. API
A series of abstracted interface methods should be made available. The categories are pretty simple... User Interface (menus, buttons, inputs), Text (static & editable text), Media (audio, video, pictures)... this is all off the top of my head, so feel free to improve on it. Each category simply defines a type of data, and then you can build ways to retrieve or interact with it.
2. Registration
I don't care if everyone puts their close button in the same place with the same icon. Visual users can typically locate these things. What they should do is then register that component with the UI Manager. Components could fall into multiple categories, i.e. a graphic on a web page with URLs mapped on it is both a picture and a series of links. Add a "group" indicator or hierarchy to properly collect controls and data together, and I think you have the basic needs covered.
Using these two parts, we should be able to build simple command interfaces. The ability to define the set of controls, displays, and texts for a given interface means you can see them all at once, or hear or feel them in sequence. Your interface can choose to discard or delay extra media (no sudden advert noises on audio interfaces or no need to waste processing time on decoding the video portion of a media file) through a variety of user-adjustable settings.
For visually-impaired individuals, I think the vector-based interface could make huge strides. Right now, you can buy a 21-inch monitor and set it to 800x600, or use a projector, but new laptops are still 1024x768 or higher. I listened in on a Dell Customer Service call from an older gentleman who loved the laptop he purchased, but couldn't read the high-res screen. If a vector-based interface was available that allowed his to change the point size - similar to Mozilla's Ctrl-Scroll size changes - he would have been fine.
I think the key, and the hard part, is getting buy-in on this kind of pervasive detailing of interfaces. HTML/XHTML is a great start for this, because this kind of extension is very easy based on the nesting and pre-defined components on a page.
Interfaces for the disabled or impaired could come in handy for everyone. These same advances are where the "technologies of the future" come from. Until we push the mouse away, we're stuck to the desktop metaphor.
That what was all this school was for... to teach us how to solve our own problems. -- janeowit
My wife uses some of these devices (BrailleNote QT) and the biggest problem is using the wrong underlying operating system. DOS and Linux work just fine when translated to the command line; the WinCE that underlies her machine sucks. Hard. Sure, it gives a bare modicum of device and software (read: Outlook) compatibility, but at the cost of needless hardware overhead, and a special version of the three finger salute.
/. response is "did they contribute for their market research?" To which I say "sod off". The market is fairly small, and should be one with a tighter feedback loop between vendors and users. My email address is non-obfuscated should you need some ideas or contact info for some groups.
Asking 'what about blind and/or deaf' shows a bit of lack of understanding. The best replacement for the blind is speech. The only option for DB is refreshable braille. I suggest you and/or your employer make some contact with DB groups. At a DB camp a couple of weeks ago, some devs and sales reps from some tech outfit (forget which one) displayed their wares. The kneejerk
Finally, expose the API and make some generic libraries available to use to people who like to program. PulseData/HumanWare wants, I belive a dev kit fee. Umm, screw that. Entry level prices on the BrailleNote are ca. $3000, IIRC, so there's plenty of money out there. Not sure about you, but Franklin Scientific, Blazie, PD, etc. are Hardware companies. Let a little bit of "Open Source"ism do some development for you. There is tremendous word of mouth, and if some third party makes some brilliant add on, it will get around the community. Depending on how you license it, you may be able to ship later. (Personally, I would go BSD or LGPL.)
In summation:
1. Don't confuse blind and deaf and deaf-blind. Each is unique with unique needs. An attempt to be all things to all people will either have stratospheric costs or poor quality or both.
2. Ask your customers instead of some random slashbots. They are out there.
3. Build it from the ground up, since you'll likely have bizarro hardware anyway.
4. Make it possible to program for it. It (the DB group) is a tight community, with lots of people looking out for others, so it's not like you'll be helping a competitor. Think of third parties as value added.
5. (Not mentioned above, but kinda goes with 2) Update your freaking website with real, up to date, and complete information. Have a company policy of a real, human reply to all correspondence within x hours, where x72. Even "I got your note and am investigating your concerns" is better than some of what I get from PD from time to time.
And a freebie. My wife never uses the voice prompts on her BN, but I activate them if I have to do tech support (needing an onsite geek is a bad thing, BTW). Try to get a speech synth chip that sounds better than the WOPR or Speak and Spell I had as a kid. Seriously. It's 2005, and every time I hear the voice, I expect to hear "Would you like to play a game?"
Jesus was all right but his disciples were thick and ordinary. -John Lennon
* Most deaf-blind people are not born deaf and blind. Most deaf-blind people are born either deaf or blind and then lose the other sense as they get older. I don't have any numbers to back this up, but the overwhelming majority of deaf-blind people we met suffered from Usher Syndrome. Most people who have this genetic disorder are born deaf and then start going blind around 30. From my experience, most of these people only start to learn braille when they start to experience vision loss. It can be difficult for adults to learn braille. As a result, it's very useful to have a display suitable for people with reduced vision in addition to a braille display to ease the transition as these people are learning braille.
* Deaf-blind people communicate by using sign language and feeling the speaker's hand as things are signed. However, deaf-blind people often need to communicate with people who do not know sign langage (a repairman, neighbor, paramedics, etc). To do this, they often rely on a computer, text telephone, or similar device that allows them to type messages back and forth to the person they need to communicate with. It's important to keep secondary uses like this in mind when designing a product.
* For the severely vision impaired, we found that LCD and CRT based displays were not easy to read. The displays that were easiest to read had about 2-inch letters and emitted a bright light in the blue-green area of the color spectrum. Scrolling text is very difficult to read for people with severe vision impairments. Many of the people we worked with had severe tunnel vision that made multi-line displays confusing. These people would keep their head a couple of inches from the screen and move their head to scan across the line. We ended up using a large vacuum-flourescent display that would advance a line at a time at the user's control. We also found that having interchangeable color filters for the display made it useful for more people since everyone's vision loss is different.
* We found that multi-line braille displays were confusing for a lot of people and didn't serve much purpose since you can only read one line at a time. The design we used that people seemed to like the best was a 20 character braille display with a button on the left-hand side for scrolling up one line and a button on the right hand side for scrolling down one line.
* Some people we met were good typists, but most people tended to one-finger it. That's bad enough if you can see, but it's even worse for blind people. We had braille lettering on our keys, and we found that a lot of people would scan the keys with their fingers to locate the letter they wanted to type. However, they often ended up inadvertantly pressing these keys as they were scanning the braille. Using very stiff key switches greatly reduced this problem.
* Placement of keys, power switches, etc is very important. Things need to be easy to locate but difficult to inadvertantly hit. This is harder than it sounds and you probably won't know the mistakes in the layout until you ask a blind person to use it.
* Before I worked on this project, I had no idea that there are two different kinds of braille: 6-dot and 8-dot. Far more people are familiar with 6-dot since it's what's used for most books, but 6-dot has a lot of limitations. The only symbols you get are period and comma. Numbers are letters that are prefixed by the number symbol. As you can see, there can be some information l