Slashdot Mirror
Interfaces For The Handicapped?
"Pete, a long time family friend and founder of Pedal with Pete, was born with cerebral palsy (check the Web site for more information). This greatly inhibits his motor control. He enjoys bike riding but last June, was in serious bike accident (once again, more info on the Web site) and this only reduced his motor control. The man has incredible spirit and wants to live on his own, though he has to have home care nurses stop by every day to help him perform most of his daily activities.
He used to do lots of work on his computer pecking away with sticks strapped to both hands. Since his accident he has virtually lost control of one side of his body and has seriously reduced control on the other side, so this is no longer possible.
My current line of thinking is that if we can get him to be able to control a computer, we might be able to automate many of the other parts of his life so he doesn't have to rely on so much help from other people. While I definitely have several ideas on how to go about this, I'ld really like some other input. Does anyone have any ideas or new products they can point me to?"
We've already had a discussion about Voice Recognition in one of the first Ask Slashdots but that article is from '98 and I expect there has been a lot more progress in this areas since then. I may have to run a follow-up post to this if folks are interested.
I work at a community college where we have a few blind/visually impaired students. With the aid of a piece of software called Jaws, they can use a computer about as efficiently as any of us. It reads aloud the title of windows as your mouse moves over them, reads aloud text, adds some more shortcut keys, and really makes the monitor uneccessary...almost. I'm going to download the demo and see for myself. I've seen a blind man demo it as he trained some of the faculty and the results suprised me.
We also have zooming software for those who can still see a little. Many of these tools can be very usefull even if we aren't handicapped.
How am I supposed to hallucinate with all these swirling colors distracting me?
New Mobility Mag. had a write up on Dragon Dictate in their latest issue. Only parts are available online and I've requested a full copy be made available to /. The article really goes a long way in giving a "disabled" spin on this. I've pasted what of the article I could grab online at the bottom.
Ruthie - Compu-babe w/Multiple Sclerosis
Is the Dragon Draggin'? The Rise and Fall of DragonSystems
By Ben Mattlin
Last spring, I replaced my 3-year-old computer, which included a 5-year-old version of DragonDictate, the voice recognition program that allows you to talk to your computer instead of typing on the keyboard or using a mouse. What a miracle it had been when I first got it! I'm a functional quad--have little use of my arms and no use of my legs--and I'd been unable to find any truly effective way of interacting with a computer until DragonDictate came along.
I first saw a prototype demonstrated at a university laboratory in the 1980s, when it worked slower than catsup pouring out of a Heinz bottle and the complete setup ran about $20,000. Yet by 1991, a faster and more accurate DragonDictate was available for less than $5,000; thanks to the Department of Vocational Rehab, I was soon a grateful customer.
I had the greatest respect for the company itself, DragonSystems, a small Newton, Mass.-based software developer that had practically invented voice recognition technology. What a team they had working there!
Nowadays you can pick up a starter version of the software at any Best Buy store for about $20. But as DragonSystems has gone mainstream, it seems to have sold its soul. Service is no better or smarter than what you'd expect at McDonald's, and the program itself has become about as accessible and welcoming as a flight of stairs ...
... is http://www.hj.com/JFW/JFW.html. An evaluation version is available for download.
ian.
ian
Designing an i/o device for a physically impaired person is a difficult task. I've been thinking about this for quit a while and to get any where you have to understand the ideas behind the current way of doing things.
A computer, for most of the desktops, has two primary input devices, the keyboard and the mouse.
The keyboard is a conglomeration of many one dimensional buttons. A button is either on or off.
The mouse is on a 2 dimensional plane with 1 to 3 one dimensional buttons.
This combination has been very effective for creating input for a computer. In fact it can be argued that a keyboard is more productive than a mouse due to the time/movement factor. I can, with a couple of key strokes, do X but to use the mouse I have to move my hand and move the mouse to get the same thing done in 10x the amount of time. So following that line of logic a 1 dimensional input device is more efficient than a 2 dimensional input device. For the sake of efficiency it would be best to use a device with many 1 dimensional inputs, buttons, for a disabled person.
The most successful input device for a physically impaired person, IMHO, is the 2 paddles that Stephen Hawking uses for communication. It is very slow and laborious but with a little patience he is able to get quite a bit done. From what I understand is that there are paddles, basically buttons, which provide 2 one dimensional inputs.
Taken that this is a 2 bit system and a keyboard is based on ASCII, an 8 bit system (7 of them useable) you have 128 combinations. A 2 bit system has 4 so 128/4 = 32. So to get the same amount of work done out of the paddles you need to spend 32 times the amount of time.
Now that all of that is understood, and probably poorly. A new form of input needs to be invented based on the specific needs of the specific person. You need to determine how many one dimensional planes can he interface with then design a system around that. If he has use of his feet they can be used in the interface also. In fact if enough 1 dimensional planes are identified and specifically tailored to his need he may be more productive on a computer than the better than average computer user.
If at first you don't succeed, skydiving is not for you.
FYI, I thought the same thing you did, until I checked out the site a little more. Pretty cool stuff that they do.
Check out IBM's Special Needs Systems Guidelines. It has info on software accessibility, hardware accessibility, web accessibility, etc, with lots of good info and links to other resources as well.
Perhaps not the example you were looking for..
With the help of his brother he has built a pretty cool system (beats either of my two) and has been using a device called an Adap2U Keyboard Emulator System which allows him to use Sip'n'Puff through a straw to "type" using Morse Code.
He also has a page on his site for disABILITY information and resources which seems to be an Ultimate collection of links.
Voice recognition is a relatively wide bandwidth source of control if you can speak. In the case of an individual that cannot speak but has good control of their eyes cornea tracking is a solution. You stare at a letter or command for a few milliseconds to toggle it then move your gaze to the next letter. This is faster than sticks but still gets tiring after awhile. I was wondering about an accurate GPS device for the blind. It would make finding bus stops a lot easier, as well as checking the progress you are making towards your destination etc. Now that the accuracy has increased for GPS maybe somebody will build one that is more usefull.
There is a company here in the DC area called Anthrotronix that does a lot of great work in this area. There was an article in The Post about them today. Check it out. The potentials for the future of handicapped access to technology are great.
My point is this - I'm sure it wouldn't be too hard to build a small (and cheap - thats why we're using it :-) system that allows device control using the computer and the TINI as a bridge to the 1-Wire network.
The 1-Wire network consists of a dasiy chain of devices using standard RJ11 networking gear. So this would be relatively cheap and easy to instal. And devices can talk to it using cheap 1-Wire interface modules (2 way for approx $10-$15 US) from Point Six Inc.
The TINI can also be used to send email - so maybe even some sort of alarm. If a button is pressed an email goes out asking for help.
It should be: http://www.x10.org
- Home Director, by Home Director, Inc. is a nifty set of tools much like X10's.
- HomeToys Home Automation Links many links to various products having to do with home automation -- even has some things on voice recognition.
- Linux Home Automation
- DMOZ's Home Automation Section
- There are many more, so I'll just point you to Google (Home Automation Search)
That's all, not much, but a startThere is a product called Words+ that is worth checking out: http://www.words-plus.com/ It is what Stephen Hawking uses Neil
They sold most of it in 1997. For more information you can go to http://www.cyberpatrol.com/handiw/def ault.htm.
It used to include screen magnifiers, morsecode to ascii, key predictors, word completion, remote control of X10 devices.
Another option might be touch screen. It might be easier to touch than to move a mouse.
Fight Spammers!
The main problem with this idea is that you're probably looking at combining a number of different technologies in addition to speech commands and X10, and there are more of those in total for Windows (ironic because Windows is inherently harder to control and use than just about anything else). And despite X10's recent nauseating push to quasi-porn spycam marketing, they have been doing home control for decades (I remember seeing an entire house controlled from an Apple ][ at an exhibition once).
One of the neatest command speech apps I've seen was an integrated solution for Mac Netscape (Speech Navigator?) that allowed one to say "Navigator Back", "Navigator Scroll Down", "Open link 'Access Technologies'", etc. It worked amazingly well, but some pages were more accessible than others (the Slashdot main page would suck, for instance, since all the important links have the same name - "Read More..."). Nowadays you could probably write custom scripts to do the same thing with a more scriptable browser, like iCab or IE5.
-- Life is short. Forgive quickly. Kiss slowly. ~ Robert Doisneau
One of the great places doing computer interface work for the disabled is Pulsar.org ( http://www.pulsar.org ) which is actually a collection of companies, researchers and other good folks. One of the lead personalities behind it is Dr Dr Dave Warner (PhD, MD) - a VERY dynamic individual with a lot of great ideas. Dave often gets accused of "stunt research" with his application of high (and low) tech in quick demos that dont become products, but most of the work is stuff you can rebuild yourself.
Like, did you know the Velostat(tm) that provides static shielding for chips and boards makes a great pressure sensor? Put some leads on it and push - you get a resistance change proportional to pressure. Makes a good cheap switch. The pulsar folks have demoed an RC car controlled by a pad placed in a shoe.
They (http://mindtel.com side of Pulsar)have a nice interface box called the TNG (Totally Neat Gadget). It provides A/D and Digital IO via a serial port. Primary progamming uses NeatTools - a true visual programming environment (connect the boxes). Very powerful for very little effort.
As for X10 - BAH! They spam way too much and their service is terrible. However, the products can go a long way towards automating a home with remote and PC control. If you watch their site, you can get some decent deals, but DO NOT USE THEIR E-COMMERCE side for any specials advertised in email - the site doesnt know about the email specials. Use the phone instead.
As for speech recog - its not really all that good, but it can be better than nothing!
As for brain interfaces - they are getting better but are still very expensive. Muscle sensors can be used nicely for therapy feedback devices. One pulsar.org app used them to drive MIDI synths and nintendos. Kids that would not do their physical therapy exercises before could not be dragged away from the drums/games.
Eye trackers are also pretty expensive and poor. I tried several at the Cal State Northridge Technology and Persons with Disabilities conference and was disappointed. IF you hold your head perfectly still, the video based eye trackers can do fairly well, but if you move around, they lose the pupil. Eyeglasses and contacts also screw up the tracking.
I'm the author of a package called GTKeyboard, located at http://opop.nols.com/gtkeyboard.html. It's an on-screen keyboard that allows users to type using only the mouse. The keystrokes can then be sent either to the regular text editing area in the application, or even to other applications, (such as typing shell commands into an rxvt or URLs into netscape and so on)
It comes with layouts for a bunch of different languages/keyboard styles.
I hope this will be useful to you or other people. I've gotten email from a bunch of people in the past that said they found it quite useful - among those were a stroke victim using linux, several people with ALS (Lou Gherig's disease) and cerebral palsy.
The good part is that if you can use a mouse and you have X, you can do anything a keyboard can do. Of course it's a bit slower than using the keyboard, (we've got some new features coming soon to address that) but one mouse and three buttons is never going to be able to compete with 10 fingers for speed.
Hope this helps.
-- Truth goes out the door when rumor comes innuendo. -- Groucho Marx
I think out of all the things out there, this has to be the coolest I've seen.
-- Dr. Eldarion --
It's not what it is, it's something else.
A little thought experiment--insert the German word that I'm thinking of; begins with a G. Don't remember it, anyway.
:-) I'll write about some speech involving tech later. Presume now the patient can't talk well.
:-)
:-) Move left side of mouth, cursor goes left. Right side, cursor goes right. Smile, up, kiss, down. Stick your tounge out to click ;-)
I'm going to assume for the moment that speech is not available. Why? Because
Thanks to the Perception midterm, I can tell you that the human eye is a busy little organ, saccading at rates we completely fail to perceive due to our ability to unnoticingly shift, tremor, and saccade our eye at high speeds.
The vaunted fovea that contains the vast majority of your visual acuity covers about a degree of sight. In other words, you're seeing little more than six to ten letter of the text you're looking at. Your eye is moving THAT FAST.
So in my mind, it's pretty fruitless to use eye tracking--the eye is just not a stable enough entity. Way too much signal in the noise. It's hard enough to track the poor thing *without* its design completely contrary to what you're trying to use it for--namely, to figure out what the eye is looking at.
However, as unstable as the eye generally is, the head position is actually pretty solid. A simple sensor that took samples of the image coming out of a monitor at a high enough speed to catch an exact pixel being drawn could correlate head position with gaze position. Assuming one could get this into a feedback loop(in other words, the patient could see where the computer thought he was "pointing" and could move his head to compensate for where he wanted the cursor to reside) would make for a reasonable interface to a number of applications.
Most interestingly, the Quikwriting scheme offered for the Palm Pilot, which offers an entirely continuous motion based alphabet for writing, would work quite well in such an environment. This deserves more investigation than anything else I've written on the topic thus far.
Of course, the disadvantage of a sensor on the user is that now the user is wearing something. This introduces moving parts. Eye tracking systems could be replaced easily by a system that doesn't try to track an eye but rather some specific element of a face over time, with quick recalibrations--the idea is to register movements when given some cue to pay attention, for a *short period of time* have a high quality system track a given point on the face in order to determine where to move a cursor(perhaps using a iterative sensitivity function whereby a user can specify "ok, I want to be pointing at something in this 256x256 block; recenter in three seconds and move slower"), and then disappear as the user takes in the new data.
But what to use for specifiers? While, again, the eye isn't something perfect, unnatural blink patterns may be. Two dark-bright regions disappearing and then reappearing twice in unison(or perhaps staying gone for a two second period, with a tone providing feedback to the user as to how long they've sent a blink signal), or possibly wink detection systems, would be far easier to design than a pupil tracker.
I can't stress how much I'd imagine having the ability to specify that the system needs to recalibrate would be. In my mind, the system should recalibrate as often as possible, detect differences from the initial state for short bursts of time, and then disappear into the background. Extended Blink Initialization is also nice for this aspect--wherever two large white objects with dark interiors open up, there's your eyes
But there's another area where there's an incredible amount of control--the mouth! There's an amazing amount of non-natural(and ridiculously silly looking) things we can do with our mouths that we don't generally. Folks, I call that a signal
What if speech is available, but only partially? (Does this happen? I dunno.) But I could imagine that tonal patterns--a low pitch to a high pitch, a high-low-high, etc., would be ridiculously easy to extract via FFT.
Again, whatever is done, the UI needs to feedback to the user what it thinks it saw, and needs to be able to be built such that both the user and the environment can learn when its doing something wrong.
Anyway, I've got two midterms, and I just spent way too long on this...somebody please contact me if this turned out useful. And, please, SOMEBODY look into Quikwriting as a useful scheme for quadraplegics?
Yours Truly,
Dan Kaminsky
DoxPara Research
http://www.doxpara.com