Slashdot Mirror

It's already long known that a lot of the seeing is done in the brain. When someone draws something on your hand or other part of the body you can still "see it" even if you are blindfolded. The resolution is just isn't as good. Humans can learn to see with their tongues: http://www.sciencenews.org/view/feature/id/1946/description/The_Seeing_Tongue

They can also see with sound - either echolocation or pitch vs left-right volume. https://www.youtube.com/watch?v=qLziFMF4DHA
http://www.seeingwithsound.com/

This transplant experiment isn't very useful in my opinion. Yeah it shows that if you grow an eye on a different spot on a tadpole it can sometimes kind of work. But how useful is that? The artificial eye experiments on humans are far more useful.

You see with your brain too. If someone draws a simple picture on your palm or back, you can still "see" it in your mind.

See also: http://discovermagazine.com/2003/jun/feattongue
The brain is able to learn to see whether the picture is generated by touch or sound.

And even with sound there are different ways for seeing,
echolocation: http://www.youtube.com/watch?v=qLziFMF4DHA
and some software that converts images to pitch and left-right volume: http://www.seeingwithsound.com/winvoice.htm

Maybe what they could try is implant a sensory array to baby/young rats and see if they can add a extra video input to rats.

I'll gladly cooperate with Sergey Brin on "Seeing With Your Ears" http://www.seeingwithsound.com/ to "hear" an otherwise inaccessible environment

Another blind side aide: http://www.seeingwithsound.com/

You might be interested in the more advanced "artificial synesthesia" advanced technology:

http://www.seeingwithsound.com/asynesth.htm ...though I am surprised they do it with a square raster. It should really be polar/arranged to the geometry of the human eye. Also they could use some pedestral-based output improvements no doubt.

I'm a bit skeptical about these things because blind people use their hearing so well, and these would mask natural sounds. They might be better for the recently blinded.

Also this Isreali system has been done before by british bodgers.

thevOICe is a computer vision system that can actually run on Android phones.

A similar product already exists. Is this one supposed to be better? http://www.seeingwithsound.com/

There was an earlier version that used a golf ball camera. Vertical was the frequency spectrum -- high for up, and low for down. Horizontal was the time delay from start. Using it was described as like learning a new language. If this is the same thing, the hardware was revised.

http://www.seeingwithsound.com/

How much power requires that pattern recognition? By standards approachs probably a lot, but the approach they seem to use there (like in compare how much fits what they have with thousands of candidate models) could require less, and far better if you use for that hardware that are more adequated for that task.

They already have:
http://www.seeingwithsound.com/

Doesn't work for me, though; just not practical.

I've been in touch with a blind user who is very interested in "the vOICe," an audio vision system. http://www.seeingwithsound.com

This potentially runs on some cellphones, although the hardware support isn't quite there for the realtime mode (no convenient way to interface a head-mounted camera on most phones).

http://www.seeingwithsound.com/midlet.htm

There's also an amazing piece of software that is helping people who are blind, called the vOICe MIDlet. It translates live views from your camera phone (so it can be used on a Nokia N95) into sounds that you hear via the phoneâ(TM)s speaker or headset. So, basically, re-training neural nets, it can enable people to 'see' using sound. It's pretty fascinating stuff. You can read more about it here: http://www.seeingwithsound.com/midlet.htm.

http://www.seeingwithsound.com/

Check out this project. It lets the vision impaired "see" using a set of headphones, a pc (laptop rig) and web cam (head mounted). Check out some of the video demos.. I was able to quickly pick out the windows and doors on the buildings the user was walking past.

I am not vision impaired, and I think using this would probably give me a massive headache, but I could get used to it if it was my only option.

http://www.seeingwithsound.com/

No surgery and apparently it works. What you should see in front of you is converted to sound. Apparently it works great. I've heard a demo on the radio and it really sounds weird. It's different than sonar, which the blind use, in that light levels are converted to sound.

Camera phones are designed to be highly portable and nowadays have enough processing power for dealing with low-res snapshots. You could even make phone calls! So perhaps something along the lines of http://www.seeingwithsound.com/midlet.htm where a regular camera phone is used to give blind users auditory feedback on shapes, colors, edges, etcetera.

Seems there are pros and cons to all approaches, and maybe a hybrid will evolve at some point,
e.g. to compensate for the low resolution and narrow field of view of contemporary implants?

Immediatly, a large range of blindness can be cured by implants, either by putting a CCD array inside the retina or, in case of damage in the optic nerve, a camera can be wired to the visual cortex. Right now, some blind people see ( with a low res, b&w image but see nonetheless) thanks to implants.

link
other link

But yes, with the technology presented in the article, I suppose one could even cure blinds that have a damaged visual cortex.

The vOICe (http://www.seeingwithsound.com/ for that reason pans 64 pixels at a time in 1 or 2 seconds, in sound. Sound processing tends to be well-organized in early blind people as well.

Probably only if you grew up with an implant, see also http://www.seeingwithsound.com and http://www.seeingwithsound.com/retinal.htm for a discussion on retinal implants and options for the early blind.

Probably only if you grew up with an implant, see also http://www.seeingwithsound.com and http://www.seeingwithsound.com/retinal.htm for a discussion on retinal implants and options for the early blind.

In late 2002 this method was up to 68 implanted electrodes (which would be about equal to an 8x8 matrix)

HOWEVER, you need more than 1000 (say 32x32 or 1028) or above for any really useful vision With 8x8 you might recognize one or two ASCII characters. A Face??? Only if it's an emoticon.

Now granted these are implants in the retina and not the visual cortex, but I have seen other claims for retinal implants over the last five years.

Why is this research taking so long to bear fruit? In 1978 progress was limited by the available CPU horsepower to translate images into usable grid stimulation patterns. Now it seems we are stalled out with our ability to put electrodes in organic systems.

Don't get me wrong, I'm not saying this is easy, but why doesn't this stuff scale like Moore's Law with integrated circuits? Given the state of research over a decade ago we should be up to VGA quality arrays of 640x480 by now.

In general prosthetics systems always seem to be on the verge of some "Steve Austin" "Million Dollar Man" arrival and then never makes it. I assure you when we watched Lee Majors in the early '70s wha-na-na-na-na'ing all over the place we assumed such feats would be common place by the year 2000. What the hell happened? Is this just hard like AI, or under-funded and poorly organized?

You may be interested in this newsworthy product called vOICe.

I remember seeing an applet (and Java application) that did something similar way back in '97/'98. It was amazing then (the guys was using undocumented internal Java code to be able to create the audio).

Here's the website of the current incarnation of that application.

So -- the eternal /. comment -- this is *not* new, though it is cool.

Computer + webcam + vOICe = something you can use in the real world, not just computerized maps. This has been around for a few years now.

Another free proggie I've used in the past is the vOICe.

I'm not blind, but I like to make wierd experimental music, and the vOICe is a neat tool for watermarking, steganography, and just plain general wierdness. It's windows only, though (Sorry, team), so I imagine no extra credit for me! :D

Tip: Fractals make really nice noises.

I was also thinking of brain implants, but The vOICe, which is similar to the website in the /. lead, changed my mind.

I think you are glorifying being blind out of PC'ness. I believe a blind with a sensory replacement aid will develop whatever senses they need just fine even if you give them another sense to work with.

I mean, it is fine to be able to perceive something well by touching it, but it is even better if you know where to touch, isn't it ?

I'd also like to give the compliment back to the other guy who said that the idea is stupid. This post is for you, too.

See here.

I'm not exactly blind, I learned about this as a student project. Doesn't seem like much at first, but long time blind users claim that they experience vision-like sensations, some of them mention seeing depth.

The technology doesn't allow reading, but is praised by users for the fact that it doesn't filter information - a video image is transformed to sound in a reversible (after training) way.

And yet the idea is as simple as fork and spoon, requiring shorter training time than learning to read.

A freely available program for Windows at http://www.seeingwithsound.com can create an audio representation of any image (such as from a webcam). It uses panning to indicate the position of brightness in the image, a tone to indicate the vertical position, and volume to indicate the level of brightness. I am blind and have experimented (minimally) with this program. I think that enough clutter and other brightness changes exist in the environment which would make repeatedly identifying objects difficult. It's definitly an interesting project though.

Indeed, OCR still has a long way to go to make it work reliably with live camera views, as is stressed also at the Mobile OCR for the Blind page. It at least gives some proof of concept for others to improve upon. The open software interface makes it easy to try other command-line driven recognition engines. Anyone who knows of some useful recognition engine to plug in there? By the way, complementary to the remote sighted guide idea, there is also mobile camera phone software that gives blind people direct access to their visual environment, while including a talking color identifier. It appears to work on the Nokia 6600 and various other camera phones.

Indeed, OCR still has a long way to go to make it work reliably with live camera views, as is stressed also at the Mobile OCR for the Blind page. It at least gives some proof of concept for others to improve upon. The open software interface makes it easy to try other command-line driven recognition engines. Anyone who knows of some useful recognition engine to plug in there? By the way, complementary to the remote sighted guide idea, there is also mobile camera phone software that gives blind people direct access to their visual environment, while including a talking color identifier. It appears to work on the Nokia 6600 and various other camera phones.

That's right, and the software for seeing with sound is available as The vOICe Learning Edition It can be complemented with verbal feedback from automatic OCR, face and object recognition once reliable vision-based recognition engines become available, but at present it still seems beyond the state of the art in computer vision to do this in typical real-life environments.

Related experimental technology for the blind is also available for free elsewhere ("The vOICe"): Mobile OCR for the blind includes speech recognition and speech synthesis support. Currently the proof-of-concept demonstrator uses the GOCR OCR engine, but other (object?) recognition engines can be easily added. Stereo vision for the blind

Related experimental technology for the blind is also available for free elsewhere ("The vOICe"): Mobile OCR for the blind includes speech recognition and speech synthesis support. Currently the proof-of-concept demonstrator uses the GOCR OCR engine, but other (object?) recognition engines can be easily added. Stereo vision for the blind

Or try for making image sounds The vOICe Learning Edition There some more images reconstructed from sound at the URLs http://www.seeingwithsound.com/vbme6.html http://www.seeingwithsound.com/tocarmov.htm http://www.seeingwithsound.com/aumodel.htm

Here is the Windows equivalent of MetaSynth.

Speaking of records, consider what this technology will do for sound when everyone who attends a concert (or just listens to the radio) is able to play back portions of their augmented memory at will. Whether this is in the form of speakers or jacked in more directly to your neurons doesn't change the basic problem. When technology makes augmented memory available it will be a 'killer ap,' and whole industries will be opposed to it, attempting to cripple it by schemes such as forcing watermarking into content and mandating blocking or degrading of that content in memory augmentation devices.

But do we want to enter the new era intentionally crippling ourselves in service to the profits of old business models? Current research shows that the blind can be enabled to see by transforming visual information into aural patterns. Since the process uses a laptop, what's 'seen' can be saved to disk. Should this data be bowlderized if it includes Mickey Mouse on a TV in the background? We could make an exception for the 'disabled,' but 20 years from now not being 'borged with fully capable equipment will be seen as a disability.

The cleanest thing to do is establish a doctrine that experience belongs to the experiencer, and can be freely shared with others. Anything less than this, and we'll be inviting all sorts of corporate and governmental characters to become literally engaged in censoring what's 'inside' our 'borged heads.
___