Slashdot Mirror
Converting Images Into Sounds for the Blind
prostoalex writes "ACM News Service links to a page on Cornell University Web site that describes a technology developed by researchers to enable blind people to read maps. According to the article, the software package consists of "Java computer code that could translate images into sound, and a rudimentary software program capable of converting pixels of various colors into piano notes of various tones"."
Finally blind people can experience online porn ;-)
-- look sir droids...
goatse?
This has been done before!
Will an input picture of a badger make it play "badger badger mushroom"?
Just noticed that the first three post were about porn ;-)
;-)
There most be better use for this technology, but apperently there arent.
-- look sir droids...
What happen when you goto a site with 100 pop ups?
For FREE NO ADS! 1GB/20GB PHP MySQL With a Control Panel Hosting
that'd work, surely? :)
I imagine that the Slashdot color scheme, when "played" to a blind person, will make some of the American Idol contestants sound good.
ravers would rave about
So now they can read maps, but somehow I would't feel very secure getting in on a car with them. "Keep your eyes on the road!!!" "Okay, if it makes you feel better..."
Visit http://ringbreak.dnd.utwente.nl/~mrjb/growingbettersoftware to download your free copy of the book
I remember seeing a few 'black boxes' (Sparc 20's to be vaguely specific) that were running some fairly interesting algorithms (around the 1997/98 ish era) that would identify logo's from various transmissions, mostly faxes, thus identifying the sending entity.
It was more miss than hit, though I'm sure the recognition software has improved since then, it still relied upon a mathmatic description of the original image, much the same as a voice print.
This was before I defected from Military to civil defence.
Some people seem to have porn on the mind.
News for nerds, that's sure right. =)
It's like FARK.com.
Man, I was converting non-sound files into sound since 1996. I was trying to get my soundcard to work in Linux, and to verify my success I'd:
/boot/vmlinuz | /dev/dsp
bash$ cat
If I heard sound, my soundcard was working. This could presumably be done with *.jpg's as well.
SimTunes and many other works by Toshio Iwai?
A fun drawing program in which tone is determined by pixel color.
like this
"This could be done much faster in my favourite language X"!
Starting countdown to first comment..NOW.
Seriously though, this sounds great. My previous job was at the Swedish national library for the blind/visually disabled. Their lives have gotten a LOT easier with technology, and especially the net, but there are still lots of problems.
The greatest service you can do to them is make sure all web pages you make are HTML 4.01 compliant though. Alt tags for pictures are of course important (even if it just saying "logo"), and screen reader programs are not as forgiving as IE/Mozilla/Firefox et al when it comes to confusing tags.
Being bitter is drinking poison and hoping someone else will die
I didn't RTFA very thoroughly, but would this actually work? Reading with your fingers is already quite hard, but that is compensated by a better sense of touch in a blind person. But constructing a mental image from a series of beeps? Seems very hard to do, at least for fairly complex images like maps.
pfffff
Let me be the first to say I can't wait till they run my /pub/porn/pictures/ through this baby.
--
The last digit of pi is four.
It's an interesting story, but I can see why you are getting modded offtopic =). You should repost it at a more appropriate time so you wont get modded down and more people will see it.
That's easy. /dev/dsp
cat lastmeasure/hello.jpg >
It sound like 'shshshshshshshshshh'.
C-x C-s C-x k
you'll ruin many a slashdotter's midnight snacks
I wonder if they could practice doing it with different voices in each ear? That would be pretty amazing...
Freedom = (Meaningful - Coerced) Choice != (Speech | Beer^2), and sad sock puppets' bad mods avail them naught.
Reversing the application so that music was converted to an image would be interesting as well.
"Oh my g--*urp.*" [sound of test subject running to the bathroom]
"OH SHIT, THERE'S A HORSE IN THE HOSPITAL!"
Sound might be a very important way to convey images, either an additional one to textures, or replacing the textures completely. It may instantly inform about the kind of a surface. Sound might also convey edges, but then there is a problem of detecting edges: it is usually easy if the map is in a vector form, but in the case of general raster images a good edge detector or even a human that would mark the edges might be needed.
Partly offtopic: a free software to convert images to tactile graphics using edge detection and textures: JTactileGraphics. It does not have sound support currently, but one is being added.
WHoah! Hear the hooters on that one!
"I used to have that really cool,funny sig
Brings to mind Coagula, the "Industrial Strength Color-Note Organ", which converts .bmp files to synth sounds.
"echo '(.)(.)' > /dev/audio"
would make, though.
Don't you just dump a bucket of water on blind people to get the same effect?
my best sig is this one.
Were I blind, I'd want my pr0n. So this is highly on-topic.
deliver.
Converting PEL color to tone is a trivial hack.
There's nothing else there.
The tactile tongue imager has a short learning curve. Perhaps the brain could learn 'color' through multisensoral(not necessarily the tongue and not necessarily close in space sense organs) stimulation.
Two senses in two dimensions may be adequate for translation by the brain into a color image.
- X-Y location of PEL (P-Q stimulation)
the X-Y grid may not have to be in the same place on the body for P as it is for Q)
- Phase relationship between P and Q
- relative levels of P and Q stimulation
the senses P and Q need not differ in nature if located in different placs on the body
Sense organ areal density and count would determine the number of PELs and response time would determine the, "damn 60ms response tongue", refresh rate.
Already existed in 2002 ;-)
/* Steinar */
(This comment is of course GPLed.)
O 'Nuff said
BSOD:
Runtime Error. Image goatse cause an illegal operation 0x0000000A IRQL_NOT_LESS_OR_EQUAL
0x00000080 NMI_HARDWARE_FAILURE
ABORT ABORT!
Can't be any worse than http://video.google.com , which reads hastily typed subtitles. It can be amusing. :)
For example:
A snippet of Conan O' Brienhere.
at 20 minutes >> Conan: this is very nice. >> Conan: you might want to boil ebay and sayy Conan wore it. S get our priorities ackson." Ings.
_________
The world doesn't just disappear when you close your eyes, does it?
http://www.fotopasaj.com/displayimage.php?album=la stup&cat=0&pos=4
no one's mentioned Metasynth yet?
It's been doing this for a long time. But I guess because it's Mac only many people won't have heard of it.
And yes it sounds much better than Coagula, but no it's not free.
Albert Hoffman, way back when(1941), discovered something that not only allows you to hear colors, but just about any other funky combination of senses you could imagine. It's called LSD.
Another similar project is BATS here at UNC-Chapel Hill.
I believe it is called braille. And it isn't new.
I am Bennett Haselton! I am Bennett Haselton!
Anyway, some car manufacturer was releazing a new convertible, Renault I think, and had an ad campaign which essentially said that the new car was so fun that even Ray Charles liked it.
So they wanted to film him driving it and singing.
They flew him and the cars and cameras out to Bonneville Salt Flats and discovered something odd.
Ray knew how to drive. He liked driving. Turns out he had an E-Type that he would drive, with his chauffer in the passenger seat giving advice.
What will be the sound of...goatse?
That's and easy one. It will sound hollow.
I hadn't the slightest objection to his spending his time planning massacres for the bourgeoisie... (P.G. Wodehouse)
I wonder if they could practice doing it with different voices in each ear? That would be pretty amazing...
As I understand it, there's a hardwired limit in the human brain that keeps one from processing information coming in from seperate ears, at least without some surgery on the corpus callosum, same as how there was that recent study showing that you can't ignore an angry voice; your brain will still process it and act upon it.
This sig has absolutely no significance and serves only to take up screen space and waste the time of the reader.
http://slashdot.org/article.pl?sid=03/12/23/151423 8
Probably a lot like the Wilhelm.
"Darling, you sound beautiful, and have some awesome sin curves on you, but your voice is crap, and to me that is quite important... but I like the way you let me feel my way around you ;-) haha lots of ladies fall for that one" ;-) I will go to heck for this, or Elbonia
#hostfile 0.0.0.0 primidi.com 0.0.0.0 www.primidi.com 0.0.0.0 radio.weblogs.com
Aphex Twin amongst others, has been using images to make music for a long time already.
This site has more information about the Windowlicker song.
Sample this!
i wonder how that would work: (electronic voice): breast, nipple, nipple, vagina, leg.... edit: damn, someone beat me to it!
A little to the left.
No, a little to the right.
Okay, now forward.
Wait! Stop! Back up a little.
A little to the left....
Hell, we could teach 'em Logo and get more done.
I think the sound would be similar to the sound of one hand clapping.
It's not offtopic, dumbass. It's orthogonal.
...see me now?
Another free proggie I've used in the past is the vOICe.
I'm not blind, but I like to make wierd experimental music, and the vOICe is a neat tool for watermarking, steganography, and just plain general wierdness. It's windows only, though (Sorry, team), so I imagine no extra credit for me!
Tip: Fractals make really nice noises.
We've got one of the Washington DC area up on the wall, with different textures for water, forests, highways, etc. It's about 5 by 5 feet.
There are others that're book sized, of the U.S. and the world. They're pretty cool.
http://www.independentliving.com/prodinfo.asp?numb er=309300&variation=&aitem=4&mitem=5
I didn't think the house band in Hell would play this badly.
Wow, thanks for pointing that out, you self-absorbed 32 year-old nude-salsa-dancing shitfuck!
May the fleas of a thousand camels infest your crotch, and may the sun never set upon your scratching!
After the software starts up, they move their mouse around the screen clicking - this generates audible "marco"'s. Once their mouse is over the map, it returns "polo"'s.
At this point, the user can hold shift and enter "icyhot mode", whereupon their mouse clicks generate audible "warmer" and "colder" and various incarnations of such descriptions, to help them find the route that the computer has generated for them.
Another assistive technology is in the works for assisting users in knowing when an image has fully loaded, so they know they can begin using their map software. This technology is called redlight greenlight, but is having trouble finding non-suicidal testers.
cyn, free software and *nix operating systems enthusiast.
OMG !!a /anime.php?id=612>) ;) ...
If they could implement this soon enough, it would help Cobra (<URL:http://www.animenewsnetwork.com/encyclopedi
to counter the secret move of false king Babel of Sodes sword men race (episode 21)
Surely you mean the sound of one hand fapping.
I can't wait till they run my /pub/porn/pictures/ through this baby.
I read this and my first thought was:
Please don't run porn through a baby!
Cozinha para as massas (e para geeks)
Another picture to sound program is the Enscribe project. It is designed for embedding images or adding visual watermarks into audio files. Very cool stuff but it might not be that useful to the blind. Sort of like listening to the raster scan of a TV signal.
I heard the color green.
It is better to be the hammer than the anvil.
[Johnny and Bough are in a dark tunnel]
;-)
Johnny English: It may be pitch black, but we can still see.
Bough: Can we, sir? How?
Johnny English: The Bedouin monks of the Al Maghreb mountains developed a system of sonic chanting.
Bough: I see, sir.
Johnny English: The sound of their chanting would bounce back off any obstacles, and using their highly tuned ears they could paint a mental picture of the path ahead.
Bough: Brilliant, sir.
Johnny English: However, you must always sing in E-flat.
Johnny English: [singing] Thank you for the music / The songs I'm singing
Bough: Is it working, sir?
Johnny English: Extremely well, thank you, Bough.
Johnny English: [singing] Thanks for all the joy that...
[Johnny hits the tunnel wall]
Johnny English: Ow!
Thanks to imdb.com
Ok, not quite what the article was getting at.
Vip
Would it be "AHHHH!!! OHH FUCK THIS IS GROSS!"
Or maybe the sound of a guy getting hit in the junk with a hammer. That is how I feel whenever I get tricked into clickint it.
.......that would be beige I guess.
I've read of several experiments converting images to pressure arrays attached to one's back, scalp, or tongue (lots of nerves there). You may have a 16 by 16 or 32 by 32 image-to-actuator array. These pressure arrays resemble those pin arrays in science museums and novelty stores, which convert the shape of your hand or face into a shiny metal surface.
Users of these arrays, both blind and sighted, say that after a few days you brain starts automatically "seeing" them as visual images. One guy mentioned in Hawkins book "On Intelligence" could walk through hallways and doors because he could "see" the crude outlines via the pressure array. Braille students report similar magic after a few weeks. The brain starts "seeing" letters and words rather than arrays of dots. Hawkins and other psychologists explain this as the brain's plasticity to re-wire and re-use old circuits.
*pshhhhhhhhh* *kshhhhhhh* *shhhhhh* *boing* *boing* *cashhhhhhhh* *boing* *boing*......
I dunno, but it will have one hell of an echo.
"Yeah, well, Dracula called and he's coming over tonight for you and I said okay."
Are we now talking about Georgi of Star Trek?
What happen when you goto a site with 100 pop ups?
A symphony of bullshit.
The brown noise.
It's easy to convert image from sound but getting the blind people to understand it is another thing.
From Steven Pinker's The Blank Slate, pages 95-96:
Let me turn to the most amazing plasticity of all: the rewired ferrets whose eyes fed their auditory thalamus and cortex and made those areas work like a visual thalamus and cortex. Even here, water is not being turned into wine. Sur and his colleagues noted the redirected input did not change the actual wiring of the auditory brain, only the pattern of synaptic strengths. As a result they found many differences between the co-opted auditory brain and a normal visual brain. The representation of the visual field in the auditory brain was fuzzier and more disorganized, because the tissue is optimized for auditory, not visual, analysis. The map of the visual field, for instance, was far more precise in the left-right direction than in the up-down direction. That is because the left-right direction was mapped onto an axis of the auditory cortex that in normal animals represents different sound frequencies and thus gets inputs from the inner ear that are precisely arranged in order of frequency. But the up-down direction was mapped onto the perpendicular axis of the auditory cortex, which ordinarily gets a mass of inputs of the same frequency. Sur also notes that the connections between the primary auditory cortex and other brain areas for hearing were unchanged by the new input.
So patterns in the input can tune a patch of sensory cortex to mesh with that input, but only within the limits of the wiring already present. Sur suggests that the reason the auditory cortex in the rewired ferrets can process visual information at all is that certain kinds of signal processing may be useful to perform on raw sensory input, whether it is visual, auditory, or tactile:
The suggestion that the auditory cortex is inherently suited to analyze visual input is not far-fetched. I mentioned that frequency (pitch) in hearing behaves a lot like space in vision. The mind treats soundmakers with different pitches as if they were objects at different locations, and it treats jumps in pitch like motions in space. This means that some of the analyses performed on sights may be the same as the analyses performed on sounds, and could be computed, at least in part, by similar kinds of circuitry. Inputs from an ear represent different frequencies; inputs from an eye represent spots at different locations. Neurons in the sensory cortex (both visual and auditory) receive information from a neighborhood of input fibers and extract simple patterns from the. Therefore neurons in the auditory cortex that ordinarily detect rising or falling glides, rich or pure tones, and sounds that come from specific places may, in the rewired ferrets, automatically be capable of detecting lines of specific slants, places, and directions of movement.
Computer + webcam + vOICe = something you can use in the real world, not just computerized maps. This has been around for a few years now.
Not that this wasn't entirely predictable.
I remember an old kids science show on PBS called 321 contact. In one show there was this blind man who aquired the skills to do sonar. He made clicking noises (or used some kind of mechanical clicker) and based on what he heard, could say the rough location and size of an object. He could say if a sign post was ahead or if it was a car. He could tell if where the edge of the sidewalk was. Pretty amazing to see humans do that. He at that time was trying to train other blind kids to do the same thing. I wonder what happened to him.
UNC has an article about a similar project.
a technology developed by researchers to enable blind people to read maps
How about next having a technology that will help sighted but clueless people read maps? So few people nowadays have this skill. We need a tutor program that starts telling people "North is up". :/
Just because it CAN be done, doesn't mean it should!
I tried that link and it onyl bring a lightbulb. I'm interested in learning more about ths map, but perhaps you intended to post a different link.
Urban Detail
Nothing to hear here, please move along...
Will you listen to the curves on her!
If Google really cared they would fix Android Chrome to reflow text, instead of discriminating
and a rudimentary software program capable of converting pixels of various colors into piano notes of various tones
So it's a reverse-Spielburg device then?
Oh, say does that Star-Spangled Banner entwine / The myrtle of Venus with Bacchus's vine?
These cross-sense accessibility techs offer benefits for everyone, not just those of us with biological disabilities. We're all blind when we're on the phone, and this tech might offer a UI to existing content otherwise denied us. "Naturally" blind people might soon be viewed as pioneers in sensory modes in which we all strive for competence, to compete in accessing the mediasphere.
--
make install -not war
I remember seeing an applet (and Java application) that did something similar way back in '97/'98. It was amazing then (the guys was using undocumented internal Java code to be able to create the audio).
/. comment -- this is *not* new, though it is cool.
Here's the website of the current incarnation of that application.
So -- the eternal
This is fairly neat, and somewhat related to one of my side projects.
I've been working on adapting my UPC reading code for series 60 mobile phones to query upcdatabase.com and read the output via a screen reader such as talx. My biggest obstacle has been trying to contact developers of such of programs in order to get information on how to get their programs to speak the text I print on their screen. Another problem is the lack of a web database which maps EAN-13 values to product names, which is necessary for global use.
More information on my project is located here
I wonder what reading sheet music sounds like?
Mommy. What's a karma whore?
RIIIIP!