Japanese Scientists Claim To Reconstruct Images From Brain Data

In a world first, a research group in Kyoto Japan has succeeded in processing and displaying optically received images directly from the human brain. Here's the Japanese press release for good measure. One step closer to broadcasting your dreams? The research is due to be published today in the US scientific journal Neuron

Comments

Predictably

[AltGrendel]

It was a male subject and the image was Hentai.

Dreaming Is A Private Thing

[mcgrew]

The group of researchers at Advanced Telecommunications Research Institute International, including Yukiyasu Kamitani and Yoichi Miyawaki, from its NeuroInformatics Department, said about 100 million images can be read, adding that dreams as well as mental images are likely to be visualized in the future in the same manner.

And once again Isaac Asimov predicted this.

Re:Dreaming Is A Private Thing

[geekoid]

No he did not.
He wrote a story about something like this. People ahve thought about doing this for years.
There is a difference in predicting something, and writing a story.

He also wrote about a bunch of stuff that never happens, and won't likely happen.
I like the mans work, but come on if he gets put any higher on a pedestal he'll be able to touch the moon.

Re:Dreaming Is A Private Thing

[matthewsmalley]

Maybe that's how we'll get a space elevator?

Re:Dreaming Is A Private Thing

[gnick]

He can probably already touch the moon. Don't you know your Vonegut?

Isaac's in heaven now.

I assume that Asimov got to tag the moon on his way by. Best speech opening ever.

Re:Dreaming Is A Private Thing

[jollyreaper]

As to writing about stuff that never happened, THIS never happened - until now. The "hyperdrive" (what Roddenberry renamed "warp drive") was never invented - yet. Roddenberry and his writers were prescient, too. I remember a world without cell phones, flat screen talking computers, self-opening doors, and space shuttles (I remember a world without space travel at all).

Wow. Your UID should have a minus sign in front of it.

Re:Dreaming Is A Private Thing

[gnick]

Sorry - If you don't recognize the quote and have no context, it's not funny at all.

Vonnegut (misspelled in my post above) became honorary president of the American Humanist Association after Isaac Asimov, their former president, passed on. As such, he had the somewhat awkward honor of addressing the Association at their first meeting after losing their president and had to come up with some way to say goodbye to Isaac and start his speech. (If you're unfamiliar with Humanism, it's an entirely human-based religion/philosophy. Its members are largely atheist or agnostic and practice strict scientific skepticism while shunning religious superstitions or unsupported beliefs - Heaven/Hell included. The idea that Asimov, as president of the AHA, would have any literal belief in Heaven would be ludicrous.)

According to Vonnegut, opening his speech with

Isaac's in heaven now.

not only did a great job of breaking the ice in a very awkward situation, but set the entire ball-room laughing out loud.

Re:Dreaming Is A Private Thing

[mcgrew]

I looked it up on wikipedia and found that A Logic Named Joe is posted on the internet, with a link from wikipedia.

The tank is a big buildin' full of all the facts in creation an' all the recorded telecasts that ever was made--an' it's hooked in with all the other tanks all over the country--an' everything you wanna know or see or hear, you punch for it an' you get it. Very convenient. Also it does math for you, an' keeps books, an' acts as consultin' chemist, physicist, astronomer, an' tea-leaf reader, with a "Advice to the Lovelorn" thrown in. The only thing it won't do is tell you exactly what your wife meant when she said, "Oh, you think so, do you?" in that peculiar kinda voice. Logics don't work good on women. Only on things that make sense.

It appears that Leinster beat Asimov to the punch; it's possible, being a science fiction fan before he was a science fiction writer, that Asimov even read "Joe". Wikipedia puts "Joe" at 1946, but Multivac in 1955.

No more lack of artistic skills for me

[Ifandbut]

I have lots of cool images in my head for comics and wallpaper, however I lack the artistic talent to bring those images from my mind to paper/photoshop. Maybe soon I will be able to compensate for my lack of artistic ability.

Re:No more lack of artistic skills for me

[fastest fascist]

Maybe, although I personally find, as someone who has spent a considerable amount of time developing technical skill at drawing and painting, that the process of learning to draw has also considerably altered my aesthetic sensibilities. Drawing is ultimately not that much about knowing how to move your hand just right. In fact, it seems to me it is largely about forgetting about the hand, and concentrating on the form of what you are drawing instead... In any case, I doubt a direct mind-to-picture system would in itself be enough to make anyone an artist. Maybe to reproduce what a person sees in front of them, but to be able to make a picture without exact reference, you're still going to need to know very precisely what you want each detail to look like. I think even with this kind of tech there'd still be a pretty intense learning process involved.

Re:No more lack of artistic skills for me

[lahvak]

I have a memory from my childhood that I can almost recall at will, at almost any time. When I was 6, my father took me on a week long canoeing trip. I remember standing on a road, looking through some trees down to the river onto some cool rapids which we were about to go down. I can see in my mind what seems to be a perfect picture: the side of the road, the wooded slope covered with dry leaves, trees, and the river. It seems that I could just sit down a draw the scene from my memory. The funny thing is, every time I try to focus on some detail, for example when try to identify the trees, or look at a number on a mile-post next to the road, or something like that, the whole picture completely disappears, and I have hard time recalling it again. The details are simply not there at all.

Now if I were to draw the scene, I would undoubtedly substitute some sort of simplified shapes, or maybe just a pattern of shades of green, for the leaves. But you could look at the details of the drawing and see how it was done. You probably would be unable to identify the trees by their leaves, the drawing would not contain that much details, but you would be able to see the way the image is rendered on paper. I cannot do that with the mental image. I believe that in my mind I am able to render the overall image without actually rendering the details at all, not even as some sort of impressionist jumble of colors and shades. If that's the case, transferring this image onto paper would require filling in all the details in some way, which, IMHO, is exactly the hard part of drawing or painting.

Kinda neat, not that exciting though

[Improv]

The visual cortex is one of the more understood areas of the brain, and decoding V1/V2 is low-hanging fruit. To the extent that memory and dreams back-project to these areas, perhaps recording parts of these experiences would be possible.

Making this practical and inexpensive would be quite a practical breakthrough though - imagine being able to imagine something and import it into GIMP from a headband. Doing this through MRI would be impractical unless someone would be able to keep the image stable in their head for long enough for a high resolution scan of the area (and bear the ~$700/hour cost of MRI).

Re:Kinda neat, not that exciting though

[Futile Rhetoric]

Can we "keep images in our heads" at all? When I try to, it is more of a feeling than an image, and it's a fragmentary one at that. Wouldn't it make sense if our imagination worked a lot like our vision, i.e. we can only focus on small bits of the visual field at once, and so would only be able to imagine those pieces and attributes of an image pertinent to our needs or wants?

I'm free-balling here, mind. I can't seem to put coherent, complete images in my head, but others very well might.

Re:Kinda neat, not that exciting though

[geekoid]

And imagine spending the next week try to figure out GIMP to be able to do anything with it~

Re:Kinda neat, not that exciting though

[fastest fascist]

I'd mod you up if I could. It doesn't seem to me that when I imagine something in my head, there is actually a picture being made somewhere in my brain. It's an impression, a sense of shapes, something very fluid and ephemeral. I can, in a way, turn around three-dimensional objects "in my head", but the experience is far from looking at a video of an object turning. A simple dumb read of that kind of thing would probably be very difficult, although a person could perhaps train themselves to solidify their ideas into image form via a neural interface, much as they can ordinarily do with pen and paper, for example.

Re:Kinda neat, not that exciting though

[Pedrito]

[i]The visual cortex is one of the more understood areas of the brain, and decoding V1/V2 is low-hanging fruit.[/i]

Low-hanging fruit? I agree, it's fairly well understood, but given the pre-processing that happens in the retinal ganglion cells, and the kind of data that actually ends up getting to V1 (after being relayed from the LGN), I'm surprised an actual image can be reconstructed from the information. After all, the RGCs tend to pass on things like movement, edges, contrast and color, but it's not even remotely pixel by pixel type data, which is precisely the informaiton that gets passed on to V1.

Since it's coming from an fMRI, there's no way the image can be very detailed. I suspect it will be very low resolution.

No pictures?

[pwnies]

Honestly? Come on now. Saying you can retrieve images from the mind, then not showing said pictures is the same as claiming you've achieved cold fusion without showing any energy for it.

I think this is the first time I can scientifically say, "Pics or it didn't happen."

Re:No pictures?

[Peeet]

It's right there on the Japanese press release page, you can see at the bottom of the image at the top left of the article, they have the before and after of the word "neuron". Here, I'll make it even easier for ya: http://www2.asahi.com/kansai/news/image/OSK200812100099.jpg

This is NOT new

[oroborous]

A Berkeley group has already reported this in Nature using similar methods: http://www.nature.com/nature/journal/v452/n7185/abs/nature06713.html)

World first? I think not!

[Colin Smith]

THEY have been able to do this for decades! Where is your tinfoil hat now? Ha!

This is so cool!

[CreatorOfSmallTruths]

This , if true , will have HUGE implications - we'll be able to see what people THINK. I don't know if you actually grasp the monument dimensions of this. Checking for terrorism, knowing if you are really loved, truth telling machines, like the internet, something like this can level the plain field for a long long time...

Re:This is so cool!

[Omestes]

This , if true , will have HUGE implications - we'll be able to see what people THINK.

Data in V1/V2 does not constitute cognition, those areas constitute pretty much a visual map of data gathered by the eye (roughly). Its doubtful that imagined visuals are even represented in these areas. This, in other words, doesn't provide any insight into thoughts, just what people see.

I admit, though, that this is awesome. If we can read it, we theoretically could write to it, which would allow for direct neural interfaces.

I still wonder

[bigattichouse]

Years ago I was a sign language interpreter (ASL), and after a few years realized that I was thinking in ASL and "visually" instead of the usual auditory monologue... I always wondered if you use a completely different part of the brain to process the language - or if it just gets translated into language concepts before processing... I wonder how long before "telepathic" audio is available.

For the last time...

[philspear]

People read blurby summaries, which don't include the results, the full reasoning, methods, etc, and then act as if it's the fault of the researchers. It's absurd, that's neither the paper nor the direct work of the researchers, it's some non-scientist working for a news source. Read the actual paper, TFA in these cases are rarely any better than TFS.

http://download.cell.com/neuron/pdf/PIIS0896627308009586.pdf

There's the PDF. It does have the very pixelated images. I haven't had time to read through it.

As always, don't complain to me if you don't happen to have a subscription, and not having a subscription is no reason to act as if the results aren't real.

Not impressed

[IronChef]

Dr. Walter Bishop (Cambridge) was doing this in the '70s.

Feedback Loop?

[drpentode]

Would looking at the image your brain is generating at the same time you are generating it create a feedback loop much like holding a microphone too close to a speaker?

Re:Feedback Loop?

[bridgeco]

Would looking at the image your brain is generating at the same time you are generating it create a feedback loop much like holding a microphone too close to a speaker?

Malkovich Malkovich Malkovich Malkovich Malkovich?

Nope, dreams would just be noise

[Dr. Manhattan]

From what I can gather, they're pulling in rather low-level data - essentially 'listening in' on the very lowest level of pattern-recognition that's applied to the data coming in from the optic nerve. That's certainly interesting, but a whole lot more processing happens at higher levels before you 'see' anything. (C.f. people who lose sight early on due to eye problems, and have sight restored later - their brains can't do much with the information at first.)

Dreams appear to be based on the 'noise' coming in, but a lot of interpretation is applied (and without imposed constraints of consistency or logic). A common game/prank involves people asking yes/no questions about an alleged dream, but the answers they get are based on some simple scheme like "yes if the last word in the question they ask ends in a consonant". Surprisingly detailed 'stories' get constructed... by the person asking the questions. (Here's what appears to be an online version.) Actual dreams seem to be built in an analogous way, with the subconscious 'asking questions' of the senses (which are just feeding in 'static') and weaving an experience out of them.

I'd guess that 'eavesdropping' on dreams via this means would only get the kind of swirling colors and such you 'see' when you close your eyes.

Re:Pics or it didn't happen

[philspear]

Maybe this image will not require a subscription, although I suspect it will.

http://www.sciencedirect.com/cache/MiamiImageURL/B6WSS-4V4113M-P-7/0?wchp=dGLbVtz-zSkzk

On the off chance it does, keep in mind this is not the full article. Critiques along the lines of "this doesn't prove anything," or "They should have done X" are premature if you haven't read the full (journal) article. If you thought of it, they probably covered that in the article you're not willing to pay for.

TLJ

[CannonballHead]

Reminds me of Dreamfall: The Longest Journey.

Yes, it is

[philspear]

Didn't read the full article, but from the abstract

We show that these receptive-field models make it possible to identify, from a large set of completely novel natural images, which specific image was seen by an observer...

Our results suggest that it may soon be possible to reconstruct a picture of a person's visual experience from measurements of brain activity alone.

The article you linked to seems to only be able to tell which object a person saw from their fMRI. I believe it required established measurements too, IE "this part of the brain lights up when they see a face. In blind studies, that part of the brain lit up, so they must have seen a face."

Whether it required a calibration for each individual or not, no image reconstruction was done: it's not the same thing at all.

Yet Another Unnew Result

[DynaSoar]

The primary visual cortex (V1) has already been shown to be retinotopic. What's being seen can be mapped directly from the cortex. It's crude and low-res, but it works.

20 years ago a researcher working with Karl Pribram at Radford University was able to detect signals from small cellular assemblies of the visual cortex that represented a particular shape being viewed without mapping the entire shape from V1.

In both these, the images were received directly from the brain. In both they were digitally processed and presented. In all three what was retrieved was not an image, but was a pattern of neural electrical activity that they had already determined represented a particular visual field. They could not (in keeping with the /. tendency to represent reality with fiction) for instance, retrieve the third frame of a series of images that had been briefly presesnted. They would have had to show the image for some time that record EEG from the appropriate areas for long enough that they could get a good correlation when showing it a second time.

Does it get lonely up in your crystal tower?

[ovu]

"The current accomplishment is low hanging fruit and therefore uninteresting. Surprising, really, that they found funding for such an unnecessary demonstration at all! By commercializing this technology, it would become sufficiently interesting to deserve my royal approval."

Belittling humanity's incremental advancement as if you're a third party, how's that working out for you?

I think it's tremendously exciting. Thanks for the buzzkill though, it reminds me to get off the computer and interact with people of my choosing.

Some great potential here

[Drakkenmensch]

If scientists can understand perfectly how the brain translates images into neural signals, it opens up the possibility to build full cybernetic eye replacements, even if the nerve tissue is damaged and non-functional. The medical applications are mind-boggling.

Also, I'm reminded of the interrogation device from the movie Barb Wire, the one that pulls out images from your brain whether you want it to or not...

The Opposite

[Clandestine_Blaze]

I wonder if the process can be reversed, and images can be fed into the brain to create a dream sequence? Will people who really hate their reality use this as an escape and never try to wake up again?

Cool story!