Brain Imaging Reveals the Movies In Our Mind

wisebabo sends word that scientists from UC Berkeley have developed a method for scanning brain activity and then constructing video clips that represent what took place in a person's visual cortex (abstract). The technology is obviously quite limited, and "decades" away from any kind of sci-fi-esque thought reading, but it's impressive nonetheless. From the news release: "[Subjects] watched two separate sets of Hollywood movie trailers, while fMRI was used to measure blood flow through the visual cortex, the part of the brain that processes visual information. On the computer, the brain was divided into small, three-dimensional cubes known as volumetric pixels, or 'voxels.' ... The brain activity recorded while subjects viewed the first set of clips was fed into a computer program that learned, second by second, to associate visual patterns in the movie with the corresponding brain activity. Brain activity evoked by the second set of clips was used to test the movie reconstruction algorithm. This was done by feeding 18 million seconds of random YouTube videos into the computer program so that it could predict the brain activity that each film clip would most likely evoke in each subject. Finally, the 100 clips that the computer program decided were most similar to the clip that the subject had probably seen were merged to produce a blurry yet continuous reconstruction of the original movie."

Comments

This is late, House did it

[joaommp]

A couple seasons ago...

Re:This is late, House did it

[joaommp]

yay for /.'s lag that made me think I forgot to press submit...

Recording

[binkzz]

Does this mean I can record my dreams with Scarlett Johansen and Natalie Portman and view them at a later date? Or sell them on ebay?

Re:Recording

[Doc Ruby]

Wake me when I can record on my PC dreams with Scarlett and Natalie, then view them in my dreams when I can fully interact with them.

On second thought, don't wake me then ;).

Re:Recording

[Sockatume]

I suppose that in principle, if you can synthesise an image from a set of neural patterns you can do the reverse. Give the computer your video, and it'll give you the neural activity associated with that video. Just get your brain wired up at a sufficiently fine resolution and let the computer stimulate your visual centres to create the appropriate visual pattern. (Of course this is probably so coarse-grained that you'd just wind up recreating the "sex" scene in Demolition Man.)

Re:Recording

[twistedsymphony]

Watch out the MPAA might sue you for having dreams about their precious content.

Re:Recording

[Doc Ruby]

Well, what this system is doing is selecting from a library of video clips the 100 clips whose brain signatures when viewed were closest to the brain signatures sensed later. So the video is lo-fi because of averaging of the images. More signal processing, particularly a stats model for excluding red herrings, will give this system higher specificity in selecting the video to match the sensed signature.

It will take another breakthru to generate a 3D image (more likely than a 2D image, which is really just a derived artifact of the 3D brain activity) directly from the sensed brain activity rather than selecting a correlating video. But that breakthru is at least as likely to occur in the SW/data video realm as in the brain/sensor realm. Because we might be able to use large video libraries, and swatches within their 2D images, as primitives from which to synthesize the 3D visual image, rather than building the recreated images from raw voxels. Which is, I believe, precisely how the brain does it.

What will also come along is reading and stimulating other brain sensory (apperceptory, really) regions that are active when we think we're having a hi-fi memory. Often the details are not remembered, but we "remember" that we are remembering the details, in the "metadata". That level of resynth will require the breakthru of stimulating those brain regions, not just reading mappable sensory regions. But with this research at UC Berkeley I think we are now in the (very long and difficult) phase of "working out the details". We are over the watershed into the new age.

I wonder..

[Mordermi]

If I go watch a movie in the theater, then replay it to my friends later from my mind.. Would that be an illegal bootleg?

Re:I wonder..

[joaommp]

That would be a very cool way to make spoilers.

misleading demo

[Thud457]

The video showing the original source and "output" is misleading.
The output is not synthesized directly from the fMRI data.
Rather, they take a bunch of samples from youtube and try to find a sample that generates the closest match the fMRI data.

Still impressively neat. It's just that they need to more explicitly explain what they're doing.

Re:not mentioned

[vlad30]

Actually the MPAA are investing in the technology so that every time you think of any movie they can charge you

nightmarish

[notgm]

that video validated every single nightmare i've ever had

Re:Really constructed?

[Sockatume]

Its output is generated by interpolating together YouTube clips. The computer has a model of how a video maps onto a pattern of fMRI activity. Given a pattern of fMRI activity, it can attempt to generate an interpolation of YouTube clips that closely recreates that pattern.

Re:Worrisome

[pnewhook]

Conversely, I think this is an AWESOME development.

Just think. Once perfected this could be used in trials to definitively prove innocence or guilt. Massively cut back on the slimeball lawyers and jackasses lying in court to get off.

Re:Worrisome

[AmonTheMetalhead]

That's what they said about flying cars and look.... oh, wait

Re:Hey thanks Slashdot editors!

[samzenpus]

I could be getting paid? We actually do this with a lot of stories, and you're welcome!