A 3-D View of the Brain

Jamie found a nifty story about Researchers at Thomas Jefferson University Hospital coming up with new 3D Brain Imaging Software. The interesting bit is that it merges data from MRIs as well as various other types of brain scans to create a single visualization for your noodle.

Comments

Check out the 'MultiMedia'

[Silver Sloth]

As well as TFA there's a 'Multimedia' link which give much more info - as well as having some pretty pictures.

Re:Check out the 'MultiMedia'

[morgan_greywolf]

OMG, tt's yarn! The brain is full of yarn!

That will be $213,134.56, please.

[xxxJonBoyxxx]

There's your referral. That will be $213,134.56, please.

Matching images.

[iknownuttin]

The interesting bit is that it merges data from MRIs as well as various other types of brain scans to create a single visualization for your noodle.

FTFA:...including conventional magnetic resonance imaging (MRI), functional MRI (fMRI), and diffusion-tensor imaging (DTI).

They're all forms of MRI. Unless there's a MRI machine that can do it all, it would seem to me that you would have to have the patient go from one machine to another. I'm curious how they match up the structures exactly from one scan to another. Do they use a bunch of points on differing brain structures and then the software matches those up?

Re:Matching images.

[tOaOMiB]

All MRI machines can do it all; they are just different programs you feed the machines to get different images. Unfortunately, the images at the end still have to be lined up. This is typically done by allowing the brains to rotate in 3 dimensions until the registration maximizes some function; for example, the mutual information between the two images. See the package fsl (http://www.fmrib.ox.ac.uk/fsl/) for some great fMRI analysis tools, including FLIRT for aligning brains (of multiple patients, or one patient's fMRI scan to MRI scan).

Re:Matching images.

[Anonymous Coward]

Fortunately, most brains (unlike arms and legs) aren't in the habit of moving around a lot during an MRI scan, so they're relatively easy to match up!

Re:Matching images.

[glueball]

they're relatively easy to match up!


Not always true. A GRE, SE, or FLAIR image sequence for anatomy will not line up well with the EPI sequence of fMRI due to B field non-linearities and shift even if the patient doesn't move. The nice thing, though, is that unless there is surgery and deformation due to swelling, tissue void, or skull shifting, the skull shape stays constant and one can use it as a rigid body for starting the registration.

There are some software programs to attempt it but it still comes down to an expert setting some landmarks and using some validated software. Sorry, but the "free software" world is behind the commercial software and I see the gap widening.

Bill

Re:Matching images.

[MillionthMonkey]

Fortunately, most brains (unlike arms and legs) aren't in the habit of moving around a lot during an MRI scan, so they're relatively easy to match up!

You've obviously never spent an hour inside one of those machines. I used to do research in an fMRI lab and even something like post nasal drip eventually makes you swallow just to keep breathing and the slight movement pushes your head into a new pixel lattice so when you subtract the images you just see gray everywhere.

Re:Matching images.

[buswolley]

My thought exactly... I swore I was showing a subject their brain in 3d just the other day after their run through our fMRI memory study.

The machine takes many 2-d image slices, and a program aligns them into a 3-d model.

Re:Matching images.

[Anonymous Coward]

That's why I said 'relatively' :)

Or are you suggesting that your brain actually moves around inside your skull when you swallow..?

Re:Matching images.

[glueball]

Or are you suggesting that your brain actually moves around inside your skull when you swallow..?

The brain really moves the B field during swallowing, resulting the the brain apparently moving.

Re:Matching images.

[Anonymous Coward]

Far out, man!

It was originally meant as a joke, but you learn something new everyday. :)

Re:Matching images.

[MillionthMonkey]

The field isn't jarred too much because the brain has low magnetic permeability and it doesn't dictate the field topology. But your skull moves enough for subpixel displacements to skew the later analysis. I used to hate these experiments. They all had to be done at 3 AM because the machine was constantly in use by paying customers. We were measuring pain, too, which really really sucked for a bunch of reasons:

- Nothing shows up on an fMRI (it seems a few neurons fire somewhere and you're in excruciating pain)
- It's hard to inflict pain in a reproducible way; "rate this from 1 to 10" is how you calibrate it
- The pain must be inflicted nonmagnetically
- Inflicting pain will easily cause injury; everybody comes in with rectangular burn marks the next day
- Afterwards nobody volunteers for the experiments so you end up doing them all on yourself

To inflict reproducible pain without using electric currents, the lab used a clumsy water bath heated to some point between 40-45 C. You get a two degree window of injury-free pain before protein denaturation occurs in tissue, so the water bath has to be heated just right. Then you pump it into the room via leaky rubber tubing that cools the water down a bit before it runs through an aluminum waterblock velcroed to the subject's arm. So you have this back and forth over the intercom as you fiddle with a thermostat and the guy either experiences no pain or is getting a first degree burn.

Meanwhile the vision people would come in and get to watch little movies in there, and they'd have this rich intense fMRI signal to work with the next day. They hardly had to do any alignment correction at all. We couldn't see anything without extensive processing of gigabytes of noise (this was a decade ago before CD burners became reliable) and it still ended up looking like faint noisy crap. I can't even remember if we got anything to light up. Pain sucks.

We all know what a man's brain really looks like-

[tomhudson]

http://www.suddenlysenior.com/Images/malebrainwome n.gif

The ultimate geek

[GregPK]

Would order pictures of his brain and keep one on his desk in a frame.

I'm such a dork, because I kinda want one too. :-D

Re:The ultimate geek

[morgan_greywolf]

No, no, I'm afraid, GrekPK, that it's only you.

Re:The ultimate geek

[xaxa]

No, the ultimate geek makes a 5 metre by 8 metre (I'm guessing) colourful brain scan and sticks it to the wall of a university.

The identity of the brain scan, in the main College entrance - a closely guarded secret until now - was revealed before The Queen departed. "The Queen asked whose brain it was and Jo Hajnal said 'it's mine,'" said Danish artist, Per Arnoldi, who worked with the professor, an imaging scientist at the Hammersmith campus, to create the visualisation.

http://www.imperial.ac.uk/p5374.htm

I can't find a picture.

Re:The ultimate geek

[normuser]

No, no, I'm afraid, GrekPK, that it's only you.

No, he`s not the only one.
I would love to have a picture of my brain.

Re:The ultimate geek

[talljuan]

For $2-3000, you can. Just go to any MRI facility and make an appointment.

The alternative is to get to have it prescribed by a Neurologist, and then your medical insurance should pay for it. At least, that worked for me.

In my case, I am actually able to say the following:
    <ahnold>It's not a too-mah. Wait. It IS a too-mah!<\ahnold>

(yes, I did in fact have a brain tumour)

Not likely

[$RANDOMLUSER]

If it really was a 3D representation of my brain, all you'd see would be tits and code (and maybe some beer).

for the lazy

[bumby]

http://www.technologyreview.com/player/07/08/06Sau ser/1.aspx

nothing new

[j00r0m4nc3r]

Computer-assisted stereotactic neurosurgery has been around for a long time. The software takes MRI slices and uses a marching-cubes-type algorithm to convert from texels to voxels. I don't see how this software is anything new really, other than maybe using some other kind of input image.

Re:nothing new

[chrplr]

The program displays fiber tracks computed from the diffusion tensor images. This involves more than just fusioning images from different modalities (fiber tracking is far from obvious). Also, some images are 3D surfaces reconstructed from 2D slices. This being said, it has been possible to make and manipulate this kind of images for a while with the (freely available) program "anatomist/brainvisa" (cf. www.brainvisa.info).

Nothing new here...

[perrin]

I work within the field of medical imaging, and this is nothing new. People have been doing image fusion with images from different image modalities for over a decade. There are lots of products like this one, some even open source and with more impressive screenshots. Why is this particular product, which is not even named or referenced, featured? If you want to see impressive open source work within the medical world, check out ITK and VTK (http://www.vtk.org/ and http://www.itk.org/). Now that is really cutting edge work done with free software.

Re:Nothing new here...

[cbelt3]

.. a decade.

How about over THREE Decades. As a high school student (in the 70's), I worked on software to merge CAT scans and thermal scans of the brain during an NSF summer program at Mizzou. Fortran IV, big honkin IBM 360 mainframe, etc. The first run with a full data load took the entire University mainframe down (hey, I was only 15 and didn't understand JCL, shoot me). We were trying to auto-diagnose tumors.

The basic engineering has been refined, but the science is still the same.

Re:Nothing new here...

[Anonymous Coward]

I was also disappointed to not have the package/link to the software mentioned. I'm pretty sure that this article, posted Aug 3,2007, can provide a bit more detail - there they use MediCAD:

http://www.sciencedaily.com/releases/2007/07/07073 0173404.htm
http://www.cs.sunysb.edu/~vislab/papers/Xin_ISMRM. pdf - paper from Thomas Jefferson University and Stony Brook University

Other posters here note the concept is not unique; but perhaps using it for surgical procedures regularly, rather than pure research, might be a 'first.' Thanks for the other link, I'll pass that along to our imaging folk as well. Still looking to see if either place has a page up for it.

Re:Nothing new here...

[mclaincausey]

Nothing new unless you bother to RTFA.

PET/CT scanners can produce coherent datasets that can be fused by a simple overlay, and software like Mirada 7D Fusion (which we integrate with) can fully deformably fuse all sorts of datasets from differing modalities. This is usually a 2D overlay, and it looks different than what this software is doing. They are claiming that specifically fibrous tumor growth can be visualized in new ways. So perhaps the fusing isn't novel, but the method of fusing and/or visualization may be.

Re:Nothing new here...

[RockoTDF]

I doubt your mainframe in the 70s could render 3D imagery...so I don't think this is 3 decades old.

Re:Nothing new here...

[perrin]

I did RTFA. 3D image fusion is not new, either. There is a lot of exciting research in the area, though, and if they had spoken about new ways to accurately co-register different image modalities, it would have been interesting, but this was apparently all about visualization, which is not. I am also not all that much of a fan of completely merging image modalities. It can be useful to get an overview of where you are, but I am afraid you could easily lose important tissue boundaries since the algorithms used are not yet better than a trained human eye. In any case, visualization is only as valuable as its accuracy under real world conditions. Once you surgically open the brain, you can often expect significant brain shift, which means that your preoperative images can be way off course. IMHO, correcting such shifts during surgery is far more important than fancy 3D visualization. Although, fancy 3D visualization is starting to look quite cool.

Re:Nothing new here...

[cbelt3]

Yes it could. It's not the rendering math, or the 3d basic algorithms that have experienced advances (although they have). It's the resolution of the sensors. Our brain scan was a cube approx 1024x1024x1024. That was a boatload of data for the 70's. The ones discussed are ginormous data sets.

Re:Nothing new here...

[RockoTDF]

But could you view and rotate the brain, or was this a CLI type thing? I mean 1024x1024x1024 is probably enough for detecting tumors or brain damage, but neuroscience requires more which is why this article isn't as irrelevant as some are saying.

Re:Nothing new here...

[cbelt3]

Oh, you were looking for a display, and maybe a mouse and overlapping WIndows ? Sorry, hadn't been invented yet. We did build a color map display and rotate it, though. This was the Biomedical Image Analysis Lab. Very cool stuff for 1975. Imaging and display ran on a PDP-11...mmm. back in the day. had to boot that sucker by loading in the punch tape boot code sequence in octal, then load paper tape, THEN we could actually load the programs from Mag tape. you guys and your newfangled hard drives don't know how easy you've got it ! (mumbling about steam powered computers and young whipper-snappers..)

Re:Nothing new here...

[eozh]

Actually, clinical MR images rarely go beyond 512x512x200.

Re:Nothing new here...

[RockoTDF]

Yeah but in any science or medical field there is always a difference between the clinical usage of something and what scientists need to do the base research.

Of Course

[alexj33]

Of course such an immensely complex and magnificent entity (the human brain) came about purely by accident; something which scientists cannot even hope to even copy and reproduce on purpose.

Re:Of Course

[Bohnanza]

Who says the human brain came about by accident? Evolutionary theory certainly does not say this.

Re:Of Course

[ChromeAeonium]

You're half right. Evolutionary theory states that the best random mutations survive. Due to their superiority, the survival of organisms with beneficial mutations(and therefore, their ability to pass on their genes) is not random, however, the existence of their desirable traits is purely random.

In other words, the brain in its entirety, as a compilation of traits, would not be random, but each and every individual function would be. I believe that the point that the parent poster was trying to make was the improbability of a collection of random beneficial mutations, each one highly unlikely to emerge in the first place, forming an organ as complex as the brain, which, due to its extreme complexity, is still not fully understood, nor can it be replicated.

Re:Of Course

[Bohnanza]

I do understand what the parent poster was saying, and it points out the great misconception of those who see "intelligent design" in greatly complex organisms. I believe this great complexity is a sign that we are NOT intelligently designed. Take, for instance, the human immune system, or in particular the inflammatory system. There are many different systems functioning, many at cross purposes. And the effect of inflammation is rarely beneficial to the organism. I would hope that an "intelligent" designer would have done a much better job. The "design" could certainly be much simpler and work at least as well. No, the great complexity of living organisms is a sign that that they have indeed evolved by "accident".

Re:Of Course

[glitch23]

Of course such an immensely complex and magnificent entity (the human brain) came about purely by accident; something which scientists cannot even hope to even copy and reproduce on purpose.

You think humans can't reproduce a human brain on purpose but yet you think that the same immensely complex and magnificient entity came about by accident? You do realize the odds of picking out the winning numbers to the Powerball lottery game, right? Now, increase those odds by an astronomical amount to get the chances of a complex entity like the brain (let alone the entire human body) spontaneously and slowly forming plus the odds of that soon-to-be brain forming within the actual entity that is to become the first human. Carl Sagan and a couple others once calculated the odds of a man evolving and came up with 10^2,000,000,000 and coincidentally Borel's Law of Chance states that when odds are more than 10^200 an event will never occur even given enough time. By the way, scientists have attempted to re-create evolutionary processes (which by your statement seems to be easier than doing something on purpose) in the lab but have failed to do that as well.

Re:Of Course

[ChromeAeonium]

Huh? Inflammation raises the temperature of a specific part of the body to kill, or at least inhibit the growth of, the infecting pathogen. It's not good for the organism, but its not good for the causative agent, either.

I can't say I understand what you mean when you say that a system as complex as the immune system (what with all the differentiated cells, specific antibodies, cytokines directing cells, the compliment system, the antigen recognition and memorization of the adaptive immune response, ect., all preformed with no outside direction by nonthinking cells that are highly complex machines on their own) is indicative of randomness. Personally, I always thought that such a finely tuned molecular clockwork was evidence to the contrary.

Re:Of Course

[Anonymous Coward]

Think about it this way for a second: If it weren't finely tuned and highly complex, it probably wouldn't be doing the job.

I understand your line of reasoning, but it really isn't an argument in either way, IMO.

Anyway, put that way, the whole ID vs. evolution debate comes down to something pretty simple. On one hand, we have complex organisms and a theory about how they came to be. On the other hand, we have disbelief about this mechanisms - evolutions - potential to create those organisms.

From a scientific standpoint, the case is quite clear: Stick to the scientific theory and keep an open mind about the future. For disbelievers: try to find evidence for an alternative theory. All this "evidence" that's basically just an exercise in finding examples of really complex biological structures cannot, ever, by principle, "disprove" evolution.

Re:lol

[thatskinnyguy]

I think I lost all faith in /. peer moderation. I drop a Star Trek reference and get modded-down while some other guy talks about titties and beer and gets praised. *shakes head* [/whining]

Re:lol

[$RANDOMLUSER]

While I'll grant that Dr. McCoy is important to the /. community, I'll submit that titties and beer are sacred.

"noodle"

[jo42]

My "noodle" is nowhere near my brain...

Re:"noodle"

[B4D BE4T]

Same here. Although I tend to get into trouble for thinking with that noodle from time to time...

House

[Elsapotk421]

great...now Dr. House will be ordering up a million of these on every episode.

Not really a new thing

[fragMasterFlash]

While it is good to see more talented people working in the medical visualization space, this is not really a new thing. Image Fusion has been around for a while now but it has not yet become a mainstream technique.

Re:Not really a new thing

[magbottle]

> While it is good to see more talented people working in the medical visualization space,
> this is not really a new thing. Image Fusion [wikipedia.org] has been around for a while now
> but it has not yet become a mainstream technique.

It is very mainstream for PET/CT fusion. Many manufacturers make combo PET/CT machines for just this purpose since the acquisitions are done at the same time, they align very closely and little if any rotation/translation has to be done for a good volume match.

New Dizzy Dean quote

[Bohnanza]

The doctors created a full 3-D image of my head using multiple imaging technologies and found nothing.

Getting close

[dissy]

One step closer to Mind Uploading :D

The Mind Uploading home page is dedicated to the putative future process of copying one's mind from the natural substrate of the brain into an artificial one, manufactured by humans. This technology will radically alter society in many ways, as science fiction authors have begun to illustrate. Through this server, explore the science behind the science fiction! -- Dissy

10 years ago

[broothal]

"That is why I am excited about something that should have been here 10 years ago."

It was. I actually wrote my master thesis about it exactly 10 years ago. But one thing is the technology. Another thing is someone to fund the development of a fully functional package. Technology is many years ahead of reality when it comes to medical imaging.

Re:10 years ago

[jonscilz]

so true!!! my entire senior design project dealt with biosignal analysis in a drowsy driver simulation. we have working models just not in commercialized forms. the technology and science is way ahead of reality in biomedical engineering fields. its sad that the fda and large private companies control the rate of growth and release of these technologies!

This is your brain

[VinB]

... and this is your brain on Dual 768MB Nvidia GeForce 8800 Ultra graphics cards.

Re:We all know what a man's brain really looks lik

[struppi]

Yea, but TFA said 3D image of The Brain.

My Brain

[Coppit]

A friend of mine was doing fMRI research and was kind enough to give me the data she got from my brain. At the time AFNI was the best software for converting the data to movies. The software wasn't exactly easy to use. Since then I think the options have gotten much better. Anyway, here's my brain. I especially like the axe-to-the-head view.

Re:My Brain

[dogmatixpsych]

I hate to say this, but as someone who deals with MRIs of brains pretty much every day, there is something seriously wrong with your brain!


Ok, I'm just kidding. It looks perfectly normal. (I wasn't kidding about dealing with MRIs on a daily basis though). Nice images; thanks for the links.

not news

[jonscilz]

this is not new! we have been doing this in the CILab for years at rutgers http://biomedical.rutgers.edu/faculty.php?id=23

Old News

[conspirator57]

SGI Volumizer did this like 10 years ago...

http://www.sgi.com/products/software/volumizer/

Getting your data

[gr8_phk]

My wife recently had an MRI done (She's fine). The thing that struck me was this: On the way out to the waiting room they told her to wait a minute while they made her CD. They handed us a CD containing all the slice images (DICOM) and an auto-run (yea windows) program for viewing them. I always thought it would be cool if you could get your scan data, but here we didn't even have to ask - they just hand it out. I figured the hospital would be afraid of some perceived threat of getting sued. I also wonder if they still hand it out if there's a big obvious tumor in there. You know 'cause the tech isn't supposed to tell you anything - that's the radiologists job. Don't want the patient to go home and look and freak-out without having someone to explain. OTOH, if you know the policy is to give it out unless there's something wrong, and they don't give it to you...

I hope this becomes the norm. er - giving out the data, not my wife needing an MRI.

Re:Getting your data

[talljuan]

In fact, they do give you the CD if there is something obviously wrong.

After many episodes of Really Bad Headaches, I finally had a scan prescribed. I noticed that the tech and assistants seemed to look at me quite a bit differently after the scan. It turned out that I in fact had a tumour and a very large cyst which would have been immediately obvious to anyone, no medical training required. In retrospect, they were probably thinking "you shouldn't be walking around with that thing in your head!" about me. And they did give me the CD. It was probably a good thing I didn't look at it before I went to see my Neurologist!

Anyways, two weeks later I was on the table at UCLA medical center, where Dr Linda Liau and her wonderful team fixed me up.

Re:lol

[Anonymous Coward]

The last few days, there seems to be an explosion of overjealous and very humorless clueless idiot(s) flinging 'offtopics' and 'flamebaits' left and right throughout many stories and posts. I hope they or he settles down soon, or else takes some aspirin, or has an 18th birthday. I offer the guilty parties $50 off a sense of humor transplant. This offer ends soon, so call in now!

Re:lol

[Anonymous Coward]

Ooops, meant 'overzealous'. Bet I get modded down by a grammar Nazi, too. Blimy, might as well slap me with a wet fish, too.

The Original Project

[big dumb dog]

I think I heard that this software project originated in Redmond, but it failed in the QA process when they were unable to find a brain to test it on. (...it's a joke)

Oblig..

[jd]

Imagine a beowulf cluster of brain-imaging systems...

Re:Oblig..

[Anonymous Coward]

No.

FYI,one way to compute 3d brain models from MRIs..

[syn1kk]

is done through the application of 'level sets' its pretty fun stuff to study. There are some cool videos of level set stuff like this one: http://graphics.stanford.edu/~fedkiw/animations/wa ter_oil.avi . And the application of level sets is very broad. I've only dabbled in Level Sets but I am very tempted to do a Master's thesis on it =P.