Slashdot Mirror
This Is Your Brain On Magnets — Or Maybe Not
conspirator23 writes "Jon Hamilton of National Public Radio brings us a story about 'voodoo correlations' in fMRI studies that seek to learn more about emotional states, personality, and social cognition in the human brain. Many of us outside the scientific community have been treated to fascinating images of brain activity and corresponding explanations about how the images reveal which portions of the brain are engaged in certain kinds of thinking. But these images are not actual snapshots; they are visualizations of data generated by repeated scans during experiments. Flaws in the statistical methods used by researchers can result in false images with a variety of inaccuracies. Yet the images produced are so vivid and engaging that even other neuroscientists can be misled by them."
There's a "zombies" joke in there somewhere.
REALLY?
*clicks link to look at pictures*
*curses internet*
These sort of images are pretty familiar to me and I must admit I was never skeptical of research showing that you could classify brain patterns based on the object they were looking at or how they were feeling. I had thought this had gone so far as to be used to classify terrorists and used in trials (which is quite unnerving)! Well, it saddens me to say this but in a field where we normally take two steps forward today, we are taking one giant step back. The brain is such a complex thing to study concerning biology, chemistry, electromagnetic physics, psychiatry and psychology. The line where the physical sciences stop and the psychological science starts is so blurred and confusing, it's a shame that one of the few tools used to determine the hows and whys of it is being called into question. I think a lot of us hoped there was some hard scientific way to unravel this mystery of cognizance and conscience. After reading the article, it's a good thing this happened but a shame for quite a bit of research out there that must now be re-examined.
My work here is dung.
I've always wondered how useful these images really are. Perhaps to the trained eye they can reveal a lot about how a persons brain works but they have always struck me as being too abstract. We can point at a portion of the image and say that bit controls movement, for example, but if anything goes wrong we are stuck because at a fundamental level we don't understand how it controls movement. I suppose it's a bit like looking at a block diagram for a CPU and not understanding how each bit works.
It will be interesting to see how we achieve the next level of understanding of the brains functioning. I can't see that we will ever get there with MRI or electrode probes because, I think, they are simply too large to get a true understanding of what is going on. I suspect we will gain our understanding through modelling but I'm not sure I'll be around when we do.
I used to have a better sig but it broke.
It's easy to dismiss the results as noise, but any decent researcher will estimate her or his error bars, and show that the signal measured is, indeed, small or comparable to the error bars. The way the article is written, it just sounds like they just totally ignored the results because they don't like them or something. To be fair, the paper itself probably does a better job of defending its position, but I don't have time to understand all its details.
I will say, it would be easy to make wild claims about what areas of the brain "do" things just by looking at a scan and showing a pretty picture.
That said, consider these things:
While non-peer-reviewed publications often publish exciting results, the scientific community typically does not accept brain regions without corroboration from many different studies with different stimuli, often including monkey studies where real electrodes and not just low-res fMRI can be used
It is difficult to get the numbers of subject that would be considered standard in other studies for fMRI studies. First off you actually need subjects who will do the assigned task, then you need them to do it perfectly still, for anywhere between 20 minutes to several hours (usually in no more than 1-hour segments). So the likelihood that just one study could prove something is quite small.
In many (perhaps most) studies, all the subjects brains are averaged together for data analysis, there are several different ways of doing this, none of them particularly accurate. This again calls attention to the need for multiple studies
It's also important to actually know what you're looking at when you see pictures of "brain activity", usually you are looking at the averaged activity of many subjects, after it has been run through (most likely) some form of general linear model or event-related analysis. Both of these methods estimate and fit a hemodynamic response function (the pattern of brain response to a stimulus), and what you're actually looking at is the fit or perhaps t-values (roughly fit/std. deviation) for each voxel.
Also note, that for almost any study, I could pick some random brain areas that are "lighting up" and claim a response, but they would almost certainly be shot down with more subjects, another study, etc.
bottom line, responsible investigators can make good sense out of fMRI data, but doing one experiment and claiming you "found the love [or insert whatever emotion/though] center is irresponsible and should be correlated with other studies and hopefully monkey studies as well.
Ze Atomic Device! It iz Ztolen!
I think most people in the neuroscience community are aware of the limits of current fMRI approaches. The general linear model, which is used to compute blood flow, is rightly under considerable attack from a number of directions (it assumes, among other things, that all measured hemodynamic response is the result of changes in underlying neural activity, and there is now quite a bit of evidence that this is not the case). And the basic paradigm for most fMRI experiments, especially ones examining 'higher cognition' and emotion is the deeply limited subtractive inference (which was once described to me as "You show somebody a picture of their ass, and you show them a picture of a hole in the ground, subtract the two responses, and claim you've figured out the area responsible for discerning your asshole from a hole in the ground). The 'double dipping' described in the article is actually quite a minor concern compared with the above, although certainly a real one.
But the real value of fMRI, regardless of the deeply flawed current methodologies, is that it does give us a very good idea what areas of the brain we should be looking at with other experimental techniques, such as various types of electrophysiology, anatomical tracing, inactivation etc... And the good news is that there are fMRI methods being developed which are much more robust and will be able to tell us a great deal more about what's going on in the entire brain over various tasks, such as multispectral MRI and attempts to use stronger magnets to directly measure currents, rather than blood flow. So while I certainly agree that there is an 'Oooh! Shiny!' element to a lot of current fMRI research on higher cognition, and one should be deeply skeptical about many of the assertions made on the basis of such data, that doesn't mean fMRI is not an incredibly useful research method, and is likely to become even more so.
fMRI is one of many imaging techniques which continue to evolve and give us more and more amazing data about the brain. But its just data, and subject to the limitations of its tech. Any neuroscientist worth their salt knows the limitations of their technology. The problem desribed in the article is something people have been dicussing as a valid methodological criticism of some studies, not all fMRI data. The summary is misleading, basically its saying a tool used incorrectly results in bad data. Duh.
fMRI has great spatial but bad temporal resolution
EEG and MEG and great temporal resolution but bad spatial
PET has amazing metabolic resoultion but fuzzy temporal and spatial resolution
These are a few examples. The media and bad researchers can't get past the pretty visualization but fit into a proper theoretical model of exploration it still remains one of the most amazing tools of brain exploration thus far.
Oh wait yes they are!
These issues were explored several years ago in a book by Joseph Dumit, _Picturing Personhood_, which covers both the problems in constructing these images, and the way that these images are taken up as facts by courts, the news, etc.
If I buy some of those powerful Neodymium magnets and move them over my head can I get high that way?
I would like to know because that would same me a lot of money, or would I have to keep buying more and more powerful magnets to keep achieving the same "high" effect?
Tsukasa: All I really want, is to be left alone...
A fundamental difference between MRI and CT is that computing the MRI image involves (at least in some cases) a 2-D FFT. While signal to noise ratio is an issue, the processing is not an ill-posed problem.
A Shadeless room is a brighter room.
I've been lucky enough to work with MR and CT imaging researchers for a while now. One of the benefits of this job is that I've gotten to learn a lot about how these images are acquired and reconstructed. It's not quite as bad as making sausage, but it's a lot more involved than a "snapshot".
For CT, we acquire a bunch of 2D images through you from different angles, then do a lot of number crunching to generate a 3D volume. The problem is that you don't hold still while we're doing it. You can try; you can even hold your breath, but you can't "hold your heart". As your organs move between views, we get motion artifacts -- shape distortion, bright or dark areas, even "things" that aren't really there.
For MR, it's even worse. I can barely tread water in the physics of it, but we're effectively capturing a line at a time in 3D space. (We're actually acquiring data in "k-space", then running it through a Fourier transform to make it spatial.) Not only is it subject to motion artifacts, it's also subject to susceptibility artifacts (distortions because of the magnetic properties of certain materials), flow artifacts (blood moves through vessels between the time that we apply a magnetic pulse and the time that we read back emitted signals), and lots of other things.
fMRI is just adding yet another layer of aggregation and interpretation on top of all this. Sure, it's a "visualization of data generated by repeated scans", but so is every CT or MRI image.
3D imaging, especially MRI, is hideously complicated and indirect. It's almost inconceivable that it could yield results with any physical significance.
...and yet, it does. It's become so routine, so reliable, so well-understood and well-controlled, that doctors and researchers know they can rely on it as a matter of course. They still have to be aware of the errors and distortions that can arise, but that's true of every imaging or monitoring system, all the way down to the stethoscope and the fever thermometer.
A fundamental difference between MRI and CT is that computing the MRI image involves (at least in some cases) a 2-D FFT. While signal to noise ratio is an issue, the processing is not an ill-posed problem.
Hmm. My group definitely calls the process of moving from k-space to image space "reconstruction". There are good reasons for doing more than a simple 2D FFT (most of which, I'll admit, are over my head).
Whoever tagged this story 'correlationisnotcausation' is a fucking idiot. You're not as smart as you think you are.
Mod parent up.
3D imaging, especially MRI, is hideously complicated and indirect. It's almost inconceivable that it could yield results with any physical significance.
...and yet, it does. It's become so routine, so reliable, so well-understood and well-controlled, that doctors and researchers know they can rely on it as a matter of course. They still have to be aware of the errors and distortions that can arise, but that's true of every imaging or monitoring system, all the way down to the stethoscope and the fever thermometer.
Modelling the measurement is fundamental to physical science, even something as simple as measuring the temperature of air, and these are much more complex measurements to model.
Just wanted to mention my favorite diss of fMRI - Lite-Brite Phrenology.
I suspect the pretty-picture effect is likely to generalize to just about any science, though I am only aware of this having been directly tested and confirmed in neuroscience (e.g. McCabe & Castel, 2007; Cognition). Although the philosophy behind peer review is sound, it still suffers from the fact that our peers are humans - encumbered by all manner of flawed reasoning. Unfortunately, one of those flaws is that we are easily compelled by impressive looking pictures - regardless of our expertise. Perhaps this is because we implcitly assume that impressive tools and impressive research are necessarily correlated. Sadly, they are not, and we are left with islands of extremely valuable fMRI (and equally EEG / ERP, MEG, DTI - what have you) research, floating amidst a host of absolute rubbish.
...and yet, it does. It's become so routine, so reliable, so well-understood and well-controlled, that doctors and researchers know they can rely on it as a matter of course. They still have to be aware of the errors and distortions that can arise, but that's true of every imaging or monitoring system, all the way down to the stethoscope and the fever thermometer.
The problem with the activation maps is precisely that one is NOT looking at an image, so there's no way to fine tune the algorithms. Therefore, fMRI is NOT well understood in the way that CT or MRI are.
Consider that in imaging, you have the luxury of comparing the output of a brain scan to the known physical structure of the brain. Is there a hippocampus? No? Well then it didn't work, go back and fiddle until you can show me a hippocampus.
In fMRI, apart from low level sensory corticies (where visual field mapping techniques can reproduce broad level retinotopic maps), researchers are operating in a vacuum in which there is no hard and fast error signal to fine tune the methods.
Science has to proceed very cautiously in such a situation. This is particularly true when one has hundreds of thousands of voxels to sift through because it's easy to find any pattern in noise, if you have enough noise.
So I would argue that fMRI offers a very different set of challenges compared to MRI and CT scans, and therefore it's very important to keep a sharp, critical eye on the statistics used, as these authors are doing.
To illustrate this point further, here is a link to a poster in which someone put a dead salmon into a magnet and found that (in the absence of proper statistical controls) its decomposing brain was apparently reacting to the emotional content of pictures:
http://prefrontal.org/files/posters/Bennett-Salmon-2009.pdf
If little magnets were going to affect your brain, wouldn't anyone who'd had a brain scan end up a vegetable?
Of course it pays to sell idiots little magnets and claim all sorts of health benefits. Some may even benefit from a placebo effect. (It doesn't pay to try to sell them MRI machines...there are so few idiots THAT rich).
I think i'll remain skeptical unless more solid evidence turns up.
These posts express my own personal views, not those of my employer
Comment removed based on user account deletion
MRI f you!
> http://prefrontal.org/files/posters/Bennett-Salmon-2009.pdf [prefrontal.org]
> We further argue that the vast majority of fMRI studies should be utilizing multiple comparisons correction as standard practice in the computation of their statistics.
Oh, wow. An abstract from HBM 2009 where the authors have "discovered" the multiple-comparison problem and conclude that one should control the family-wise error rate using methods that have been known to fMRI analysis for at least a decade. The subtitle is "An argument for multiple comparisons correction" as if anyone working in the field would accept results otherwise. Impressive work. Their references consist of two seminal articles from 1994 and 1996 addressing the same problem, solutions of which have all been implemented in all widely used analysis packages for ages. There must be a lot of idiots that get abstracts accepted to HBM.
Most of my psychology colleagues have no idea what they're looking at in fMRI. They assume if it lights up, it's making something go. They may know full well that neural activation can be excitatory or inhibitory, but fail to make the connection and figure out that what's lighting up may be de-activation. Both the gas pedal and the brakes appear the same to fMRI and nobody can tell which is which with this technology alone.
Even fewer even bother to try to grasp the math behind the analysis technique, statistical probability mapping. Every cubic pixel ("voxel") has to be compared to every one of its neighbors across the multiple data collections in each condition, and a T test applied. In order to prevent artifactual results due to this massive amount of statistical testing an error correcting normalization is applied. Statistical results of 0.05 or 5% are considered the limit for acceptable results. With the correction factor applied, each voxel that lights up is passing a statistical test with anywhere up to 22 digits below the zero that I've seen myself. Their lighting-up numbers are more far fetched than being hit by a falling meteor.
They almost invariably fail to mention areas that work with an area of their interest but fail to light up when their target does.
The problem extends beyond just the researchers. A recent article in the Proceedings of the National Academy was filled with these errors yet has become a part of a well respected body of works, poisoning it and giving others cause to continue believing their fallacies and compounding them with works justified by this one.
"I may be synthetic, but I'm not stupid." -- Bishop 341-B
Couldn't the image sampling be done synchronously with the heart beat for instance or is the integration time too long?
For CT, we acquire a bunch of 2D images through you from different angles,
Wasn't that "a whole bunch of 1D images, which are then computed into 2D-slices, which are then assembled into 3D volume representations"?
Last time I checked, CTs acquired one-dimensional images. However, it's been a while, and I work in a different field of biomedical engineering, so I haven't really kept up with more recent developments.
That's exactly what you have to do, and assume (usually safely) that every heart cycle is pretty much identical. We call it "cardiac gating". You can either do it "prospectively", which means triggering acquisition at a particular point in the cardiac cycle, or "retrospectively", where you acquire freely and then go back and pick out the acquisitions that happened at the cardiac phase you want.
It's still tricky, though, particularly for MRI, where you're generating a lot of electrical noise -- electrical monitoring (conventional ECG) just won't work. We use optical sensors. Of course, we're imaging mice with 600bpm pulse rates, which makes it trickier still.
For CT, I think each "exposure" we do is typically 10ms or less, which lets us get multiple phases even in a mouse. For MR, well, remember when I said the physics is over my head? Yeah, that. But I know we can routinely split the mouse cardiac cycle into 8 phases, and sometimes more. You're imaging for a pretty long time to do that, though -- collect views through many cardiac cycles at 0ms offset, then at 10ms offset, then 20, then...
Anyone who is actually interested may want to check out one response paper to the Vul Voodoo Correlations paper which points out a number of problems that Vul himself has in his analysis. http://www.scn.ucla.edu/pdf/LiebermanBerkmanWager(invitedreply).pdf
Good catch. "We" (well, our actual researchers, not me) built a small-animal CT system that captures 2D images, rotating the animal instead of the gantry between views. That's very different from a clinical CT system.
However, those clinical scanners are getting wider. You can now get scanners that acquire 64 slices at a time -- instead of an N-pixel linear sensor, they use an N x 64-pixel sensor, and scan in a helical pattern. It's all about speed in the clinical arena.
Thinking about it - using a 2D-sensor is a logical evolution to get faster and better pictures. I mean, the very first generation didn't even use a linear sensor array, they used a single sensor that was moved in order to get the measurements at different points in space.
Then again, my last practical experience (as a patient) with a CT system was in the early 80s, when the whole procedure still took half an hour or more (at the same time, getting some MRI scans to three-and-a-half frickin' hours). Now, we're down to seconds for the CT and half an hours for the MRI.