Slashdot Mirror
The Neuroscience of Screwing Up
resistant writes "As the evocative title from Wired magazine implies, Kevin Dunbar of the University of Toronto has taken an in-depth and fascinating look at scientific error, the scientists who cope with it, and sometimes transcend it to find new lines of inquiry. From the article: 'Dunbar came away from his in vivo studies with an unsettling insight: Science is a deeply frustrating pursuit. Although the researchers were mostly using established techniques, more than 50 percent of their data was unexpected. (In some labs, the figure exceeded 75 percent.) "The scientists had these elaborate theories about what was supposed to happen," Dunbar says. "But the results kept contradicting their theories. It wasn't uncommon for someone to spend a month on a project and then just discard all their data because the data didn't make sense."'"
The WIRED piece threads what is written in the summary around the story of how Arno Penzias and Robert Wilson at Bell Labs discovered Cosmic Radiation after being puzzled for a year about background noise on their radio telescopes ... even scraping pigeon poop off their gear as a possible source until they realized the signal was real - Homer Simpson would have said D'OH! ;-)
Hulk SMASH Celiac Disease
"It wasn't uncommon for someone to spend a month on a project and then just discard all their data because the data didn't make sense."
That doesn't mean the data is wrong, it means the /hypothesis/ was wrong, if not the theory, and needs to be modified.
If they're really throwing out date just because it 'doesn't make sense', they're doing religion, not science.
If the data don't make sense according to your theory, you don't discard the data, you discard the theory and work out a new one that fits the facts as you've observed them. TFA says that Dunbar was watching postdocs doing research, and if so, they should have known better. Alas, too many people who call themselves scientists are more interested in proving their pet theory true than in finding out what's actually going on.
Good, inexpensive web hosting
If problems occur as you postulate elaborate hypothesis, then stop piling up the elaborate hypothesis! But be sure and still make available your existing (complex) hypothesis, methodology and unexpected data - preventing others from going down the same path with the same methodology is still highly valuable!
Let's say you're looking at a production and consumption cycle involving neurotransmitters and neuroreceptors of some sort, and the various channels of input and output involved. Your starting presumption you base your hypothesis on is that there is a buildup which triggers an electrical signal to stop consumption and clear the channel. The only evidence you can realistically gather for now is protein density at a certain output channel - but others have worked to ensure this is a reliable approach specifically under these circumstances.
So, you do the specific experiment, trigger the signal, but you get a wildly different result - the stop in consumption occurs, but the protein density does not change at all in the output channel. What actually happened is still unknown, only you haven't verified any correlation with your hypothesis. You still have valuable data, but no mechanism to verify under the circumstances. Either your methodology failed, or you misunderstood what was happening - and the world of knowledge is made larger by either... even if your paymasters won't get happy about the result.
Science is often like throwing pebbles in complete darkness - it takes a lot of stones and close listening to make out a mental picture of the scene - especially when there's a lot of noise already around. Everyone would love it if we could just flip the lights on - but we have yet to invent a light that can see into the inner workings of the functioning brain very well. Gotta keep throwing those pebbles for now.
Ryan Fenton
I think the parent post is a brilliant example of what happens when someone perfects trolling to a science.
It takes a man to suffer ignorance and smile
Be yourself no matter what they say
The dirty little secret is that the Y is not always unexpected, just too politically incorrect and dangerous to be released to the public.
So, when reality is racist, you change it?
Yeah, it's just you. AP News found no evidence of massaged or ignored data (http://news.yahoo.com/s/ap/20091212/ap_on_sc/climate_e_mails). So climate science is a poor example of this thesis.
I can't help but think that Neuroscience needs to calm down, sit back, and take a deep breath. We are examining a system and we are trying to reverse engineer it. We can't start out by trying to create elaborate hypothesis for large systems, we need to go low level and examine the simpler systems. I really think they should hold on to the higher cognitive models for a later time because we can't even completely model C. Elegans and it has the least neurons of any, current, living organism. The way I see it, I total expect their hypothesis to be wrong, because they don't thoroughly understand the low end of the system.
Eat sleep die
By "almost as well" I assume you mean "all the time". The "sceptic" arguments are nothing but a parade of cherry picking with little attempt at genuine investigation.
And there's no real evidence of the proper scientists massaging or ignoring anything. Just because a detailed, written account of everything doesn't exist in stolen, incomplete private documents doesn't mean it doesn't exist at all.
They'd get 42 dollars?
Or when the results you get aren't acceptable to the people responsible for continued funding.
Years ago, I worked for months trying to reproduce the Polywater research,
http://en.wikipedia.org/wiki/Polywater
and eventually reported that I was unable to do so.
The department considered my work a failure (as in, I must have been incompetent) and did not publish my findings. When, years later, the publications reporting successful discovery/creation of Polywater were shown to be fraudulent, and my results were correct, I did not even receive an apology.
Throwing out results is unethical as well as irresponsible. Many discoveries have come from re-evaluating what appears to be "bad" data. It might not be possible to use it now, but it should be at least stored.
For instance, it has been reported that the "bit of "scruff" on her chart-recorder papers that tracked across the sky with the stars"[1] looked like bad data to Jocelyn Bell Burnell's supervisors. Today we call the phenomenon a pulsar.
[1] Wikipedia
If the data doesn't fit your theory, the problem is most likely neither with the data (which is fine) nor with your theory (which may also be fine) but with the method you used to produce your data. You probably wired in an incorrect resistor, forgot to close a parenthesis in your Perl code, forgot to add the correct amount of EDTA to your reaction, etc. Then your results ended up looking like shit, and not surprisingly. Doing science is hard.
There's no need to postulate any grand conspiracies or take pot-shots at science in general. This paper is examining real people doing real shit. Most of the time we fuck up, and we're not smart enough to figure out where we made the error.
And there's no real evidence of the proper scientists massaging or ignoring anything. Just because a detailed, written account of everything doesn't exist in stolen, incomplete private documents doesn't mean it doesn't exist at all.
The behaviour surrounding the data is certainly indicative of a lack of confidence in the findings. Refusing FOI requests and claiming that "the dog ate it" do not show a group filled with the belief that their research is unassailable.
The idea that race is a fiction is a bad, well, fiction, and a clear example of the distortion of thought due to political correctness.
There are a number of human traits (and the genes which cause them) that statistically cluster into groups that correspond to what we consider race. You can test a person's DNA and determine their racial heritage, to a fairly accurate degree. Obviously race is real, if you can nearly automate measuring it. The fact that statistical clusters don't have firm boundaries doesn't mean those clusters don't exist.
Is race relevant? Not for most purposes, but it is for some. I understand that Asians are more likely to have difficulty digesting milk, for example; blacks have a higher tendency to have sickle-cell anemia. Declaring that any test that shows a tendency for races to vary based on genetics is CERTAIN to be flawed because you don't believe race exists is ludicrous.
"It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with experiment, it's wrong."
- Richard Feynman
"The most exciting phrase to hear in science, the one that heralds new discoveries, is not Eureka! but rather, "hmm.... that's funny...."
- Isaac Asimov
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
I am calling this neuroscience because it has nothing to do with how the nervous system operates. In this sense I am following the lead of WIRED and/or Dunbar, who can't tell a neuro from a social. From TFA: "Kevin Dunbar is a researcher who studies how scientists study things". OK, he studies things called scientists. scientists are people. The study of people and how they behave is psychology. Science is a social activity. Investigations of social activities are sociology when taken as a whole, or social psychology when considered in terms of the activities of individuals operating within a social group. Dunbar studied social psychology, not neuroscience. There's not a speck of neuroscience cereal in it anywhere. There's very little if any actual social psychology, and psychology, or any science at all. There's talking about science, there's talking to scientists about doing science, and there's watching them do science. There's watching and talking about getting good results and not getting good results, and what people do in the matter case. If Dunbar thinks he's doing neuroscience, I suspect he's not even very clear on science itself, much less the various branches. And it does say he's "a researcher in", not that he's a scientist. I do research in curry recipes from different countries and cultures. I'm a researcher, but not a cultural curriology scientist.
In fact I'll go s far as to say he's a researcher because he knows precious little and is trying to find out basic things, not as is the case with most scientists, someone who knows a fair amount and is trying to build on that with new knowledge. He is apparently not clear on the difference between 'screwing up' and not getting good and/or clean results. This may well be because he was unclear himself as to what it was he was looking at and talking about, and he thought he was just not getting good or clean results, when actually, guess what?
He doesn't let loose any secrets. Anyone can talk to scientists and as what happens if and when things don't turn out as expected. If you get an honest (ie. less concerned with appearances than truth) scientist, anyone would get the same answers. Or one could simply read work from real social psychologists and others who study science and scientists and learn the same things. I myself always recommend Collin's & Pinch's "The Golem" as an illuminating, instructive and entertaining starting point.
And a technical point on methodology: a study that does not find a difference between groups, treatments, whatever, 'fails to reject the null hypothesis' (the assertion that there is no observable difference). It does not prove there is no difference, it merely fails to find one. It fails, but only to find a difference, not to produce a result. It can't say there is no difference, it can only say that it couldn't find one. And, it fails to find a difference, no matter how nicely or hapazardly the data come out. The only studies that "fail" produce no data. Scientists may further fail to find an interpretation, but there's no limitation on trying to figure this out, and it applies to both 'results' (reject null hypothesis) and 'no results' (fail to reject null). Studies that produce data that 'makes no sense' produce data that fails to reject the null. The 'making no sense' is a post hoc evaluation of the data based on an incomplete understanding of the design, collection, analysis or interpretation. Such evaluations are done in science, but they are not part of the scientific process. Therefore when this occurs, it is not a "scientific" result and cannot be taken to reflect in the nature or quality of the work done. If you can't figure what it means, you can't figure out. You cannot say that since you cannot figure it out, then you figure out that it fails. If you think you can take something that 'doesn't make sense' and then say that it makes sense in that it represents a failure, then you've contradicted the assertion that it makes no sense. All you can say is that you don't understand it, and since you d
"I may be synthetic, but I'm not stupid." -- Bishop 341-B
Here's the problem. If you can't order every single human into one race or another, your model is flawed. If you're forced to resort to mixes of races, well, then you don't have any distinct race left.
Race concepts fall apart once actual taxonomic principles are applied to them. Your examples actually illustrate the problem quite nicely: not nearly all asians have problems with milk - specifically the Japanese the do. Indians (from the Indian subcontinent in Asia) do not. Blacks do not have a higher tendency for sickle-cell anemia, a certain group of people in Africa do. Blacks in the US do not have that trait.
How much does it suck to be so wrong? Your cognitive dissonance must be at a record high.
Those who can, do. Those who can't, sue.
Here's the problem. If you can't order every single human into one race or another, your model is flawed. If you're forced to resort to mixes of races, well, then you don't have any distinct race left.
Race concepts fall apart once actual taxonomic principles are applied to them.
Sort of. In a traditional hierarchical phylogenetic taxonomy, yes, race concepts fall apart. But if they don't necessarily fall apart with a cladistic genetic taxonomy.
Defining race is a classic problem of, well, classification. Put another way, it's like organizing books. Where do you place 'War and Peace'? In the fiction section? In the history section? In the classics section? In the russian literature section? It could legitimately be placed in any of those sections. The problem is that the book has a single physical instance. The book only exists in one place at one time. So, it can only be placed in one category at a time. And this is the problem with any phylogenetic based hierarchical taxonomy. It's not unique to race; it also applies to species, books, weblinks, and any other number of objects. It's why, before search engines, we had all these portal sites, like Yahoo!, who were focused on creating giant taxonomies of weblinks. And it was always a pain, because we had this intuition that a weblink should only exist in a single category at a time. This was a hold-over from library systems, where any particular book can only be placed on a single shelf at a time.
But then we discovered tagging. With tagging, a new type of taxonomy is possible, where a single entity can be placed in multiple categories at a time. And it turns out that tagging is equivalent to a genetic taxonomy. Each tag is equivalent to a gene (or meme, to be more precise). And we now give webpages lists of keywords, which function like a genome of sorts.
So, you're correct that race concepts fall apart at a hierarchical, phylogenetic based taxonomy. But with a genetic based taxonomy, race is 'tagged' by combination of genes... melanin count, lactose sensitivity, sickle-cell anemia, etc.
And what's more, this tagging and clustering, is a precursor to speciation. Consider the following simplified hypothetical example: a) mutant gene (A) interacts with the gene for lactose sensitivity such that, together, they cause a change in sperm mobility due to a lack of calcium, and b) another mutant gene (B) interacts with the gene for sickle-cell anemia such that, together, they cause a change in permeability to an egg due to lack of iron. If these two things were to hypothetically occur, it would make for a situation where sperm and egg couldn't unite, and a lactose intolerant father and sickle-cell anemic mother couldn't have children. Now then, one more consideration: say that these two mutant genes were actually very advantageous. Mutant gene A protects against flu and pnemonia, and mutant gene B codes for sexy pheremones. If these mutant genes are advantageous, then they'll spread throughout the population. But as the mutant genes spread through the population, the carriers of those genes, who also carry the genese for lactose intolerance and/or sickle cell anemia, would lose the ability to breed together. And this would be defined as a speciation event. Not only would those people be of different races, they would be unable to breed together, and would be different species.
Anyhow, it's worse than people fear. Not only does race actually exist, it's a precursor to speciation. Race just doesn't fit neatly into hierarchical phylogenetic taxonomies. Genetic taxonomies allow for overlapping, fuzzy boundaries. And that's exactly what Race is. Race doesn't fit into neat little hierarchical tree structures; rather, it's a fuzzy network of genes.