Slashdot Mirror
The Neuroscience of Screwing Up
resistant writes "As the evocative title from Wired magazine implies, Kevin Dunbar of the University of Toronto has taken an in-depth and fascinating look at scientific error, the scientists who cope with it, and sometimes transcend it to find new lines of inquiry. From the article: 'Dunbar came away from his in vivo studies with an unsettling insight: Science is a deeply frustrating pursuit. Although the researchers were mostly using established techniques, more than 50 percent of their data was unexpected. (In some labs, the figure exceeded 75 percent.) "The scientists had these elaborate theories about what was supposed to happen," Dunbar says. "But the results kept contradicting their theories. It wasn't uncommon for someone to spend a month on a project and then just discard all their data because the data didn't make sense."'"
The WIRED piece threads what is written in the summary around the story of how Arno Penzias and Robert Wilson at Bell Labs discovered Cosmic Radiation after being puzzled for a year about background noise on their radio telescopes ... even scraping pigeon poop off their gear as a possible source until they realized the signal was real - Homer Simpson would have said D'OH! ;-)
Hulk SMASH Celiac Disease
"It wasn't uncommon for someone to spend a month on a project and then just discard all their data because the data didn't make sense."
That doesn't mean the data is wrong, it means the /hypothesis/ was wrong, if not the theory, and needs to be modified.
If they're really throwing out date just because it 'doesn't make sense', they're doing religion, not science.
If the data don't make sense according to your theory, you don't discard the data, you discard the theory and work out a new one that fits the facts as you've observed them. TFA says that Dunbar was watching postdocs doing research, and if so, they should have known better. Alas, too many people who call themselves scientists are more interested in proving their pet theory true than in finding out what's actually going on.
Good, inexpensive web hosting
If problems occur as you postulate elaborate hypothesis, then stop piling up the elaborate hypothesis! But be sure and still make available your existing (complex) hypothesis, methodology and unexpected data - preventing others from going down the same path with the same methodology is still highly valuable!
Let's say you're looking at a production and consumption cycle involving neurotransmitters and neuroreceptors of some sort, and the various channels of input and output involved. Your starting presumption you base your hypothesis on is that there is a buildup which triggers an electrical signal to stop consumption and clear the channel. The only evidence you can realistically gather for now is protein density at a certain output channel - but others have worked to ensure this is a reliable approach specifically under these circumstances.
So, you do the specific experiment, trigger the signal, but you get a wildly different result - the stop in consumption occurs, but the protein density does not change at all in the output channel. What actually happened is still unknown, only you haven't verified any correlation with your hypothesis. You still have valuable data, but no mechanism to verify under the circumstances. Either your methodology failed, or you misunderstood what was happening - and the world of knowledge is made larger by either... even if your paymasters won't get happy about the result.
Science is often like throwing pebbles in complete darkness - it takes a lot of stones and close listening to make out a mental picture of the scene - especially when there's a lot of noise already around. Everyone would love it if we could just flip the lights on - but we have yet to invent a light that can see into the inner workings of the functioning brain very well. Gotta keep throwing those pebbles for now.
Ryan Fenton
I think the parent post is a brilliant example of what happens when someone perfects trolling to a science.
It takes a man to suffer ignorance and smile
Be yourself no matter what they say
The dirty little secret is that the Y is not always unexpected, just too politically incorrect and dangerous to be released to the public.
So, when reality is racist, you change it?
Is it just me or does this sound like an explanation for some of the Climategate science... But in that case they just massaged or ignored data that didn't agree with their conceptual framework of CO2 causing global warming.
Not that the skeptics are all that immune. They seem to cherry pick data almost as well (just not quite as successfully from the POV of selling their story to the media and political left ..)
I can't help but think that Neuroscience needs to calm down, sit back, and take a deep breath. We are examining a system and we are trying to reverse engineer it. We can't start out by trying to create elaborate hypothesis for large systems, we need to go low level and examine the simpler systems. I really think they should hold on to the higher cognitive models for a later time because we can't even completely model C. Elegans and it has the least neurons of any, current, living organism. The way I see it, I total expect their hypothesis to be wrong, because they don't thoroughly understand the low end of the system.
Eat sleep die
They'd get 42 dollars?
As a researcher myself, I certainly hope they don't throw out data too often. There is occasion to do so...sometimes, when trying to establish correlations (admittedly the weakest form of describing a phenomenon, etc), you learn that there is not one. There are times you obtain data that simply says, "These two phenomenon do not strongly affect each other" or "Something we do not know about or have not accounted for is happening all over this mess."
This data could be kept forever in the unlikely event it will prove useful, especially if there is something else going on...could be as simple as a RF/EM noise (which actually happened to a coworker of mine, though I helped to figure out the issue and make alterations to block/filter this noise out.) In previous years, data storage was sometimes at a premium, although lately this is not an issue as HDD climb to extraordinary capacities (until that capacity becomes the norm, then it is merely ordinary.) My point is that rejected or discarded data, at least in my experience, is due to situations such as these.
Things such as "massaging" or ignoring data are not only horribly bad scientific practice, they are a tremendous drag on humanity's progress...you usually learn through failure, but we are led away from the truth by practices such as those.
Proving a theory incorrect is often just as valuable as proving a theory correct.
I'd rather say proving a theory incorrect is just as valuable as proving a *hypothesis* correct. If it's a hypothesis, it's no fun proving it wrong (it wasn't established anyway, it might go against your intuition but nobody cares), and if it's a theory, it's no fun proving it right (what are you talking about, of course it's right, we already knew that).
I would elaborate on this but that would just be filler.
Or when the results you get aren't acceptable to the people responsible for continued funding.
Years ago, I worked for months trying to reproduce the Polywater research,
http://en.wikipedia.org/wiki/Polywater
and eventually reported that I was unable to do so.
The department considered my work a failure (as in, I must have been incompetent) and did not publish my findings. When, years later, the publications reporting successful discovery/creation of Polywater were shown to be fraudulent, and my results were correct, I did not even receive an apology.
Throwing out results is unethical as well as irresponsible. Many discoveries have come from re-evaluating what appears to be "bad" data. It might not be possible to use it now, but it should be at least stored.
For instance, it has been reported that the "bit of "scruff" on her chart-recorder papers that tracked across the sky with the stars"[1] looked like bad data to Jocelyn Bell Burnell's supervisors. Today we call the phenomenon a pulsar.
[1] Wikipedia
If the data doesn't fit your theory, the problem is most likely neither with the data (which is fine) nor with your theory (which may also be fine) but with the method you used to produce your data. You probably wired in an incorrect resistor, forgot to close a parenthesis in your Perl code, forgot to add the correct amount of EDTA to your reaction, etc. Then your results ended up looking like shit, and not surprisingly. Doing science is hard.
There's no need to postulate any grand conspiracies or take pot-shots at science in general. This paper is examining real people doing real shit. Most of the time we fuck up, and we're not smart enough to figure out where we made the error.
Indeed. The sort of thing being discussed in TFA is one of the classic themes of late 20th century philosophy and history of science: the disconnect between traditional philosophy of science and the actual practice of science.
Kuhn's Structure of Scientific Revolutions is a good place to start. Just one tiny example of the book: Kuhn goes on about how during normal science, scientists perform experiments to confirm the results that they expect to get. When an experiment contradicts the theory, they don't automatically assume that the theory is wrong; on the other hand, they assume that the experiment was flawed.
Feyerabend and many other philosophers of science take a complementary stand to this by stressing the theory-ladenness of "facts." The claim that the "facts" contradict a hypothesis is never a theory-independent observation, but rather, the conclusion of a different theory that we may overthrow. Feyerabend's classic example is the Tower Argument that Aristotle used to refute the theory that the Earth moves. Wikipedia's article on Paul Feyerabend has a decent, if terse, explanation of this:
Feyerabend goes on to argue that many of our most successful contemporary scientific theories (e.g., heliocentrism and geodynamicism) became so because their Renaissance and Enlightenment proponents held on to them and continued to elaborate on them despite them being contradicted by "the facts," as judged by the application of theories that were better established at the time (e.g., Aristotelian mechanics). That is, new scientific theories often succeed because their proponents keep working on them and improving them despite being contradicting by the "facts"; then as the new theories become stronger and better accepted, people start juding the "facts" by the lens of the new instead of the old, and forget the problems that the new theories were judged to have and never resolved (e.g., things like Newtonian physics not having the same explanatory range as Aristotelian physics).
Are you adequate?
Really? Or is it that you are SO politically correct that you cannot see truth.
I happen to have mod points and my on-the-fly ranking went from insightful to interesting to troll and back to interesting.
I've lived long enough to understand that each of the 6 billion people on this earth is different than every other. Some are remarkably good and some are remarkably bad. Most of us are just average in our own interesting ways.
But still, I do believe that genetic differences affect what we are and that genetic differences can be attributed to where our genes came from.
Those who choose to never risk offending anyone are perhaps the most intellectually dishonest among us.
Should I post this or should I mod YOU the troll?
in my experienced - I'm a physical chemist doing atomic resolution condensed phase computer modeling. It's so common that I am troubled when the first analysis gives the answer I expected. I likely spend more time looking for errors when the answer makes sense the first go through. Really.
46 & 2
"Those who choose to never risk offending anyone are perhaps the most intellectually dishonest among us."
All fine and good, except the OP does not contain anything more intellectual than a bunch of bald assertions wrapped in the emotions of a xenophobe. In other words, you should have modded the GP informative, the OP is a well formed troll.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
Are groups of people from very different locations, such as whites and blacks, different? Of course they are! Not so different as to be separate species (yet, and with global communication, maybe never), but evolved in different directions to adapt to different conditions. Just being different means there are activities for which one group will be better suited than the other group. There has to be, otherwise they aren't different, are they? Get over both the racism and the political correctness, and admit this basic fact. Skin color is a fairly superficial difference. Africa has by far the most diversity-- the Eastern African perhaps has less in common with the Western African or the Pygmy than with whites.
Much more harmful is the tendency to oversimplify fitness to a single, grossly over broad measure of a difficult to define concept that is not universally relevant, such as IQ, and declare one group superior to another based on only that. When a declaration of superiority is made, you may be sure it is for purposes of propaganda. Geniuses (defines as people with IQ > 140, or perhaps > 160) make their share of fatal mistakes, have flaws that can render their supposed advantages much less valuable, take risks and sometimes lose, sometimes let success go to their heads (most recently, Mike Leach), just like everyone else. Bobby Fischer was a genius, but that monomania which made him able to be World Chess Champion hurt him in so many other ways. The Soviet Union really bought into the idea of chess (and other contests such as the Olympics) as a good measure of a society's fitness, and devoted so much effort to it that except for Fischer, they pretty much mopped the boards with other nations' best players. And what was it all for? Propaganda that ultimately proved empty when Communism collapsed. I expect the Space Shuttle astronauts all do very well on IQ tests, but is that a smart gamble, risking their lives on that thing, for the fame and money they get? For other sorts of fitness, there are many fantastic athletes, rock stars, leaders, and the like who ended tragically. A small change in conditions can at a stroke reverse the fitness of most any trait.
Intellectual Property is a monopolistic, selfish, and defective concept. It is "tyranny over the mind of man"
But still, I do believe that genetic differences affect what we are and that genetic differences can be attributed to where our genes came from.
The theory that race has nothing to do with intelligence has nothing to do with political correctness, and all with science: specifically, the scientific discovery that the taxonomy of human races is not definitive, not specific and has no basis in genetics. Which in turn means that the ggp's assertion that race was a statistically significant factor in their research means that their research was utter crap to begin with.
So let me ask you this then: what makes you think that race is the same as genetics, or that you can even reliably a race? I mean, outside of some outdated and non-scientific notions of physiognomy and phrenology?
Those who can, do. Those who can't, sue.
The idea that race is a fiction is a bad, well, fiction, and a clear example of the distortion of thought due to political correctness.
There are a number of human traits (and the genes which cause them) that statistically cluster into groups that correspond to what we consider race. You can test a person's DNA and determine their racial heritage, to a fairly accurate degree. Obviously race is real, if you can nearly automate measuring it. The fact that statistical clusters don't have firm boundaries doesn't mean those clusters don't exist.
Is race relevant? Not for most purposes, but it is for some. I understand that Asians are more likely to have difficulty digesting milk, for example; blacks have a higher tendency to have sickle-cell anemia. Declaring that any test that shows a tendency for races to vary based on genetics is CERTAIN to be flawed because you don't believe race exists is ludicrous.
It's pretty rare for everything to go right.
I work with holography. I shine a laser at a piece of film, then develop the film. And presto, I get no image. Do I throw out the theory that exposing film to light should produce an image? No, I assume that I screwed up and go back and start again. It's not uncommon for me to spend 3 months of cleaning, aligning, measuring and so on until I produce a proper image. I then throw away all the "bad" data. Maybe, theoretically, that data could be useful, but there's too many parameters to account for.
"It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with experiment, it's wrong."
- Richard Feynman
"The most exciting phrase to hear in science, the one that heralds new discoveries, is not Eureka! but rather, "hmm.... that's funny...."
- Isaac Asimov
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
Don't think the summary quite found the central point of TFA.
"Dunbar found that most new scientific ideas emerged from lab meetings, those weekly sessions in which people publicly present their data. Interestingly, the most important element of the lab meeting wasn't the presentation -- it was the debate that followed. Dunbar observed that the skeptical (and sometimes heated) questions asked during a group session frequently triggered breakthroughs, as the scientists were forced to reconsider data they'd previously ignored. The new theory was a product of spontaneous conversation, not solitude; a single bracing query was enough to turn scientists into temporary outsiders, able to look anew at their own work."
"I saw this happen all the time," Dunbar says. "A scientist would be trying to describe their approach, and they'd be getting a little defensive, and then they'd get this quizzical look on their face. It was like they'd finally understood what was important."
So that's it: The keys are multiple viewpoints, skepticism, and intellectual competitiveness.
We know where leadership by an anti-intellectual "strongman" who scapegoats minorities and likes boisterous rallies goes
I am calling this neuroscience because it has nothing to do with how the nervous system operates. In this sense I am following the lead of WIRED and/or Dunbar, who can't tell a neuro from a social. From TFA: "Kevin Dunbar is a researcher who studies how scientists study things". OK, he studies things called scientists. scientists are people. The study of people and how they behave is psychology. Science is a social activity. Investigations of social activities are sociology when taken as a whole, or social psychology when considered in terms of the activities of individuals operating within a social group. Dunbar studied social psychology, not neuroscience. There's not a speck of neuroscience cereal in it anywhere. There's very little if any actual social psychology, and psychology, or any science at all. There's talking about science, there's talking to scientists about doing science, and there's watching them do science. There's watching and talking about getting good results and not getting good results, and what people do in the matter case. If Dunbar thinks he's doing neuroscience, I suspect he's not even very clear on science itself, much less the various branches. And it does say he's "a researcher in", not that he's a scientist. I do research in curry recipes from different countries and cultures. I'm a researcher, but not a cultural curriology scientist.
In fact I'll go s far as to say he's a researcher because he knows precious little and is trying to find out basic things, not as is the case with most scientists, someone who knows a fair amount and is trying to build on that with new knowledge. He is apparently not clear on the difference between 'screwing up' and not getting good and/or clean results. This may well be because he was unclear himself as to what it was he was looking at and talking about, and he thought he was just not getting good or clean results, when actually, guess what?
He doesn't let loose any secrets. Anyone can talk to scientists and as what happens if and when things don't turn out as expected. If you get an honest (ie. less concerned with appearances than truth) scientist, anyone would get the same answers. Or one could simply read work from real social psychologists and others who study science and scientists and learn the same things. I myself always recommend Collin's & Pinch's "The Golem" as an illuminating, instructive and entertaining starting point.
And a technical point on methodology: a study that does not find a difference between groups, treatments, whatever, 'fails to reject the null hypothesis' (the assertion that there is no observable difference). It does not prove there is no difference, it merely fails to find one. It fails, but only to find a difference, not to produce a result. It can't say there is no difference, it can only say that it couldn't find one. And, it fails to find a difference, no matter how nicely or hapazardly the data come out. The only studies that "fail" produce no data. Scientists may further fail to find an interpretation, but there's no limitation on trying to figure this out, and it applies to both 'results' (reject null hypothesis) and 'no results' (fail to reject null). Studies that produce data that 'makes no sense' produce data that fails to reject the null. The 'making no sense' is a post hoc evaluation of the data based on an incomplete understanding of the design, collection, analysis or interpretation. Such evaluations are done in science, but they are not part of the scientific process. Therefore when this occurs, it is not a "scientific" result and cannot be taken to reflect in the nature or quality of the work done. If you can't figure what it means, you can't figure out. You cannot say that since you cannot figure it out, then you figure out that it fails. If you think you can take something that 'doesn't make sense' and then say that it makes sense in that it represents a failure, then you've contradicted the assertion that it makes no sense. All you can say is that you don't understand it, and since you d
"I may be synthetic, but I'm not stupid." -- Bishop 341-B
Here's the problem. If you can't order every single human into one race or another, your model is flawed. If you're forced to resort to mixes of races, well, then you don't have any distinct race left.
Race concepts fall apart once actual taxonomic principles are applied to them. Your examples actually illustrate the problem quite nicely: not nearly all asians have problems with milk - specifically the Japanese the do. Indians (from the Indian subcontinent in Asia) do not. Blacks do not have a higher tendency for sickle-cell anemia, a certain group of people in Africa do. Blacks in the US do not have that trait.
How much does it suck to be so wrong? Your cognitive dissonance must be at a record high.
Those who can, do. Those who can't, sue.
Indeed!
No scheme of inequality can be defended as corresponding to natural fact.... Superior and inferior can be determined only with respect to a single quality for a single purpose. Nor can a man's qualities be added together and averaged to give a final score or merit. In short, men are incommensurable and must be deemed equal.
- Jacques Barzun
Your brain is not a computer.
Which is why people who claim scientists only care about the truth are wrong. You only cared about the truth and were fired. Plenty of other people would have been looking out for their job first, and made sure their results confirmed what the department expected.
That said- the great thing about science is that eventually the truth will be discovered despite the pressure for money/jobs. It may not happen in a lifetime, but as long as science continues, it will happen.
Here's the problem. If you can't order every single human into one race or another, your model is flawed. If you're forced to resort to mixes of races, well, then you don't have any distinct race left.
Race concepts fall apart once actual taxonomic principles are applied to them.
Sort of. In a traditional hierarchical phylogenetic taxonomy, yes, race concepts fall apart. But if they don't necessarily fall apart with a cladistic genetic taxonomy.
Defining race is a classic problem of, well, classification. Put another way, it's like organizing books. Where do you place 'War and Peace'? In the fiction section? In the history section? In the classics section? In the russian literature section? It could legitimately be placed in any of those sections. The problem is that the book has a single physical instance. The book only exists in one place at one time. So, it can only be placed in one category at a time. And this is the problem with any phylogenetic based hierarchical taxonomy. It's not unique to race; it also applies to species, books, weblinks, and any other number of objects. It's why, before search engines, we had all these portal sites, like Yahoo!, who were focused on creating giant taxonomies of weblinks. And it was always a pain, because we had this intuition that a weblink should only exist in a single category at a time. This was a hold-over from library systems, where any particular book can only be placed on a single shelf at a time.
But then we discovered tagging. With tagging, a new type of taxonomy is possible, where a single entity can be placed in multiple categories at a time. And it turns out that tagging is equivalent to a genetic taxonomy. Each tag is equivalent to a gene (or meme, to be more precise). And we now give webpages lists of keywords, which function like a genome of sorts.
So, you're correct that race concepts fall apart at a hierarchical, phylogenetic based taxonomy. But with a genetic based taxonomy, race is 'tagged' by combination of genes... melanin count, lactose sensitivity, sickle-cell anemia, etc.
And what's more, this tagging and clustering, is a precursor to speciation. Consider the following simplified hypothetical example: a) mutant gene (A) interacts with the gene for lactose sensitivity such that, together, they cause a change in sperm mobility due to a lack of calcium, and b) another mutant gene (B) interacts with the gene for sickle-cell anemia such that, together, they cause a change in permeability to an egg due to lack of iron. If these two things were to hypothetically occur, it would make for a situation where sperm and egg couldn't unite, and a lactose intolerant father and sickle-cell anemic mother couldn't have children. Now then, one more consideration: say that these two mutant genes were actually very advantageous. Mutant gene A protects against flu and pnemonia, and mutant gene B codes for sexy pheremones. If these mutant genes are advantageous, then they'll spread throughout the population. But as the mutant genes spread through the population, the carriers of those genes, who also carry the genese for lactose intolerance and/or sickle cell anemia, would lose the ability to breed together. And this would be defined as a speciation event. Not only would those people be of different races, they would be unable to breed together, and would be different species.
Anyhow, it's worse than people fear. Not only does race actually exist, it's a precursor to speciation. Race just doesn't fit neatly into hierarchical phylogenetic taxonomies. Genetic taxonomies allow for overlapping, fuzzy boundaries. And that's exactly what Race is. Race doesn't fit into neat little hierarchical tree structures; rather, it's a fuzzy network of genes.
Blacks do not have a higher tendency for sickle-cell anemia, a certain group of people in Africa do. Blacks in the US do not have that trait.
ORLY? The US Government says:
...making your closing amusingly ironic:
How much does it suck to be so wrong? Your cognitive dissonance must be at a record high.
Dewey, what part of this looks like authorities should be involved?