Slashdot Mirror


The Journal of Serendipitous and Unexpected Results

SilverTooth writes "Often, when watching a science documentary or reading an article, it seems that the scientists were executing a well-laid out plan that led to their discovery. Anyone familiar with the process of scientific discovery realizes that is a far cry from reality. Scientific discovery is fraught with false starts and blind alleys. As a result, labs accumulate vast amounts of valuable knowledge on what not to do, and what does not work. Trouble is, this knowledge is not shared using the usual method of scientific communication: the peer-reviewed article. It remains within the lab, or at the most shared informally among close colleagues. As it stands, the scientific culture discourages sharing negative results. Byte Size Biology reports on a forthcoming journal whose aim is to change this: the Journal of Serendipitous and Unexpected Results. Hopefully, scientists will be able to better share and learn more from each other's experience and mistakes."

18 of 153 comments (clear)

  1. So... by Biff+Stu · · Score: 4, Funny

    If the LHC generates an Earth-eating black hole, will it be published here?

    1. Re:So... by Asadullah+Ahmad · · Score: 5, Funny

      I don't think so. This Journal will not publish any results that were expected

  2. A great idea by al0ha · · Score: 4, Interesting

    but the obstacles are immense. Egos are massive and competition is fierce, so asking researchers to admit a mistake or give the competition a short cut is a tall order.

    --
    Did you ever wake up in the morning, with a Zombie Woof behind your eyes? -- FZ
    1. Re:A great idea by Cassius+Corodes · · Score: 5, Interesting

      I've published a paper with negative results before - there is no great pressure against it - and sometimes failing to re-create claimed results is big news. Perhaps the reason why people think negative results are not published as often is because you don't write "why my study was a big fat failure" - you report on the results you did get - why they are not conclusive / their limitations and what you think future researchers can do to improve on it. I.e. you turn what is ostensibly a failure into a win for science (not to mention a paper for you). I have read many such papers - so they are hardly uncommon.

      --
      Control is an illusion, order our comforting lie. From chaos, through chaos, into chaos we fly
    2. Re:A great idea by T+Murphy · · Score: 3, Interesting

      Given failed results wouldn't need as much verification, it may be possible for researchers to submit under pseudonyms to avoid embarrassment, and I should think not all researchers are so full of themselves to fear helping others. I agree we won't see the best stories reach this journal, but if nothing else it will be a good way for the honest, cooperative researchers to know they aren't alone.

    3. Re:A great idea by electrons_are_brave · · Score: 5, Interesting
      I agree - I have on occassion partially replicated previous research and failed to find anything significant. In psychology, at least, this is needed because so many people claim significant results on relatively small correlations (i.e. many psychs are bad at stats).

      Repeating the study on a different population and failing to find a significant result can also show that the results don't generalise to that population.

    4. Re:A great idea by Trepidity · · Score: 5, Interesting

      I agree (my first paper was a negative-results paper), but I think there are some kinds of negative results that are relatively hard to get published. Papers along the lines of: "here's an approach you might have thought would work, but it turned out that it didn't, and in retrospect we can see why, which this paper will explain". If you try to submit a paper like that, you often get push-back of, "oh well, yeah it's obvious why that wouldn't work, dunno why you didn't see it earlier". And of course it often is obvious once you've read why it doesn't work.

      As you point out, it's quite a bit easier to get negative results published if someone else had already claimed them as positive results. In that case, you're not both proposing and shooting down the idea simultaneously, but shooting down (or failing to confirm) someone else's idea, which has the advantages that: 1) you have evidence that at least one presumably smart person really didn't think it was obviously a bad idea (in fact, they thought it was a good one, and even that it worked); and 2) you're positioned as correcting an error in the literature, rather than as introducing a correction for a hypothetical error nobody has yet made.

      It's a bit tricky to fix, because some negative results really are obvious: it does nobody in the field any good to publish "we tried X on Y, and it didn't work", if genuinely nobody who was competent in the field would've thought X would work on Y, and the reason was exactly the reason you discovered.

      Incidentally, here's one previous attempt to start such a journal that didn't really get off the ground. Their one published article, which is quite good, is of the form I mention: the authors of a system called Swordfish recounted an idea they had to produce an improvement, Swordfish2, that in the end turned out to do be better than the original Swordfish. It was hard to get published elsewhere, because it wasn't correcting an existing result---nobody had previously proposed that doing what they tried to Swordfish2 was actually a good idea---but it's interesting (to me, at least) because it really does seem like a plausible idea, and I feel I learned something in reading why it didn't work.

    5. Re:A great idea by irp · · Score: 4, Interesting

      In my experience it has nothing to do with egos or competition.

      But it is damn hard to publish something that doesn't work!

      I was recently involved i developing a microfluidic system for diagnostics. Every milestone and sub-problem was solved. But when the final injection molded devices were tested, they failed due to an sort of interesting non-obvious combination of factors. Two issues with publishing this; the problems were very specific to our system and the conclusion could be written in 5 lines of text.

      It would have been like a movie with huge setup, but within the first 3 minutes the hero stumble, break his neck, and dies. End credits. It was a EU founded research project, no more money no more time. You can't get founding to continue a failed project. End of story.

      But my point is, in all my experience as scientist. I've never seen one of my colleagues say "we should hide this", but I've often heard "I would like to tell about this, but I don't know of a paper that would accept it".

      Also when something fails we need to carry on, but now we're behind schedule...

  3. Fantastic idea by nine-times · · Score: 4, Insightful

    Sometimes talking to people with very pro-sciene views, you get the idea that "science" is either an accumulated set of known facts or a perfect method which, because of peer review, is infallible at learning absolute truth.

    In reality, it's just a set of processes that we've developed and which has been generally more successful at producing helpful results than other methods. No reason to think that the way we go about it couldn't be improved. I can't imagine that failing to share the results failed experiments doesn't sometimes result in the loss of important information.

    Coincidentally I just saw this talk which raises the question whether helpful data can be gathered even if it's not gathered through conventional rigorous scientific methods. It seems like an interesting idea-- they're essentially gathering lots of data from various sources and using statistical analysis developed by economists to try to draw conclusions. My biggest concern would be purposeful manipulation by someone with an agenda.

    But anyway, all of this is to say that this has gotten me thinking about how the scientific process may still be open to some innovation.

  4. Re:This could be good by dgatwood · · Score: 5, Informative

    Penicillin? Not a mistake so much as general messiness. The guy stacked up some bacterial cultures and went on vacation. One of them grew mold. He noticed that the bacteria near the mold were dead.

    --

    Check out my sci-fi/humor trilogy at PatriotsBooks.

  5. Problem is by JanneM · · Score: 5, Insightful

    The problem with any change or reform of the publishing system is that publications are so important for the individual scientist. A paper isn't just a neat way to disseminate results. They are your work evaluation and your CV; they are keeping the score as it were. Where you publish and how often you publish directly determines where - or if - you work another year or two down the road.

    And even a short paper takes a lot of time and effort to write. For an informal "don't do that; we tried and it didn't work"-email to a colleague you could just jot down three or four paragraphs after lunch. Make a paper out of it and you have weeks or more of work ahead of you - looking up other previous published reports on the same kind of experiment; doing your best to figure out and explain the exact causes; square your (lack of) results with the apparent success of other groups that did something similar; make neat, clear graphs and illustrations as needed; get formal permission from your lab and your funding agency (and your co-authors labs and funding sources) to actually publish the thing. Then revise and edit the paper multiple times after comments from your co-athours and reviewers.

    So, getting good publications is vital for your ability to make rent and buy food for your family. Writing publications take a lot of time and effort - time that is pretty limited. So, even though the will to spread the word on a negative result may be there, chances is, writing it up will be relegated to the "when I've got a bit of spare time"-pile, where it will likely sit until well after retirement.

    --
    Trust the Computer. The Computer is your friend.
  6. Awesome! It's about time! by Fantastic+Lad · · Score: 3, Insightful

    This is a fantastic idea! It takes a great deal of strength to do this; one has to learn how to have fun and ignore the pangs of the ego.

    James Burke's Connections was based on similar philosophy. Non-linear thinking is a very powerful method of moving through time. Many geeks live in the clutches of an obsessive desire to control everything so that they don't get hurt by being wrong. If they could just relax and roll with the ups and downs and not be so hard on themselves, not care if they are laughed at, then they would find their power and perhaps start living lives of consequence.

    One university professor described an enormously powerful way of doing research; When you're up against a wall, seeking fruitlessly to find a specific title to continue your line of thinking, instead just pull out some random book nearby. Doesn't even have to be from the same shelf or Dewey code. It will have the answer. -But only if you're tuned to your inner Jedi.

    Those who deny their inner Jedi are forever lost. But the upside, I guess, is that nobody will laugh at them.

    -FL

  7. This is a great idea! by Dr_Ish · · Score: 4, Insightful

    One of the things that many scientists lack, is a good grounding in the Philosophy of Science. The public version of science, largely pushed by science teachers has an origin in the Vienna Circle of Logical Positivists. This is now largely known to be problematic, but is still the prevailing view. Folks should read Feyerabend's *Against Method* , or Ravetz's *Scientific Knowledge and Its Social Problems* for a more realistic view.

    As a scientist, I can also tell tales about how the scientific method gets distorted by ideology. When I was in grad school, I was working on a complex set of problems that were a horror -- a week doing eight hours a day pumping numbers into a scientific calculator is not my idea of fun. However, back then, it was a necessary evil. So, I was about to have to do another horror week with the calculator, which I did not want to do, so I was wasting time and did something silly. It turned out to be a great idea. It gave a whole new method to solve the problem type at hand. A number of other people had a hand in the final paper, but I got to be first author. Unfortunately, as only one author amongst many. The paper made claims about the hypotheses that was being tested, I objected very strongly to this -- there was no hypothesis, but we just got lucky. However, there is a paper with my name on in, published in the 20th Century, that contains claims about what we discovered which are false, at least with respect to hypotheses and all that stuff, in order to ensure that we were following someones idea of the scientific method. It irks me even today. Fortunately, a book about the issue now gives a more accurate account. However, there is no doubt that scientific ideology can drive out the truth. Thus, what is proposed here is a good idea. Telling the truth (even if it does not conform to the ideologically driven official method) is something I teach my grad students even today.

  8. Great scientists weren't very scientific by syousef · · Score: 5, Insightful

    Einstein wasted the last half of his life on wishful thinking "God does not play dice". Well turns out we're pretty sure he does. See Bell's theorum which shows that it can't just be hidden variables. And by all accounts for a theoretical physicist he sucked at advanced math.

    Isaac Newton was a horrible little man. Ill tempered, neurotic, and did wild experiments that he was lucky didn't blind him. Let's not forget the nastiness with Leibniz.

    Galileo had the social skills of a village idiot which led to the suppression of his work and his imprisonment by the authorities that he angered. (They were idiots too but that's beside the point)

    They're three of the greatest but I could go on.

    We like to pretend our scientists are great men with a couple of eccentricities that are way too smart to socialise or tolerate fools but the fact is their thinking isn't so superior OR logical OR scientific EXCEPT in their areas of expertise. THAT is why they are remembered. Not because they were above being unscientific.

    --
    These posts express my own personal views, not those of my employer
  9. failed experiment by InlawBiker · · Score: 4, Funny

    Don't date Wendy from the admissions office. Spectacular failure.

    1. Re:failed experiment by jamesh · · Score: 3, Funny

      And you are basing this on one datum? Have you learned NOTHING??? Go back and try again and see if you get the same outcome.

  10. Old Problem by RobinEggs · · Score: 3, Insightful

    Trouble is, this knowledge is not shared using the usual method of scientific communication: the peer-reviewed article. It remains within the lab, or at the most shared informally among close colleagues. As it stands, the scientific culture discourages sharing negative results.

    This sort of complaint goes back a very long ways, and it's certainly as good a time as any to address it head on.

    We have a habit in writing articles published in scientific journals to make the work as finished as possible, to cover all the tracks, to not worry about the blind alleys or to describe how you had the wrong idea first, and so on. So there isn't any place to publish, in a dignified manner, what you actually did in order to get to do the work, although, there has been in these days, some interest in this kind of thing.

    - Richard Feynman, Nobel Prize Acceptance Lecture, 1965

  11. Re:This IS a great idea, and the cause of our fail by Ruke · · Score: 3, Insightful

    You may joke, but facebook is a data-mining goldmine. Never before have advertisers had such free access to the personal lives of the very people they hope to sell their products to.