The Journal of Serendipitous and Unexpected Results
SilverTooth writes "Often, when watching a science documentary or reading an article, it seems that the scientists were executing a well-laid out plan that led to their discovery. Anyone familiar with the process of scientific discovery realizes that is a far cry from reality. Scientific discovery is fraught with false starts and blind alleys. As a result, labs accumulate vast amounts of valuable knowledge on what not to do, and what does not work. Trouble is, this knowledge is not shared using the usual method of scientific communication: the peer-reviewed article. It remains within the lab, or at the most shared informally among close colleagues. As it stands, the scientific culture discourages sharing negative results. Byte Size Biology reports on a forthcoming journal whose aim is to change this: the Journal of Serendipitous and Unexpected Results. Hopefully, scientists will be able to better share and learn more from each other's experience and mistakes."
If the LHC generates an Earth-eating black hole, will it be published here?
but the obstacles are immense. Egos are massive and competition is fierce, so asking researchers to admit a mistake or give the competition a short cut is a tall order.
Did you ever wake up in the morning, with a Zombie Woof behind your eyes? -- FZ
No longer having to remake the broken wheel each time. Or it could lead to a bad side effect having a positive outcome like Viagra and Zyban. Both of these were not what was planned but had amazing results. Hell, penicillin saves millions and if I remember right, was a total mistake at the beginning.
Attention... all grammer nazi"s! Is they're anything; wrong with: my post,
If we ONLY publish and peer review our successes, our failures and errata is discarded.
In this data could be a LOT of really amazing potential, given peer review and continuance.
No wonder we don't have a theory of everything yet, we're not looking at nearly all the data.
Sometimes talking to people with very pro-sciene views, you get the idea that "science" is either an accumulated set of known facts or a perfect method which, because of peer review, is infallible at learning absolute truth.
In reality, it's just a set of processes that we've developed and which has been generally more successful at producing helpful results than other methods. No reason to think that the way we go about it couldn't be improved. I can't imagine that failing to share the results failed experiments doesn't sometimes result in the loss of important information.
Coincidentally I just saw this talk which raises the question whether helpful data can be gathered even if it's not gathered through conventional rigorous scientific methods. It seems like an interesting idea-- they're essentially gathering lots of data from various sources and using statistical analysis developed by economists to try to draw conclusions. My biggest concern would be purposeful manipulation by someone with an agenda.
But anyway, all of this is to say that this has gotten me thinking about how the scientific process may still be open to some innovation.
I hope they get some courage and post even the embarrassing results and experiments. Will remove the illusion that "Scientists" and "stupidity" can't appear in one sentence.
The problem with any change or reform of the publishing system is that publications are so important for the individual scientist. A paper isn't just a neat way to disseminate results. They are your work evaluation and your CV; they are keeping the score as it were. Where you publish and how often you publish directly determines where - or if - you work another year or two down the road.
And even a short paper takes a lot of time and effort to write. For an informal "don't do that; we tried and it didn't work"-email to a colleague you could just jot down three or four paragraphs after lunch. Make a paper out of it and you have weeks or more of work ahead of you - looking up other previous published reports on the same kind of experiment; doing your best to figure out and explain the exact causes; square your (lack of) results with the apparent success of other groups that did something similar; make neat, clear graphs and illustrations as needed; get formal permission from your lab and your funding agency (and your co-authors labs and funding sources) to actually publish the thing. Then revise and edit the paper multiple times after comments from your co-athours and reviewers.
So, getting good publications is vital for your ability to make rent and buy food for your family. Writing publications take a lot of time and effort - time that is pretty limited. So, even though the will to spread the word on a negative result may be there, chances is, writing it up will be relegated to the "when I've got a bit of spare time"-pile, where it will likely sit until well after retirement.
Trust the Computer. The Computer is your friend.
an article for the first issue? covers all questions
So are we talking:
Technique X fails on problem Y.
Sue all music downloads fails to stop piracy?
Hypothesis X can't be proven using method Y.
All music downloaders can't be proven to be pirates
Protocol X peforms poorly for task Y.
Suing all music downloaders performs poorly for stopping music piracy
Method X has unexpected fundamental limitations.
Forcing people to buy music only on CD / Tape / Vinyl doesn't appeal to all customers
While investigating X, you discovered Y
While trawling torrent log files for music pirates we also found some great porn
Model X can't capture the behavior of phenomenon Y.
Current Music Business Model can't capture the behaviour of generation Y
Failure X is explained by Y.
Failing to increase revenue is explained by $0.99 tracks on Apple (damn iPod users) an music pirates too (arrrgh!)
Assumption X doesn't hold in domain Y.
Assuming independant music stores will be profitable doesn't hold in the .com domain
Event X shouldn't happen, but it does.
People shouldn't want to listen to music for free (damn radio stations, ipods, internet)
This is a fantastic idea! It takes a great deal of strength to do this; one has to learn how to have fun and ignore the pangs of the ego.
James Burke's Connections was based on similar philosophy. Non-linear thinking is a very powerful method of moving through time. Many geeks live in the clutches of an obsessive desire to control everything so that they don't get hurt by being wrong. If they could just relax and roll with the ups and downs and not be so hard on themselves, not care if they are laughed at, then they would find their power and perhaps start living lives of consequence.
One university professor described an enormously powerful way of doing research; When you're up against a wall, seeking fruitlessly to find a specific title to continue your line of thinking, instead just pull out some random book nearby. Doesn't even have to be from the same shelf or Dewey code. It will have the answer. -But only if you're tuned to your inner Jedi.
Those who deny their inner Jedi are forever lost. But the upside, I guess, is that nobody will laugh at them.
-FL
Most failed results have no useful knowledge in them. Having a huge amount of them is less useful that it sounds.
One of the things that many scientists lack, is a good grounding in the Philosophy of Science. The public version of science, largely pushed by science teachers has an origin in the Vienna Circle of Logical Positivists. This is now largely known to be problematic, but is still the prevailing view. Folks should read Feyerabend's *Against Method* , or Ravetz's *Scientific Knowledge and Its Social Problems* for a more realistic view.
As a scientist, I can also tell tales about how the scientific method gets distorted by ideology. When I was in grad school, I was working on a complex set of problems that were a horror -- a week doing eight hours a day pumping numbers into a scientific calculator is not my idea of fun. However, back then, it was a necessary evil. So, I was about to have to do another horror week with the calculator, which I did not want to do, so I was wasting time and did something silly. It turned out to be a great idea. It gave a whole new method to solve the problem type at hand. A number of other people had a hand in the final paper, but I got to be first author. Unfortunately, as only one author amongst many. The paper made claims about the hypotheses that was being tested, I objected very strongly to this -- there was no hypothesis, but we just got lucky. However, there is a paper with my name on in, published in the 20th Century, that contains claims about what we discovered which are false, at least with respect to hypotheses and all that stuff, in order to ensure that we were following someones idea of the scientific method. It irks me even today. Fortunately, a book about the issue now gives a more accurate account. However, there is no doubt that scientific ideology can drive out the truth. Thus, what is proposed here is a good idea. Telling the truth (even if it does not conform to the ideologically driven official method) is something I teach my grad students even today.
If we ONLY publish and peer review our successes, our failures and errata is discarded.
In this data could be a LOT of really amazing potential, given peer review and continuance.
No wonder we don't have a theory of everything yet, we're not looking at nearly all the data.
Hmm. With the additions of Facebook and Twitter, the web is now essentially a compendium of 99% failures and errata. And according to your hypothesis, we should still find some amazing potential in it. I think you're on to something.
John
A refreshing perspective in a world of agenda oriented science. Anyone who's ever had a hunch that turned up false knows such disappointment. Less we forget that the spirit of science is about discovery, knowledge and truth first. Being right, dead last.
Einstein wasted the last half of his life on wishful thinking "God does not play dice". Well turns out we're pretty sure he does. See Bell's theorum which shows that it can't just be hidden variables. And by all accounts for a theoretical physicist he sucked at advanced math.
Isaac Newton was a horrible little man. Ill tempered, neurotic, and did wild experiments that he was lucky didn't blind him. Let's not forget the nastiness with Leibniz.
Galileo had the social skills of a village idiot which led to the suppression of his work and his imprisonment by the authorities that he angered. (They were idiots too but that's beside the point)
They're three of the greatest but I could go on.
We like to pretend our scientists are great men with a couple of eccentricities that are way too smart to socialise or tolerate fools but the fact is their thinking isn't so superior OR logical OR scientific EXCEPT in their areas of expertise. THAT is why they are remembered. Not because they were above being unscientific.
These posts express my own personal views, not those of my employer
Don't date Wendy from the admissions office. Spectacular failure.
LED flashlights?
Redundant.
Well, duh.
SB
It's old. The more humans I meet, the more I like my cats. At least they are honest.
Maybe on that list of things that "not work" are things that never worked because the experiment was not well designed.
Is my undestanding that the democracy world is better because we don't firmly control what people experiment. So people are free to try things that "don't work".
-Woof woof woof!
Trouble is, this knowledge is not shared using the usual method of scientific communication: the peer-reviewed article. It remains within the lab, or at the most shared informally among close colleagues. As it stands, the scientific culture discourages sharing negative results.
This sort of complaint goes back a very long ways, and it's certainly as good a time as any to address it head on.
We have a habit in writing articles published in scientific journals to make the work as finished as possible, to cover all the tracks, to not worry about the blind alleys or to describe how you had the wrong idea first, and so on. So there isn't any place to publish, in a dignified manner, what you actually did in order to get to do the work, although, there has been in these days, some interest in this kind of thing.
- Richard Feynman, Nobel Prize Acceptance Lecture, 1965
OK, I have an idea for an article. "Experiments on Seduction: Why 'Just Be Yourself' Is a Bad Idea. Personal Recounts and Anedoctes." Abstract: Friends and relatives often dispense wisdom on the subject of seduction of members of the opposite sex by stating "Just be yourself". In this paper we provide evidence for the failure of this conventional wisdom and provide alternate explanations to its failure. Who wants to co-author?
My sig is better than your sig.
Such a journal already exists for biomedicine:
http://www.jnrbm.com/
You may joke, but facebook is a data-mining goldmine. Never before have advertisers had such free access to the personal lives of the very people they hope to sell their products to.
Troll.
Ofcourse.
---
History of Science Feed @ Feed DIstiller
The Journal of Unreproducible Results was created in the 60's (really) and the name has become a very common joke among scientists ever since :-)
Engineers had similar reports long ago, and they are the most interesting stuff you can find. An educated investigation of failure is much more important for me (for studying purposes), than pure success. There are many interesting ways to fail. Just look at industrial accidents.
Great series. I used to read his column, the one that sticks in my mind connected the radius of the space shuttle booster rockets to the width of a horses arse. I don't think real geeks are affraid of what people think, just that inquiring minds are obsessed with knowing.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
Yeah, we've all seen the cartoon but the scientfic method does not claim to offer a method for creativity, to do so would be tautological. What the scientific method offers is a useull way to test the fruits of creativity. The bad fruit tends to complain that there are no instructions for creativity.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
never attribute to serendipity what can be explained by science
If Pandora's box is destined to be opened, *I* want to be the one to open it.
I can't help but remember Sony founder explaining how they were looking for ways of doing efficient small transistors with various materials and that they had learned from Bell labs that silicium gave very poor result so they spent minimum resources on that.
I can't help also wonder if this is a good use of "peer reviewing" which has a kind of shortage, or so I heard.
The Wise adapts himself to the world. The Fool adapts the world to himself. Therefore, all progress depends on the Fool.
In addition there is often a lot of benefit in working things out for yourself - this provides the in depth understanding to base deeper work on which can be lacking if merely following instructions...
In theory, there's no difference between theory and practice; in practice there is.
I love how people often point out "you can't prove a negative" or "you can't publish negative results". Turns out that you are very wrong if you think that either is true.
At first sight it appears that the idea behind this journal is to share failed attempts. But look at the kind of examples the website would like you to prove: "Prove that method X does not work for problem Y." This is *not* a failed attempt. You succeeded in proving something. Some great papers dealing with P?=NP problem prove exactly this kind of thing. How about the proof that you can't put real numbers into a list and so they are uncountably many?
The usual problem with a "failed attempt" is that something does not work the way you had hoped for. Not that you discover that something won't work generally. Those latter kind of statements require much more sophistication to prove.
Proving the negation of X is not the same as not being able to prove X, and vice-versa.
This idea was already executed a while ago by the Journal of Negative Results in Ecology and Evolutionary Biology, the Journal of Negative Results in BioMedicine, the Journal of Negative Results in Speech and Audio Sciences and probably a few others that Google will help you find, just as it helped me find. But, as I recall, even PLoS had publishing negative results in its charter and specifically PLoS ONE encouraged them, being all-inclusive.
The problem? Most of them (except for PLoS and PLoS ONE) have a very low impact factor because although negative results are important, they aren't sexy in the least. If they were sexy, they would have been published in more mainstream journals. Because publishing a paper requires significant effort, a scientist is unlikely to spend his most precious resource -- time -- publishing a negative result if he can publish a positive one. Positive results get referenced, negative ones, by-and-large, do not. References in important journals lead to advancement as a scientist through grants, promotion, etc. So, unless the result is going to have significant impact -- like contradicting a previous result, or disproving dogma -- there's little motivation for a scientist to expend the effort to write up and publish a negative result, rather than do more research.
Put my fist through my alarm clock with its ding-dong death inside my ear. - The Blackjacks.
... that explains how the whole Mentos+soda thing was actually a failed attempt at cold fusion?
CUR ALLOC 20195.....5804M
Electron Band Structure In Germanium, My Ass
the preceding comment is my own and in no way reflects the opinion of the Joint Chiefs of Staff
A pretty good idea... The Cosmic Microwave Background was first discovered by 2 engineers who thought they had bird shit on their receiver... Humble beginnings.