Slashdot Mirror
Journal Devoted to the Null Hypothesis
Xcott R13, 3(0,R4) writes "It may sound dull even for academia, but I personally am thrilled that someone is starting a journal devoted entirely to scientific research that fails to produce significant results. Researchers tend to publish successes, so we rarely ever read about experiments or approaches that didn't pan out, leaving future researchers to reinvent the square wheel. The "Journal of Articles in Support of the Null Hypothesis" intends to make some of this valuable boring information available. And such a wonderful title: too bad it's an online journal, else I could put it on the bookshelf next to the Annals of Improbable Research. Causing an explosion that would destroy the Universe."
I would be interested in hearing a response to this from someone in the field. Is there someone in the slashdot community who is distinguished in one of the natural sciences, who would care to comment? Thanks.
Karma: Good (despite my invention of the Karma: sig)
No, I tried this and the universe wasn't destroyed. See last month's issue for details.
Will this be the "ugly fat girl" of the world of science?
"Sorry to hear it didn't pan out, there's always that null hypothesis journal"
...and this lie crawls out of its mouth: 'I, the state, am the people.'
I am a grad student in molecular biology, and I've learned the hard way how nice this kind of journal would have been. I wrote a research proposal about something that, when it showed up in front of my committee, whas dismissed out of hand. Turns out that line of research ran into a dead end five years ago. People don't go back and publish "oh, btw, we were wrong". I, and apparently my advisor who gave me the topic, were too dumb to pick up that this was a discontinued line of research.
On the other hand, it would be a terminally dull read. And people would probably be afraid to publish in it, thinking it would hurt their career to be openly associated with failure.
Entropy gets everyone.
Not getting an answer is as important as getting one.
"Well this expirement/proof did nothing, so lets try something else."
And scientific progress moves on. Just like all the failed proofs of Fermat's Last Therom.
Feminism is the radical notion that women are people.
The whole `journal system' is flawed. It's goal is to *prevent* the acceptance of `ridiculous' ideas presented by `upstarts', i.e. anything the Big `Profs'---who had not a single original idea in the past two decades---will not like to see published, specially because it would (1) show that the `Profs' themselves are not doing much useful science, and (2) it would force the `Profs' to `upgrade' (or get dumped). NO NO NO! If the Profs don't like your ideas, you are NOT to be published, ever. Period. No publications means no funds, no appointments, you get the picture.
The ScientificCommunity(TM)---which is neither---wants no `upstarts' making ripples in the water. They impede progress. As the saying goes: academic politics are the worst kind, because the stakes are so low.
In the world of ModernScience(TM) you either conform or are crushed. And the best way for the `Profs' to have this power is the closed journal system. Why do you think scientists are the very LAST ones to use the Web, the very LEAST inclined to make the proccess transparent for the public to see? When was the last time you knew which papers were rejected and why? Never?---I thought so? Ever wonder *why*?
That's why they insist in what they call `peer review', which again is neither. The `reviewers' are NOT your peers, but the editor's choice, IF it is the all-powerful editor's whim to even grant that to the author. There is no accountability.
REAL science is pushed to be published in what (yet again) the Profs call ``grey literature'' (meaning what the Profs do not endorse) or forever keep your peace.
Sounds too harsh. Think again, check the history of science. All good ideas are rejected for 30 years, their authors ridiculed and their careers (an sometimes their very lives) wrecked, and *then* some of the Profs *discovers* your idea and mekes it his own, after you have been crushed. Think Wegener and plate tectonics. Think Semmelweiss and aseptical surgical practice. Think Plank and quantum physics. Think Mendel and genetics. (And try to find out about the ones who are being crushed this very moment, and of their much-needed ideas).
Don't like it. Well, do *something* about it.
Richard Feynman was a very strong advocate of having somewhere to publish non-positive results. He was distressed by the credibility added to the existence of psychic powers, influence of star signs, alien abductions and so on, by scientific research. It arises from the fact that if you test your hypothesis to, say, the 5% level, then on average one in twenty researchers is going to get a false positive. People tend to publish the positive results, and the nineteen negative results get forgotten because they're boring, giving a misleading picture. Of course, all the research could be completely accurate, but the skewed statistics make that irrelevant.
So Feynman dearly wanted a Journal of the Null Hypothesis. I think I found that in "Surely You're Joking, Mr. Feynman".
Any sufficiently self-referential snowcloned
So such a journal needs to have two aspects. One a list of things that won't work and why the person that tried them thought they would to reduce reinventing square wheels and the other is a critical examination/explanation/"proof" of why it wouldn't work. Those experiments that don't have the latter stand as possible areas for great gain to humanity.
Read the Journal's website, folks -- it's a psych journal, not a mol-bio/physics/hard-science journal. Most of the posts here are getting this wrong.
This is in fact the reason why this journal is such a great idea. As a social science, the field of psychology has a much greater problem than fields like physics with dubious positive experiments getting overhyped -- the media will hype the one study that says the Internet turns kids into axe murders, but it doesn't mention the 99 other studies that found no relation.
Feynmann, in fact, wrote an article called 'Cargo-Cult Science', in which he attacked the discipline of psychology for not repeating experiments to check old results. Yes, he would 100% approve of this new journal.
...consist of someone finding a correlation between some variables. They'll publish the results if they get something 95% or 99% significant. This means that if you do 20 experiments, even if there's no underlying phenomenon you still expect to get one publishable paper by chance. And of course we only see the published papers so we get a very skewed view. By publishing failed experiments we get to see the other 95%. (Of course no experiment truly fails...they all give some information.)
-- SIGFPE
I have to read normal journal articles all the time, and let me tell you, sometimes its the most tedious and boring part of being a scientist. Wading through reams of data and trying to figure out how their results are supposed to support their hypotheses is bad enough. Wading through reams of data and trying figure out what conclusions to draw from unexpected or inconclusive data... Holy cow, this will be a boring journal.
The angel in the oatmeal.
Alas, I am in psychology, and while I resent the notion that it is not a "hard" science (we have to wrestle with computationally intensive [read: supercomputers] integration of complex multivariate statistical distributions, genomics, imaging physics, etc.), I will propose that by its very nature psychology is more susceptible to emotional or political bias than other fields, such as theoretical physics.
I cannot tell you how many times I have seen articles rejected from journals because of trite reasons that have nothing to do with anything other than the reviewer's ego. Granted, many of those articles were later accepted in fine journals, but some weren't, and the sheer number of articles rejected for unjustifiable reasons is ridiculous.
The problem is that in a field that can potentially be as fuzzy as psychology, someone can raise an objection that is theoretically problematic, but pragmatically in all likelihood makes no difference. There is often no way of quantifying the magnitude of effect of a possible nuisance variable--or at least, no one does--so in effect, every possible problem can be and are treated as a real problem. This allows someone with a personal agenda to easily prevent papers from being published just by raising a possibility, of which there are an infinite number. You are left with relying on the good will and friendliness of the reviewer.
Eysenck, one of the most prominent psychologists of the last century, for example, founded his own journal out of frustration regarding this fact. Think about the issues dealt with in psychology: free will, nature vs. nurture, perceptual quality, decision making, intellectual abilities, etc. My colleague and I were just discussing yesterday how frequently incredibly rational, intelligent individuals become incoherent and insensible when discussing psychology (how many posts on Slashdot irrationally start quoting science fiction authors when discussing psych?)
There is empirical evidence to suggest the peer-review system is in trouble as well. I recently read of a meta-analysis presented at a conference (the National Academy of Sciences?) suggesting that peer-review did not improve the quality of articles eventually accepted.
About your question of how many major ideas never made it into journals: if they never made it into the literature, we would never know, would we? I'm sure we could all identify cases of famous theories being forgotten in obscure journals or manuscripts, only to be rediscovered later. How many times have we learned that so-and-so was not the original discoverer of X, because it was relegated to obscurity because of the review process?
Kuhn, a prominent philosopher of science, suggested that a prime determinant of the acceptance of a scientific theory is what he called "The Big Mouth Factor". Guess what he was talking about.
One of the most important null resultss in physics was the Michelson Inferometer experiment. It consisted of two mirrors and a beam splitter and displayed interference fringe when the 2 beams crossed again. The idea was that by situating it in different directions one could measure the speed of the earth through the ether by the diffenence in the interference patterns. The results came back negligable and eventually they realized that the reason that the results were null was because there was no ether (I'm not sure if the discovery was a result of the experiment though).
I stole this Sig