Slashdot Mirror
Journal Devoted to the Null Hypothesis
Xcott R13, 3(0,R4) writes "It may sound dull even for academia, but I personally am thrilled that someone is starting a journal devoted entirely to scientific research that fails to produce significant results. Researchers tend to publish successes, so we rarely ever read about experiments or approaches that didn't pan out, leaving future researchers to reinvent the square wheel. The "Journal of Articles in Support of the Null Hypothesis" intends to make some of this valuable boring information available. And such a wonderful title: too bad it's an online journal, else I could put it on the bookshelf next to the Annals of Improbable Research. Causing an explosion that would destroy the Universe."
So such a journal needs to have two aspects. One a list of things that won't work and why the person that tried them thought they would to reduce reinventing square wheels and the other is a critical examination/explanation/"proof" of why it wouldn't work. Those experiments that don't have the latter stand as possible areas for great gain to humanity.
Maybe you should consider a new thesis advisor (seriously). As a student, you would not be expected to know about this sort of thing, but your advisor seems way out of touch. This has a bigger impact on your future than you might think.
Read the Journal's website, folks -- it's a psych journal, not a mol-bio/physics/hard-science journal. Most of the posts here are getting this wrong.
This is in fact the reason why this journal is such a great idea. As a social science, the field of psychology has a much greater problem than fields like physics with dubious positive experiments getting overhyped -- the media will hype the one study that says the Internet turns kids into axe murders, but it doesn't mention the 99 other studies that found no relation.
Feynmann, in fact, wrote an article called 'Cargo-Cult Science', in which he attacked the discipline of psychology for not repeating experiments to check old results. Yes, he would 100% approve of this new journal.
Alas, I am in psychology, and while I resent the notion that it is not a "hard" science (we have to wrestle with computationally intensive [read: supercomputers] integration of complex multivariate statistical distributions, genomics, imaging physics, etc.), I will propose that by its very nature psychology is more susceptible to emotional or political bias than other fields, such as theoretical physics.
I cannot tell you how many times I have seen articles rejected from journals because of trite reasons that have nothing to do with anything other than the reviewer's ego. Granted, many of those articles were later accepted in fine journals, but some weren't, and the sheer number of articles rejected for unjustifiable reasons is ridiculous.
The problem is that in a field that can potentially be as fuzzy as psychology, someone can raise an objection that is theoretically problematic, but pragmatically in all likelihood makes no difference. There is often no way of quantifying the magnitude of effect of a possible nuisance variable--or at least, no one does--so in effect, every possible problem can be and are treated as a real problem. This allows someone with a personal agenda to easily prevent papers from being published just by raising a possibility, of which there are an infinite number. You are left with relying on the good will and friendliness of the reviewer.
Eysenck, one of the most prominent psychologists of the last century, for example, founded his own journal out of frustration regarding this fact. Think about the issues dealt with in psychology: free will, nature vs. nurture, perceptual quality, decision making, intellectual abilities, etc. My colleague and I were just discussing yesterday how frequently incredibly rational, intelligent individuals become incoherent and insensible when discussing psychology (how many posts on Slashdot irrationally start quoting science fiction authors when discussing psych?)
There is empirical evidence to suggest the peer-review system is in trouble as well. I recently read of a meta-analysis presented at a conference (the National Academy of Sciences?) suggesting that peer-review did not improve the quality of articles eventually accepted.
About your question of how many major ideas never made it into journals: if they never made it into the literature, we would never know, would we? I'm sure we could all identify cases of famous theories being forgotten in obscure journals or manuscripts, only to be rediscovered later. How many times have we learned that so-and-so was not the original discoverer of X, because it was relegated to obscurity because of the review process?
Kuhn, a prominent philosopher of science, suggested that a prime determinant of the acceptance of a scientific theory is what he called "The Big Mouth Factor". Guess what he was talking about.