Journal Devoted to the Null Hypothesis

Xcott R13, 3(0,R4) writes "It may sound dull even for academia, but I personally am thrilled that someone is starting a journal devoted entirely to scientific research that fails to produce significant results. Researchers tend to publish successes, so we rarely ever read about experiments or approaches that didn't pan out, leaving future researchers to reinvent the square wheel. The "Journal of Articles in Support of the Null Hypothesis" intends to make some of this valuable boring information available. And such a wonderful title: too bad it's an online journal, else I could put it on the bookshelf next to the Annals of Improbable Research. Causing an explosion that would destroy the Universe."

Comments

Not a new idea

[Spurion]

Richard Feynman was a very strong advocate of having somewhere to publish non-positive results. He was distressed by the credibility added to the existence of psychic powers, influence of star signs, alien abductions and so on, by scientific research. It arises from the fact that if you test your hypothesis to, say, the 5% level, then on average one in twenty researchers is going to get a false positive. People tend to publish the positive results, and the nineteen negative results get forgotten because they're boring, giving a misleading picture. Of course, all the research could be completely accurate, but the skewed statistics make that irrelevant.

So Feynman dearly wanted a Journal of the Null Hypothesis. I think I found that in "Surely You're Joking, Mr. Feynman".

This is Psych, folks, not 'Hard' Science

[sailordave]

Read the Journal's website, folks -- it's a psych journal, not a mol-bio/physics/hard-science journal. Most of the posts here are getting this wrong.

This is in fact the reason why this journal is such a great idea. As a social science, the field of psychology has a much greater problem than fields like physics with dubious positive experiments getting overhyped -- the media will hype the one study that says the Internet turns kids into axe murders, but it doesn't mention the 99 other studies that found no relation.

Feynmann, in fact, wrote an article called 'Cargo-Cult Science', in which he attacked the discipline of psychology for not repeating experiments to check old results. Yes, he would 100% approve of this new journal.

Re:Good idea...in theory...

[4of12]

Turns out that line of research ran into a dead end five years ago. People don't go back and publish "oh, btw, we were wrong".

For a long time, too, I've thought that careful documentation of what does not work and why is just as important as what does work and why.

Publication of "How Great Idea Blah Doesn't Really Work" is vitally important to preventing other creative, thinking-outside-the-box people from chasing down the same dead end in the maze of symptoms of the truth.

However, if you expend a little effort you can find out many of these things in two ways:

1. At the leading edge, talk to people at conferences. Let them know what you're thinking and see what they say. Talk to several people, too. Of course you need to take debunking of your idea on a professional and not personal level. Also you have to be reasonably trusting that not everyone is out to scoop your great ideas.
2. Look carefully at the written articles in your field of interest. Many times in the introductions and conclusions there are little sentences that hint at why some things are not practical; often these sentences help to round out arguments that would other suggest that some harebrained scheme would cure the world's problems.

Re:Journal System considered harmful

[krlynch]

May I ask what horrible experience you have had to so warp your view of reality? I don't disagree that there have been SOME ideas/theories/experiments that have been improperly rejected from publication, but you would be hard pressed, I think, to find an example of truly revolutionary and fundamentally correct work being rejected by the "scientific establishment" (clue for the clueless: there is no such thing as an organized, coherent scientific establishment...). In fact, your whole argument makes little to no sense, since it is REVOLUTIONARY, FUNDAMENTAL work that gets people recognition and promotion, NOT "sticking to the party line". This DOESN'T mean that all theories that disagree with the established theory will be accepted as "plausible" .... most theories by people not trained in the field are notably ridiculous, and worthy of begin ignored, simply because they don't indicate any understanding of the material they claim to be discussing.

My experience is in theoretical particle physics, and in just the last six years, every talk I've been to starts out: "The current theory is wonderful, but deficient in the following ways ... I'm going to tell you about work I've done to throw out this part of the standard theory." Standard Theories become standard, not because we want them to, but because they match the data better than any other theory that has come along, are internally consistent, are predictive, and are falsifiable.

As for your post, there are so many incorrect points that I don't have the time to correct them all, but I'll pick a sample:

1. Why do you think scientists are the very LAST ones to use the Web?: Huh? Scientists INVENTED the network, invented the protocols, and invented teh Web itself, for the dissemination of scientific data. The need for higher speed backbones (Internet2 and its kin) is being DRIVEN by the ever increasing needs of the scientific community for bandwidth. Scientific researchers are on the forefront of the open source and free software movements, and certainly not "the last ones to use the web".
   
2. `peer review', which again is neither: articles are reviewed by members of the scientific field that the journal caters to; that is, multiple practicing scientists read and comment on all aspects of the research and reporting in the submitted articles, and authors are given the chance to dispute or address the conclusions of the reviewer. Seems pretty much like "peers" "reviewing" the work to me.
   
3. When was the last time you knew which papers were rejected and why?: Whenever the authors whose papers are rejected make available the paper and the referees' comments. It is generally considered poor manners to make public that you have rejected someones work...but if they want to do it themselves, that's up to them. Personally, I've talked to a dozen people in the last year alone, and been to three talks where people started out saying "This work was rejected by Journal X; I'll tell you why I think they were wrong."
   
4. All good ideas are rejected for 30 years: this is truly an silly statement, with its all encompassing conclusion. From my own field, I can think of any number of advances that were accepted exceedingly rapidly, or were deemed interesting enough to be worth extended study and discussion: quantum mechanics, special and general relativity, the experimental confirmation of the frame independence of hte speed of light, the necessity of antiparticles, the prediction of the neutrino, the path integral formulation of quantum theory, quantum electrodynamics, the quark model, the discovery of the muon, the discovery of parity violation, quantum chromodynamics, etc. etc. etc. All of these papers demolished the status quo, and yet were accepted very early on by the journals they were submitted to, and the community who read them.
   

So, to summarize, I don't think you have a clue as to what you are talking about, and suspect that you have some personal agenda or vendetta that you feel needs to be made public. I wish you had let us know WHY you feel "rejected" by the scientific community. (remember that "made public" complaint you had?) I certainly don't claim that the process is perfect, but it is the best, most reliable one that we have found so far. Perhaps you could let us know what alternatives you had in mind?