Journal Devoted to the Null Hypothesis

Xcott R13, 3(0,R4) writes "It may sound dull even for academia, but I personally am thrilled that someone is starting a journal devoted entirely to scientific research that fails to produce significant results. Researchers tend to publish successes, so we rarely ever read about experiments or approaches that didn't pan out, leaving future researchers to reinvent the square wheel. The "Journal of Articles in Support of the Null Hypothesis" intends to make some of this valuable boring information available. And such a wonderful title: too bad it's an online journal, else I could put it on the bookshelf next to the Annals of Improbable Research. Causing an explosion that would destroy the Universe."

Comments

Good idea...in theory...

[kalyptein]

I am a grad student in molecular biology, and I've learned the hard way how nice this kind of journal would have been. I wrote a research proposal about something that, when it showed up in front of my committee, whas dismissed out of hand. Turns out that line of research ran into a dead end five years ago. People don't go back and publish "oh, btw, we were wrong". I, and apparently my advisor who gave me the topic, were too dumb to pick up that this was a discontinued line of research.

On the other hand, it would be a terminally dull read. And people would probably be afraid to publish in it, thinking it would hurt their career to be openly associated with failure.

Re:Good idea...in theory...

[nucal]

I, and apparently my advisor who gave me the topic, were too dumb to pick up that this was a discontinued line of research.

Maybe you should consider a new thesis advisor (seriously). As a student, you would not be expected to know about this sort of thing, but your advisor seems way out of touch. This has a bigger impact on your future than you might think.

Re:Good idea...in theory...

[4of12]

Turns out that line of research ran into a dead end five years ago. People don't go back and publish "oh, btw, we were wrong".

For a long time, too, I've thought that careful documentation of what does not work and why is just as important as what does work and why.

Publication of "How Great Idea Blah Doesn't Really Work" is vitally important to preventing other creative, thinking-outside-the-box people from chasing down the same dead end in the maze of symptoms of the truth.

However, if you expend a little effort you can find out many of these things in two ways:

1. At the leading edge, talk to people at conferences. Let them know what you're thinking and see what they say. Talk to several people, too. Of course you need to take debunking of your idea on a professional and not personal level. Also you have to be reasonably trusting that not everyone is out to scoop your great ideas.
2. Look carefully at the written articles in your field of interest. Many times in the introductions and conclusions there are little sentences that hint at why some things are not practical; often these sentences help to round out arguments that would other suggest that some harebrained scheme would cure the world's problems.

Sometimes....

[dalassa]

Not getting an answer is as important as getting one.

"Well this expirement/proof did nothing, so lets try something else."
And scientific progress moves on. Just like all the failed proofs of Fermat's Last Therom.

Journal System considered harmful

[Anonymous Coward]

The whole `journal system' is flawed. It's goal is to *prevent* the acceptance of `ridiculous' ideas presented by `upstarts', i.e. anything the Big `Profs'---who had not a single original idea in the past two decades---will not like to see published, specially because it would (1) show that the `Profs' themselves are not doing much useful science, and (2) it would force the `Profs' to `upgrade' (or get dumped). NO NO NO! If the Profs don't like your ideas, you are NOT to be published, ever. Period. No publications means no funds, no appointments, you get the picture.
The ScientificCommunity(TM)---which is neither---wants no `upstarts' making ripples in the water. They impede progress. As the saying goes: academic politics are the worst kind, because the stakes are so low.

In the world of ModernScience(TM) you either conform or are crushed. And the best way for the `Profs' to have this power is the closed journal system. Why do you think scientists are the very LAST ones to use the Web, the very LEAST inclined to make the proccess transparent for the public to see? When was the last time you knew which papers were rejected and why? Never?---I thought so? Ever wonder *why*?

That's why they insist in what they call `peer review', which again is neither. The `reviewers' are NOT your peers, but the editor's choice, IF it is the all-powerful editor's whim to even grant that to the author. There is no accountability.

REAL science is pushed to be published in what (yet again) the Profs call ``grey literature'' (meaning what the Profs do not endorse) or forever keep your peace.

Sounds too harsh. Think again, check the history of science. All good ideas are rejected for 30 years, their authors ridiculed and their careers (an sometimes their very lives) wrecked, and *then* some of the Profs *discovers* your idea and mekes it his own, after you have been crushed. Think Wegener and plate tectonics. Think Semmelweiss and aseptical surgical practice. Think Plank and quantum physics. Think Mendel and genetics. (And try to find out about the ones who are being crushed this very moment, and of their much-needed ideas).

Don't like it. Well, do *something* about it.

Re:Journal System considered harmful

[krlynch]

May I ask what horrible experience you have had to so warp your view of reality? I don't disagree that there have been SOME ideas/theories/experiments that have been improperly rejected from publication, but you would be hard pressed, I think, to find an example of truly revolutionary and fundamentally correct work being rejected by the "scientific establishment" (clue for the clueless: there is no such thing as an organized, coherent scientific establishment...). In fact, your whole argument makes little to no sense, since it is REVOLUTIONARY, FUNDAMENTAL work that gets people recognition and promotion, NOT "sticking to the party line". This DOESN'T mean that all theories that disagree with the established theory will be accepted as "plausible" .... most theories by people not trained in the field are notably ridiculous, and worthy of begin ignored, simply because they don't indicate any understanding of the material they claim to be discussing.

My experience is in theoretical particle physics, and in just the last six years, every talk I've been to starts out: "The current theory is wonderful, but deficient in the following ways ... I'm going to tell you about work I've done to throw out this part of the standard theory." Standard Theories become standard, not because we want them to, but because they match the data better than any other theory that has come along, are internally consistent, are predictive, and are falsifiable.

As for your post, there are so many incorrect points that I don't have the time to correct them all, but I'll pick a sample:

1. Why do you think scientists are the very LAST ones to use the Web?: Huh? Scientists INVENTED the network, invented the protocols, and invented teh Web itself, for the dissemination of scientific data. The need for higher speed backbones (Internet2 and its kin) is being DRIVEN by the ever increasing needs of the scientific community for bandwidth. Scientific researchers are on the forefront of the open source and free software movements, and certainly not "the last ones to use the web".
   
2. `peer review', which again is neither: articles are reviewed by members of the scientific field that the journal caters to; that is, multiple practicing scientists read and comment on all aspects of the research and reporting in the submitted articles, and authors are given the chance to dispute or address the conclusions of the reviewer. Seems pretty much like "peers" "reviewing" the work to me.
   
3. When was the last time you knew which papers were rejected and why?: Whenever the authors whose papers are rejected make available the paper and the referees' comments. It is generally considered poor manners to make public that you have rejected someones work...but if they want to do it themselves, that's up to them. Personally, I've talked to a dozen people in the last year alone, and been to three talks where people started out saying "This work was rejected by Journal X; I'll tell you why I think they were wrong."
   
4. All good ideas are rejected for 30 years: this is truly an silly statement, with its all encompassing conclusion. From my own field, I can think of any number of advances that were accepted exceedingly rapidly, or were deemed interesting enough to be worth extended study and discussion: quantum mechanics, special and general relativity, the experimental confirmation of the frame independence of hte speed of light, the necessity of antiparticles, the prediction of the neutrino, the path integral formulation of quantum theory, quantum electrodynamics, the quark model, the discovery of the muon, the discovery of parity violation, quantum chromodynamics, etc. etc. etc. All of these papers demolished the status quo, and yet were accepted very early on by the journals they were submitted to, and the community who read them.
   

So, to summarize, I don't think you have a clue as to what you are talking about, and suspect that you have some personal agenda or vendetta that you feel needs to be made public. I wish you had let us know WHY you feel "rejected" by the scientific community. (remember that "made public" complaint you had?) I certainly don't claim that the process is perfect, but it is the best, most reliable one that we have found so far. Perhaps you could let us know what alternatives you had in mind?

Re:Journal System considered harmful

[Therin]

That's a brilliant post.

Some additional examples of the tyranny of the orthodox:

Jenner with the Smallpox vaccine.

TG Morton and anasthesia

Pasteur (he's not a doctor, we don't have to listen to him)

de Broglie (was about to be denied a PhD when Einstein commented how insightful his work was)

Not a new idea

[Spurion]

Richard Feynman was a very strong advocate of having somewhere to publish non-positive results. He was distressed by the credibility added to the existence of psychic powers, influence of star signs, alien abductions and so on, by scientific research. It arises from the fact that if you test your hypothesis to, say, the 5% level, then on average one in twenty researchers is going to get a false positive. People tend to publish the positive results, and the nineteen negative results get forgotten because they're boring, giving a misleading picture. Of course, all the research could be completely accurate, but the skewed statistics make that irrelevant.

So Feynman dearly wanted a Journal of the Null Hypothesis. I think I found that in "Surely You're Joking, Mr. Feynman".

This is Psych, folks, not 'Hard' Science

[sailordave]

Read the Journal's website, folks -- it's a psych journal, not a mol-bio/physics/hard-science journal. Most of the posts here are getting this wrong.

This is in fact the reason why this journal is such a great idea. As a social science, the field of psychology has a much greater problem than fields like physics with dubious positive experiments getting overhyped -- the media will hype the one study that says the Internet turns kids into axe murders, but it doesn't mention the 99 other studies that found no relation.

Feynmann, in fact, wrote an article called 'Cargo-Cult Science', in which he attacked the discipline of psychology for not repeating experiments to check old results. Yes, he would 100% approve of this new journal.