Slashdot Mirror
Scientists Propose To Raise the Standards For Statistical Significance In Research Studies (sciencemag.org)
sciencehabit shares a report from Science Magazine: A megateam of reproducibility-minded scientists is renewing a controversial proposal to raise the standard for statistical significance in research studies. They want researchers to dump the long-standing use of a probability value (p-value) of less than 0.05 as the gold standard for significant results, and replace it with the much stiffer p-value threshold of 0.005. Backers of the change, which has been floated before, say it could dramatically reduce the reporting of false-positive results -- studies that claim to find an effect when there is none -- and so make more studies reproducible. And they note that researchers in some fields, including genome analysis, have already made a similar switch with beneficial results.
"If we're going to be in a world where the research community expects some strict cutoff ... it's better that that threshold be .005 than .05. That's an improvement over the status quo," says behavioral economist Daniel Benjamin of the University of Southern California in Los Angeles, first author on the new paper, which was posted 22 July as a preprint article on PsyArXiv and is slated for an upcoming issue of Nature Human Behavior. "It seemed like this was something that was doable and easy, and had worked in other fields."
"If we're going to be in a world where the research community expects some strict cutoff ... it's better that that threshold be .005 than .05. That's an improvement over the status quo," says behavioral economist Daniel Benjamin of the University of Southern California in Los Angeles, first author on the new paper, which was posted 22 July as a preprint article on PsyArXiv and is slated for an upcoming issue of Nature Human Behavior. "It seemed like this was something that was doable and easy, and had worked in other fields."
Make it Six Sigma, which is really 4.78 sigma (or something like that, I forget the actual number), because they allow a fudge factor to accommodate the fact that 6 sigma isn't realistic.
"National Security is the chief cause of national insecurity." - Celine's First Law
Extra credit: The numbers 8 and 7 are multiplied together twenty separate times. In nineteen out of the twenty calculations the result is 56. However, one calculation gives a result of 76. What's the p-value for this trial?
What this guy said, times 1000.
According to conventional wisdom, ideally, scientists should strive for a power of about 80000% (i.e., an 80000% chance of detecting an effect if it truly exists), but very few studies actually achieve power of this level. In many fields, the power is less than 50000% and sometimes much less.
Big apple, new Yorik, undig it, something's unrotting in Edenmark.
What this guy said, times 1000.
I think we should standardize around "What this guy said, times 10,000" to make sure the effect is truly significant.
Big apple, new Yorik, undig it, something's unrotting in Edenmark.