Slashdot Mirror
Are Research Papers Less Accurate and Truthful Than in the Past? (economist.com)
An anonymous reader shares an Economist report: An essential of science is that experiments should yield similar results if repeated. In recent years, however, some people have raised concerns that too many irreproducible results are being published. This phenomenon, it is suggested, may be a result of more studies having poor methodology, of more actual misconduct, or of both. Or it may not exist at all, as Daniele Fanelli of the London School of Economics suggests in this week's Proceedings of the National Academy of Sciences. First, although the number of erroneous papers retracted by journals has increased, so has the number of journals carrying retractions. Allowing for this, the number of retractions per journal has not gone up. Second, scientific-misconduct investigations by the Office of Research Integrity (ORI) in America are no more frequent than 20 years ago, nor are they more likely to find wrongdoing.
In 2016, The Journal Nature published a story by Monya Baker, where more than 70% of 1,576 researchers tried and failed to reproduce other scientist's experiments [2].
Even worse, many did claim to have reproduced the Pons and Fleischmann Cold Fusion experiment shortly after their press release in 1989 [3]. So many in fact, Nathan Lewis of Cal Tech quipped "Cold fusion has been verified by no university without a good football team" [4].
The problem has been around for decades. I'm thinking there might be reasons, like patents, contracts, grants, money, and prestige. It could be that science, or at least a bunch of scientists, ain't what they're cracked up to be. Or maybe football appendages and their cozens just aren't that important.
[1]https://en.wikipedia.org/wiki/Replication_crisis
[2]https://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970
[3]https://en.wikipedia.org/wiki/Fleischmann%E2%80%93Pons_experiment
https://bwi.forums.rivals.com/threads/scientists-fleischmann-and-pons-cold-fusion-or-cold-illusion-25-years-later.10260/
You aren't a scientist, but those of us that are know 80% of our job is writing grant proposals and networking for opportunities to fund our research. If you do not provide results beneficial to a grant provider, there will not be a second grant.
And then there are "scientists", departments full of them, even entire disciplines, that aren't even trying. It's one thing to add another "study" to the pile by "evaluating" a pile of existing studies (probably without the data that supposedly drove the study), it's quite another to just shrug and make up shit. That last bit is what happens quite a lot among the "post-fact" crowd, who therefore are entirely unscientific. But their hold on their tenure tracks is such that only post-fact idiots get in, and so the entire discipline goes to pot.
Typically not so much the "hard" sciences, but there's quite a bit of utter softheadedness to be found among economists, sociologists, humanities, and so on. There's an oft-banned twitter account highlighting papers from such people. And yes, there are "studies" that are, say, a travelogue of the "researcher" going on a sex tourism holiday in Thailand, couched in the turgid "academic" language the discipline uses to keep prying eyes out.
Add enough of that to the pile and the overall quality of research papers is guaranteed to go down.
This applies to mostly medicine and social science see John Ioannidis's research paper "Why Most Published Research Findings Are False" : http://journals.plos.org/plosm... It seems to me the sciences that deal with statistical p-value significance are all subject to false published research findings , for instance, see Craig Bennet's "Neural correlatates of interspecies perspectitve taking in post-mortem Salmon : An argument for multiple comparisons corrections". http://prefrontal.org/files/po... The paper is a deadpan gag and a veiled attack on sloppy methodology among neuroimaging researchers. Also, researchers run the Baltimore Stockbroker scam : https://somemathematicalmusing... When they selectively choose not to publish certain results in favor of other ones etc... So on and so forth etc...
Hah, negmodded for telling an inconvenient truth. Even respected economists admit that economists like to gaze at their models but the "describing reality" part of their job got left by the wayside aeons ago. Sociologists, like professors with a lifetime achievement award for 40 years worth of publications, are quite open that they think "facts don't matter". Humanities departments run on "inclusiveness" ideology telling minorities they're wonderful writers so that when they graduate are just about guaranteed no jobs for writing their doggerel, causing all sorts of follow-on trouble. But you can't say it! Not even on slashdot!
People, we have too many "scientists" (and not enough real ones). We have too many PhD programs, it's been called out a few years ago recently. "Publish or perish" doesn't help either: Gotta write something, no matter what. So we'll just use lots of wooly words to clothe the naked truth that very few people in academia have anything worthwhile to say these days. All that affects the overall output of academia negatively. It's not hard to see why.
Oh my, it just got said again. Welp, I'm sure someone will swoop in with another downmod. Good show!
You fail to name a single example and just talk out of your ass, that's why you were downmodded.
This isn't a matter of misconduct, that's the wrong way to look at the current failure of science to... do science. (I am a scientist.)
Other metrics are more useful. My favorite is "research efficiency." This is a decidedly commercial metric, it's the amount of revenue or economic activity (in dollars) generated by $1 of scientific research investment. It's been going down since about 1980. Surprisingly, research areas pitched as "basic research" (i.e. math, astronomy) tend to do well with this metric. It's the research that's sold to the public as industrially focused (i.e. my field, nanotechnology) that tends to do the worst.
Another useful metric is the % of science PhDs who stay in science for at least 10 years after getting their degree. This measures how effective we are at training our scientific workforce. That's down significantly over the last 30 years as well. What we teach people now is not what they need to succeed in science after training (which is getting longer and longer).
The metric most scientists are looking for is reproducibility, or the percentage of papers which can be repeated by simply following the instructions in the paper. Papers have grown in length and complexity in the last 40 years. It's pretty hard to argue that reproducibility has actually gone down because older papers simply don't include details we now expect. Of course, this is very hard to measure in any case. That's the thesis of TFA. It doesn't change the very real feeling (and data) that science is somehow not delivering on our investment.
Misconduct is... you're going to have some when there are people involved. You're also going to have mistakes and papers which are disproven very quickly. I have a personal pet peeve for papers that promise extraordinarily cheap hardware by assuming labor is free, manufacturing can be done at large scale without investment in tooling, and working capital is free. Things like this are not actually misconduct, no matter how misleading they are.