Slashdot Mirror
Are Research Papers Less Accurate and Truthful Than in the Past? (economist.com)
An anonymous reader shares an Economist report: An essential of science is that experiments should yield similar results if repeated. In recent years, however, some people have raised concerns that too many irreproducible results are being published. This phenomenon, it is suggested, may be a result of more studies having poor methodology, of more actual misconduct, or of both. Or it may not exist at all, as Daniele Fanelli of the London School of Economics suggests in this week's Proceedings of the National Academy of Sciences. First, although the number of erroneous papers retracted by journals has increased, so has the number of journals carrying retractions. Allowing for this, the number of retractions per journal has not gone up. Second, scientific-misconduct investigations by the Office of Research Integrity (ORI) in America are no more frequent than 20 years ago, nor are they more likely to find wrongdoing.
My research says that overall accuracy has declined at 0.65 radians per fortnight, factoring out verisimilitude mitigation factors where tensile strength is less than 2.227BeV per leapyear. Use of odd numbers and fractional percentages leads to higher levels of acceptance among those who don't know how to spell per centage.
The Russians have won. They have made the world a cesspool of distrust, greed, fear and hate.
Yes.
Lots of old reports, research, etc are known to have been grossly biased, outright doctored, or using questionable results towards questionable confirmations or refutations of the hypothesis.
As a simple example, go read up on the Coca plant, and UN level attempts to eradicate wild plants from its ENTIRE HABITAT RANGE and the ecological damage that has been done as a result.
Hint: While refined cocaine in recreational quantities is dangerous and addictive, individual leaves contain 5 percent of coca extracts per leaf mass, and have non-recreational uses especially in their natural range, as well as modern commercial success as teas and other herbal supplements. Furthermore use as a topical anesthetic requires 1/100th of the dosage used by people to get high. The danger is that unlike marijuana, and more like opiates, cocaine can saturate receptors to the point of causing cardiac arrest or other serious medical issues if misused.
My point being: Any science currently considered controversial will have politically motivated research aimed to either prove or refute the stance that is most politically favorable to the people in power, whether those people are religious, governmental, or commercial in nature. Always has been, likely always will be.
is The Journal of Irreproducible Results back now?
in the good 'ole days. Especially the good 'ole days before computers made it easy to track things.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
This is the first Slashdot post I"ve ever seen, that has a question as title, and where the answer is YES.
Almost every slashdot article in the last couple days has been a question.
Fucking knock it off.
I'm sure this paper is accurate, truthful, well-designed and repeatable.
In 2016, The Journal Nature published a story by Monya Baker, where more than 70% of 1,576 researchers tried and failed to reproduce other scientist's experiments [2].
Even worse, many did claim to have reproduced the Pons and Fleischmann Cold Fusion experiment shortly after their press release in 1989 [3]. So many in fact, Nathan Lewis of Cal Tech quipped "Cold fusion has been verified by no university without a good football team" [4].
The problem has been around for decades. I'm thinking there might be reasons, like patents, contracts, grants, money, and prestige. It could be that science, or at least a bunch of scientists, ain't what they're cracked up to be. Or maybe football appendages and their cozens just aren't that important.
[1]https://en.wikipedia.org/wiki/Replication_crisis
[2]https://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970
[3]https://en.wikipedia.org/wiki/Fleischmann%E2%80%93Pons_experiment
https://bwi.forums.rivals.com/threads/scientists-fleischmann-and-pons-cold-fusion-or-cold-illusion-25-years-later.10260/
Fuck yes. The same lobbyists that put politicians in their pockets back it up with exaggerated or blatantly falsified "scientific" research.
The summary seems to suggest that results should be reproduced before a paper is published?
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
You can, of course cite this research, right? You wouldn't just be creating strawmen because your dishonest, stupid and lazy.
The world's burning. Moped Jesus spotted on I50. Details at 11.
In order to not repeat research that has already been done, there is a pressure to look for unexpected correlations that are both novel and clickbait-worthy.
If the research pertains to politics or consumer products, there might be incentives that tarnish the results as well.
There also isn't much money put towards verifying results unless the results are groundbreaking. Even though reproducibility is the cornerstone of the scientific method, there isn't much effort put into it.
People don't get a PhD for copying somebody else's experiment.
Or more correctly not being able to. Kind of the smoking gun type of evidence that says junk science is embedded as core to modern scienceâ(TM)s purpose.
I may be dead wrong on this, but it seems like many of the studies and papers put out today are funded by grants from organizations which often have a (even if subtle) political or ideological agenda. And if the studies they fund support their position, they hand out more grants. If the studies go against it, that university sees its grants from that organization reduced. Perhaps this has an effect on the results of the studies? I'd like to hope not but it seems like anything we think is right is upside down anymore. I haven't looked into this very closely to see if my anecdotal data point is valid, but I'd like to see if anyone can validate it.
And who is the stupid cunt that modded that down? They just prove how much hate there is for the truth! They find it offensive!
All heavily and horribly biased. It would be like trusting a global warming report from Exxon, and yet we still see them referenced everywhere in the debate.
This article appears to contradict itself. It claims that the rate of inaccurate papers as a fraction of the total is not increasing. However, it also notes that countries with weaker misconduct policies, like China and India, have far higher rates of problem papers. It also notes that while the fraction of papers in these countries with issues is approximately constant the overall share of papers coming from these countries is increasing. Hence, the overall fraction of papers with problems must be increasing too because more and more papers are coming from countries with higher rates of inaccuracies while each country's individual accuracy rate (as a fraction) is constant.
It also seems very narrowly focussed on deliberate attempts to mislead since it concentrates on discipline procedures and investigations. However, the reproducibility problem is generally acknowledged to be mainly due to poor scientific practice, e.g. claiming that correlation implies causation or not understanding statistics, and not due to deliberate malfeasance.
The data also show that there does appear to be a slight increase in the number of corrections per journal - although this is only small and the plot fails to provide error bars so it is impossible to know whether or not this is statistically meaningful. It also cryptically mentions that this is for journals which issue corrections suggesting that there are journals which do not issue them.
The number of invitations I get to be an editor on new journals by predatory publishers has markedly increased over the past few years so, at least based on my experience, that there appear to be many more predatory journals than there used to be and I would be amazed if any cared enough to publish errata given that there is no money in it for them so, if the fraction of junk publications has increased this might entirely hide a large source of irreproducible papers from this study.
Papers are less truthful? This must have been written by climate change deniers!
scientific-misconduct investigations by the Office of Research Integrity (ORI) in America are no more frequent than 20 years ago
Does the Office of Research Integrity have access to more resources than they did 20 years ago? If the number of misconduct investigations is limited by their capacity to investigate then "no more frequent" is meaningless. If ORI can expand their capacity as needed, as more questionable research is reported, then "no more frequent" may be meaningful.
Politics doesn't need stats to push an agenda.
Yes.
It's better to look good than to be right. Thanks facebook!
For example, a 19th century paper will claim that the universe is no bigger than our galaxy, and that the Earth is of whatever age was fashionable at the time. Claims are now converging closer and closer to the truth. Sometimes someone is wrong, but not leeches-cure-all-diseases-wrong.
Ezekiel 23:20
Research Papers in present are probably at same or maybe even at better level of accuracy and veracity compared to past ones, except that what remains in scientific use today has gone through years of debate and filtering and has established its merit. What was bad in past is mostly actively forgotten. So in a way, we see matured, weeded-out picture of the past.
We seem to come to expect faster results and instant quality, but that is probably not going to happen. Patience is needed. Scientific progress has its pace and it won't be rushed. There are mistakes an there will be mistakes. We gain knowledge through them - we will assume something to be true, we will try to turn our alleged knowledge into actions, and reality will be our tutor. It takes time.
http://sciencenordic.com/basic-research-crisis-many-results-cannot-be-replicated
https://www.popsci.com/science/article/2013-05/half-cancer-scientists-have-been-unable-reproduce-studies-survey-finds
https://science.slashdot.org/story/12/04/06/139231/majority-of-landmark-cancer-studies-cannot-be-replicated
Because most 'scientists' are the thick kids that you had to sit through classes with, who couldn't think for themselves, couldn't invent anything, wouldn't listen to somebody who disagreed with them, and who enjoy torturing animals to death. Gee... do you think the sort of people who enjoy torturing animals all day are going to be trustworthy, reliable and intelligent people? Of course they set themselves up as 'experts', but in a field of rampant failure and fraud. Where is the cure for cancer? how many more centuries do we have to wait for it? The 'war on cancer' began 70 years ago. Where is the cure?
Every shitty little study gets tweeted and blogged and newsed around, Elsevier is dead, everybody can download the original paper from sci-hub instead of relying on popsci articles by nonscientists. So less filtering, more critical exposure. Which is a good thing, but it also makes it look like the overall quality has decreased.
Cancer science is know to have a huge problem of repeatability. But one thing to beqr in mind is.... there are sh*t loads of scientists today. Itâ(TM)s not easy, never was (if any, getting harder!). But thereâ(TM)s so much more people around than 50-60 years ago.
So, if thereâ(TM)s more people, and there are more scientists.... the sloppy work is going to be more visible ad well. Put internet, open access, etc on top of that.
This applies to mostly medicine and social science see John Ioannidis's research paper "Why Most Published Research Findings Are False" : http://journals.plos.org/plosm... It seems to me the sciences that deal with statistical p-value significance are all subject to false published research findings , for instance, see Craig Bennet's "Neural correlatates of interspecies perspectitve taking in post-mortem Salmon : An argument for multiple comparisons corrections". http://prefrontal.org/files/po... The paper is a deadpan gag and a veiled attack on sloppy methodology among neuroimaging researchers. Also, researchers run the Baltimore Stockbroker scam : https://somemathematicalmusing... When they selectively choose not to publish certain results in favor of other ones etc... So on and so forth etc...
Headline from the future:
After reviewing the findings that research papers are less accurate and truthful in the past, and suggesting that this may be the result of poor methodology or willful misconduct or both or neither, we respectfully withdraw and retract our findings.
Let's just say that the confidence levels used today would have been laughed at 50 years ago.
Chas - The one, the only.
THANK GOD!!!
If you submit a very good reason a paper should be retracted some will ask for very high fees (much more than an admin charge).
I am constantly aggravated by the amount of articles published without enough information to successfully reproduce the experiment. Even more often I find lies by omission, where a very important aspect of the experiments' protocol is not mentioned.
I just found that an experimental protocol used by a dozen papers, has a glaring problem that absolutely should have been mentioned in all of them, but at least the ones that first introduced it. I am thinking to publish a short article discussing just this glaring problem - I am only worried that it will be reviewed by one of the asshats who published the original articles.
"The agriculture ministry is not in charge of Gundam" - Japanese ministry official.
Authority plays no role in science, so the post was off-topic.
Compilation of crap with estimated probabilities and simulations instead of controled experiments - sure, this 'science' is truthful and accurate.
Probabilistic modeling bullshit instead of rigorous science based on controled, reproducible experiments everywhere. Piles of bullshit upon piles of bullshit, peer-reviewed by sectarian community.
Yes, they are less accurate and less truthful. The cause is a lack of honor and integrity in society.
There is no "-1 offended" or "-1 you don't agree with me" mod options for a reason.
STEM is a saturated field with people jockeying for position. If you publish anything there are people out to take you down. The competition can be fierce. So I would say better overall, but with the occasional poor paper getting accepted.
putting the 'B' in LGBTQ+
"Research Papers" also include many Social "Sciences" (which are not really Science).
These papers hew towards the PC and SJW beliefs and start with predetermined conclusions around which a duct tape and bailing wire construct is built in order to support those predetermined conclusions.
All one needs to look at is, "intersectionalilty" and the various "Studies" disciplines that are churning out papers in the thousands, all of which purport to support that some favored minority is oppressed because a white dude stay in school and got a job.
Unless it's "Climate Science"
Then it's all about authority and consensus.
You reached into a turd and picked out a piece of corn to present to all of us in defense of your piece of shit science.
If you don't know what he's talking about then maybe you should find out.
Just identify something that shit academics like. Like "S.C.U.M. Manifesto".
Find papers that cite it.
Find papers that cite those.
https://www.google.com/search?...
Now are these scientific studies? Not most of them at least, but you will see these opinion pieces presented as if their opinion is a mathematical proof.
It's strange the number of people demanding proof of this phenomenon but it's not hard to find, argue with an SJW on tumblr they'll show you all the papers and studies you want.
Yep. Except for particle physics, which uses 6 sigma. Most fields use 2 sigma.
This isn't a matter of misconduct, that's the wrong way to look at the current failure of science to... do science. (I am a scientist.)
Other metrics are more useful. My favorite is "research efficiency." This is a decidedly commercial metric, it's the amount of revenue or economic activity (in dollars) generated by $1 of scientific research investment. It's been going down since about 1980. Surprisingly, research areas pitched as "basic research" (i.e. math, astronomy) tend to do well with this metric. It's the research that's sold to the public as industrially focused (i.e. my field, nanotechnology) that tends to do the worst.
Another useful metric is the % of science PhDs who stay in science for at least 10 years after getting their degree. This measures how effective we are at training our scientific workforce. That's down significantly over the last 30 years as well. What we teach people now is not what they need to succeed in science after training (which is getting longer and longer).
The metric most scientists are looking for is reproducibility, or the percentage of papers which can be repeated by simply following the instructions in the paper. Papers have grown in length and complexity in the last 40 years. It's pretty hard to argue that reproducibility has actually gone down because older papers simply don't include details we now expect. Of course, this is very hard to measure in any case. That's the thesis of TFA. It doesn't change the very real feeling (and data) that science is somehow not delivering on our investment.
Misconduct is... you're going to have some when there are people involved. You're also going to have mistakes and papers which are disproven very quickly. I have a personal pet peeve for papers that promise extraordinarily cheap hardware by assuming labor is free, manufacturing can be done at large scale without investment in tooling, and working capital is free. Things like this are not actually misconduct, no matter how misleading they are.
Sadly enough, it is easier to forge results on a peer reviewed paper these days, because there are few people who are going to take the time to look at the variables and ensure that 2 + 2 = 4. If I do even a little bit of obfuscation, so my paper says 2 + 2 = 5, I wind up with notoriety, and some cash, especially if I can turn my research into something I can sell for cheap.
If I'm proven wrong, who cares. I still got the cash from the contracts and other people tossing grants my way.
Thats wacist blah blah. Truth is Chinese Nationals only ever care about their own fragile egos. So of course theyre liers.
>muh penis replacements!
Meanwhile in Canada, they have1/10th the US's population and 1/10000th the gun violence.
Youre fucking retarded son.
no.
The signal to noise in the research field varies a bit - but over all it is so bad that most medical and nutritional work is best ignored until replicated.
The norm is there are a few teams doing good work - but most of the papers are grant money prostitution. This muddies the waters for people trying to learn about some topic - reinforcing the accepted narrative in order to keep the grant gravy-train flowing.
A couple of examples:
There is a long list of rodent papers with titles such as "High fat diet causes x-y or z bad things" But when one digs into the paper, the contents of the diet are not specified. Digging further one finds the supplementary information - where only a part number is listed. Pulling up the part number revels that the so-called high fat diet has no fat - instead is loaded with sugar and hydrogenated vegetable oil. (If this was real science the diets would come from one lot (or at least one company and only the single variable would be changed and they would run an independent analysis on the diet contents.) The truth of the matter is the outcome was decided before the research began.
Even in engineering I've talked with students that were told not to write up disturbing results as it would embarrass other researchers. One student was blackmailed with losing his Phd if he didn't leave out the failure of a 'student-T test'( a statistical test that looks at distribution validity).
Academia - even in engineering has become ever more politicized and is about supporting narratives of the biggest ego on the faculty rather than finding truth.
Be skeptical - most of what we know is not from theory but form endless trial and error.
Authority plays no role in science
You're kidding, right? and who said anything about 'science'?!
Every scientific paper should be viewed with skepticism. I don't mean that in a bad way. Skepticism is part of how science works. Every paper you read, assume it may have errors and the conclusions may be wrong. Hopefully the reviewers caught the worst problems, but don't count on it. It takes a lot of other people reading a paper to spot all the problems. Experiments also need to be reproduced. It takes years for the community to reach a consensus about whether a paper's conclusions were right or not.
Science is slow, but that's how it is. Don't think of a paper as a final product. It's just one very small step in a very long process. A lot of the steps along the way eventually turn out to have been missteps. That's ok. Scientists know not to put too much weight on any single paper. Science is done by people, and people make mistakes, but the process still gets us to the goal eventually.
"I'm too busy to research this and form an educated opinion, but I do have time to tell everyone my uninformed opinion."
"Second, scientific-misconduct investigations by the Office of Research Integrity (ORI) in America are no more frequent than 20 years ago, nor are they more likely to find wrongdoing."
If the number of retractions has increased due to an increase in the amount of researched being performed and published scientific misconduct investigations would be expected to have increased proportionally all else being equal. Not having done so doesn't indicate there is less misconduct, it indicates less of the misconduct is being caught and investigated. In my anecdotal experience this sort of thing is likely because the workload for investigators has increased without increasing investigators, budget, and resources accordingly, hence they can only investigate misconduct at the same rate or reduce standards.
First a scientific study found a surge in the number of non-reproducible scientific studies. Then a second study tried to reproduce the results of the first and failed, thereby simultaneously proving and disproving the original study's point.
Just because that's how you would do it doesn't mean that scientists in general are doing it that way. Scientists in the harder sciences like physics and chemistry and yes, climate science are pretty much required to conform with reality or other scientists will take them to task for their misinformation.
Well you got 2 out of 3 right. Climate change "science" is shit. If it wasn't they wouldn't destroy data, hide formulas, fake shit up and try to oppress those who disagree with them.
Oddly, physicists and chemists don't have such a huge number of people calling them liars and they aren't running around calling others "deniers" or using other religious faith-based terms. Why is that? Because chemistry and physics are real sciences while climate "science" is a religion which has the faithful and the deniers.
Sure thing AC. And yet surface temperatures keep increasing, sea level keeps rising, ice keeps melting just like the climate scientists predicted. You'd think in the nearly 200 years of climate science (Fourier discovered the greenhouse effect in 1824) if there was something seriously wrong with it that other scientists would have been able to point it out by now. Climate science has been under intense scrutiny for over 30 years now and no one has been able to shoot it down yet. Instead we have conspiracy theories that tens of thousands of scientists from around the world are lying about it. If they're good enough to hold that big a conspiracy together for so long in the face of the scrutiny you might as well give up. I have yet to see anyone prove they destroyed any data that mattered, hid formulas (the source code of several of the big climate models including the NASA/GISS Model E are available for anyone to see), or faked stuff. In science you have to be able to support your findings and those claiming oppression have failed to provide supporting evidence that they are right.
When I was a graduate student in microbiology in the late 70s, I was working with l-forms of bacteria. The organisms were grown on a media containing polyvinylpyrrolidone (PVP). Unfortunately, some lot numbers of the PVP worked, others didn't. We stocked up on lots that we found to work. If someone were to try to reproduce it, they might not be able to unless they found a good lot of the PVP.