Are Research Papers Less Accurate and Truthful Than in the Past? (economist.com)

An anonymous reader shares an Economist report: An essential of science is that experiments should yield similar results if repeated. In recent years, however, some people have raised concerns that too many irreproducible results are being published. This phenomenon, it is suggested, may be a result of more studies having poor methodology, of more actual misconduct, or of both. Or it may not exist at all, as Daniele Fanelli of the London School of Economics suggests in this week's Proceedings of the National Academy of Sciences. First, although the number of erroneous papers retracted by journals has increased, so has the number of journals carrying retractions. Allowing for this, the number of retractions per journal has not gone up. Second, scientific-misconduct investigations by the Office of Research Integrity (ORI) in America are no more frequent than 20 years ago, nor are they more likely to find wrongdoing.

Comments

Everything was always better

[rsilvergun]

in the good 'ole days. Especially the good 'ole days before computers made it easy to track things.

Why are are the headlines now questions?

[Notabadguy]

Almost every slashdot article in the last couple days has been a question.

Fucking knock it off.

Cart before horse?

[TapeCutter]

The summary seems to suggest that results should be reproduced before a paper is published?

Re:My research says..

[IDrinkFatCashews]

The quantity of information available today is so staggering that we cannot know everything about a subject. For example, it's estimated that anyone attempting to research what's known about depression would have to read over 100,000 studies on the subject. And there's the problem of trying to decide which studies have produced reliable results.

Similarly, for information on other topics, not only is there a huge quantity available but with a very uneven level of quality. You don't want to rely on the news in the headlines of sensational tabloids near supermarket checkout counters, and it's just as hard to know how much to accept of what's in all the books, magazines, pamphlets, newspapers, journals, brochures, Web sites, and various media reports that are available. People want to convince you to buy their products, agree with their opinions, rely on their data, vote for their candidate, consider their perspective, or accept them as experts. In short, you have to sift and make decisions all the time, and you want to make responsible choices that you won't regret.

Evaluating sources is an important skill. It's been called an art as well as work—much of which is detective work. You have to decide where to look, what clues to search for, and what to accept. You may be overwhelmed with too much information or too little. The temptation is to accept whatever you find. But don't be tempted. Learning how to evaluate effectively is a skill you need both for writing papers and for your life.

Re:Grants May Have Agendas

[tlhIngan]

I may be dead wrong on this, but it seems like many of the studies and papers put out today are funded by grants from organizations which often have a (even if subtle) political or ideological agenda. And if the studies they fund support their position, they hand out more grants. If the studies go against it, that university sees its grants from that organization reduced. Perhaps this has an effect on the results of the studies? I'd like to hope not but it seems like anything we think is right is upside down anymore. I haven't looked into this very closely to see if my anecdotal data point is valid, but I'd like to see if anyone can validate it.

No, that's always been the case. Lead in gasoline, smoking, acid rain, pesticides, etc., they all had studies to prove that they caused no harm at all. In fact, there are people who basically "manage' this sort of publication, a playbook if you will. As in, the best way to counter something bad in your industry is to manufacture controversy, and the way you do that is by getting studies done in your favor. I cite those cases above because those were "managed" by the same group of people who basically do just that - manufacture controversy. (And yes, that same group is behind climate change opposition as well).

Some history of that can be found in Merchants of Doubt. They came up with the playbook on how to manufacture controversy and thus push regulations out.

And let's not forget other cases like vaccines causing autism and plenty of food related papers all paid for by various aspects of industry.

What's happened is recently the Internet has made it much easier to find information, so hunting down who the sponsors of a paper out is much easier even when they hide through 10 layers of corporate shields, and people are able to seek out the original document much more easily and thus analyze the results. The fact that everyone is moving towards open data as well makes it much easier to spot frauds.