Ask Slashdot: Why Does Science Appear To Be Getting Things Increasingly Wrong?

azaris writes: Recent revelations of heavily policy-driven or even falsified science have raised concern in the general public, but especially in the scientific community itself. It's not purely a question of political or commercial interference either (as is often claimed when it comes to e.g. climate research) — scientists themselves are increasingly incentivized to game the system for improved career prospects, more funding, or simply because they perceive everyone else to do it, too. Even discounting outright fraud or manipulation of data, the widespread use of methodologies known to be invalid plagues many fields and is leading to an increasing inability to reproduce recent findings (the so-called crisis of reproducibility) that puts the very basis of our reliance on scientific research results at risk. Of course, one could claim that science is by nature self-correcting, but the problem appears to be getting worse before it gets better.

Is it time for more scientists to speak out openly about raising the level of transparency and honesty in their field?

Comments

The fallacy of labels

[Anonymous Coward]

I think we are just beginning, more and more, to recognize the inherent limitations of terms like 'scientist'. Media outlets have to struggle to be the most clicked-on, first to break every story no matter how poorly researched or even conceived. The average citizen has access to resources that can verify the accuracy of almost anything. Unfortunately this tends to get lost among the increasingly noisy media. It also requires discipline, patience, and focus to actually apply such methods to anything. Most of the time we just take what we hear at face value - this has always been the way of things. Now, however, we feel somehow betrayed by our own conceptions when they turn out to be wrong.

Re:The fallacy of labels

[ClickOnThis]

Math is THE science. Everything is about provability and reproducibility.

No.

Math, for all of its beauty and power, is not a science. Why? Because it does not rely on experimental observations to arrive at conclusions. Instead, it relies on axioms extended by logical reasoning.

seems about the same

[phantomfive]

Once you get past the hype, the media stories, the click bait; and learn how to actually read scientific papers, they seem about as accurate as they've ever been. The second half of this paper discusses the difficulties, and that was decades ago.

Also worth recognizing that science papers are not an attempt to define absolute truth, and people who use it as such (saying, "this paper says X, therefore X is true") are likely to be disappointed. Science papers are essentially correspondence between scientists, saying "hey, look what I did and how it turned out." It's a form of dialectic, and a good one, but not every paper will be equally good, or even true......nor is it intended to be.

Re:seems about the same

[the gnat]

I think there are two other related issues at play here:

1. There has been a proliferation of relatively shoddy low-impact papers. Thanks to the Internet and the large scientific community, many of these are quickly flagged, but it's still a drag. Part of the reason for this is that the developed world (and more recently, aspiring nations) has been over-training scientists for a few decades, and a PhD is typically an essential requirement for most decent careers - which creates a big incentive to publish no matter how crappy the results.

2. Because of our f***ed-up incentive system, there is an additional huge incentive to publish in ultra-selective high-profile journals, which means the result has to be sufficiently exciting (and "citation bait"). Naturally, this leads people to either cheat or (more often) be sloppy and careless. These failures attract the most attention for obvious reasons.

Basically it's a natural side effect of the "democratization" of science. When basic research was just a gentleman's club centered at a relatively few elite institutions, there was much less incentive to game the system.

Re:seems about the same

[phantomfive]

PhD student isn't a towering intellectual giant. (Hell, even I graduated.)

"The closer I got to PhD, the less I respected PhDs."

Runaway capitalism.

[Anonymous Coward]

The increasingly vocal minority promoting no-holds-barred free market capitalism creates a race to the bottom in many fields - it's not only limited to employment, banking, etc. It started years ago with the 'publish or perish' mentality and has progressed now to where various political factions essentially 'buy off' people with college degrees (I won't call them scientists) to get up in front of people and publicly throw support behind their positions, often using sketchy numbers or questionable methods of data analysis. These days everything has to have a profit motive, and science is no different. People have learned that the best way to argue with someone who comes armed with solid facts is to invent your own facts and make them difficult or impossible to prove, hence confusing the hell out of everyone until nobody cares anymore.

But that's the problem...

[Anonymous Coward]

The post attempts to criticize scientists using assumptions not scientifically examined themselves. "Increasing inability," "appears to be," "as is often claimed," "increasingly incentivized," "widespread." Such terms don't even pass muster on Wikipedia, let alone actual scientific journals.

Really? Show me the data. Like a scientist. Is the number of retracted articles increasing in a statistically significant way? Is there a statistically significant change in the types of funding incentives? What is the level at which you call something "widespread?" Prove to me that science itself is actually getting things "wrong" at any rate higher than before. But if you want to attack science, you need to do it on their terms.

Phrasing the question in this way shows a fundamental misunderstanding of how science works. It assumes a narrative and then rapid-links a bunch of anecdotes before asking a direct question about the character of an entire profession.

Citation please on "increasing"

[hey!]

"I've noticed several incidents of this happening" doesn't constitute a trend.

And science isn't immutable truth. It's defensible belief.

Re:Citation please on "increasing"

[hey!]

The notion that "science is getting things increasingly wrong" is so misguided, it's not even wrong. Let me illustrate what I mean when I say "science is defensible belief" with a parable.

Suppose God knows that X is true. At first Alice doesn't believe X but Bob does. Later on she changes her mind to agree with Bob (and God) that X is true. Then they both die and are brought before the throne of God to prove they've been good scientists.

"I am a good scientist," Bob says, "Because I got to God's truth before anyone else."

"I am a good scientist," Alice says, "Because I believed whatever was best supported by the balance of evidence."

Then God says, "Alice has better scientific judgment, but you're both going to hell because you didn't publish."

Incentives

[duckintheface]

I've done biomedical research in the US and Sweden. The incentive structure is totally different. Swedish scientiests take baby steps and reproduce results repeatedly before moving on. American scientists are all trying to win the Nobel prize. They shoot for the big result and nobody gets a grant in the US for repeating results of someone else. Is it a surprise that people respond to the incentives before them?

Re:Incentives

[ShanghaiBill]

Swedish scientiests take baby steps and reproduce results repeatedly before moving on. American scientists are all trying to win the Nobel prize.

On a per capita basis, Sweden has three times as many Nobel Prizes as America. So the American strategy doesn't appear to be very successful. Or maybe the Swedes have a home team advantage.