Slashdot Mirror
Can Bad Scientific Practice Be Fixed?
HughPickens.com writes: Richard Horton writes that a recent symposium on the reproducibility and
reliability of biomedical research discussed one of the most sensitive issues in science today: the idea that something has gone fundamentally wrong with science (PDF), one of our greatest human creations. The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness. According to Horton, editor-in-chief of The Lancet, a United Kingdom-based medical journal, the apparent endemicity of bad research behavior is alarming. In their
quest for telling a compelling story, scientists too often sculpt data to fit their preferred theory of the world or retrofit hypotheses to fit their data.
Can bad scientific practices be fixed? Part of the problem is that no-one is incentivized to be right. Instead, scientists are incentivized to be productive and innovative. Tony Weidberg says that the particle physics community now invests great effort into intensive checking and rechecking of data prior to publication following several high-profile errors. By filtering results through independent working groups, physicists are encouraged to criticize. Good criticism is rewarded. The goal is a reliable result, and the incentives for scientists are aligned around this goal. "The good news is that science is beginning to take some of its worst failings very seriously," says Horton. "The bad news is that nobody is ready to take the first step to clean up the system."
Can bad scientific practices be fixed? Part of the problem is that no-one is incentivized to be right. Instead, scientists are incentivized to be productive and innovative. Tony Weidberg says that the particle physics community now invests great effort into intensive checking and rechecking of data prior to publication following several high-profile errors. By filtering results through independent working groups, physicists are encouraged to criticize. Good criticism is rewarded. The goal is a reliable result, and the incentives for scientists are aligned around this goal. "The good news is that science is beginning to take some of its worst failings very seriously," says Horton. "The bad news is that nobody is ready to take the first step to clean up the system."
The case against journalism is straightforward: much of the news articles, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, journalists has taken a turn towards darkness. The apparent endemicity of bad journalist behavior is alarming. In their quest for telling a compelling story, journalists too often sculpt facts to fit their preferred narrative of the world or retrofit hypotheses to fit their data.
Unlike journalists, however, science will always have to bow to reality. So, yeah, bad science practice will eventually run aground when reality hits, no matter how many epicycles one add to the model. But bad journalism will persists as long as it attracts eyeballs.
Oliver.
The MMR vaccine fiasco is of course the classic example of this; there are still people acting on the assumption that the lies were true, and that's getting people killed.
...when we replaced the scientific method with scientific consensus?
That 99 out of 100 scientists agree one thing is true doesn't make it true - it may be, it may not be, but the number of people that believe doesn't make it so.
When the scientific community is caught 'correcting' raw data and ostracizing 'non-believers' that challenge their beliefs they undermine the public trust in 'science'.
I was taught that the scientific method welcomed challenges to accepted beliefs - a return to that position would go a long way towards reforming belief in science.
Ken
There will always be shitty studies out there. With the proliferation of these pseudo-journals there will be even more bad science out there. This science is a waste of time of money but I don't think it poses much of a direct threat to progress. The bulk of the wrong studies are likely also the obviously bad and unintersting studies. These are the studies that nobody reads. The quantity of genuinely significant work (stuff that pushes forward a field) is tiny. When something that looks like this comes out it is immediately mobbed: people rush to reproduce the results and/or use the new techniques. If it's wrong we'll know very soon. In practice there is always an attempt to replicate the important stuff, even though the publish or perish nature of science means that pure replication studies are rarely carried out and instead are dressed up as a minor extension of preceeding work. The lesson is that it's dangerous to treat a single study as definitive. Wait for the field to catch up and, where appropriate, wait for the meta-studies.
soylentnews.org
Too many "scientists" are more concerned with the next big grant than with doing quality research. And getting grants is often a lot more about politicking and ass-kissing than making a case for why you actually deserve it.
SJW's don't eliminate discrimination. They just expropriate it for themselves.
Science has always been full of bad science. The people involved have always had agendas. The problem is that we are creating so much data, that it is hard to process and identify what was created in a rigorous process and what is just a pile of crap. And, it is not easy to tell them apart. Then you have people involved. Newton tried his best to discredit Hooke. Hooke was lacking in some areas, but a genius in others. Some scientists just create large quantities of data, and don't know what to do with it. Others have a specific idea, and ignore anything which proves them wrong. Science has just gotten so big, it is hard to find the good amongst the, not really bad but, useless. Scientists must publish or be ignored, so they create anything they can to keep going.
"Publish or Perish", Degrees that require new original ideas, Strict hierarchy structure...
Academic institutions are culturally stuck in victorian times. So if you want to work up, get the choice projects and research, you need to publish. The more your publish, the higher the chances you will move up. Because there is so much published material, people don't read it much, so they found that they can get credit for half ass work.
Your name becomes your brand, so when you try to get a grant your name+institution you will work for will get you the grant money.
There isn't any reason why Say State University of New York Buffalo can't get a grant to study seismology, but chances are it will go to University of California Berkeley not because they will do a better job, but because of the name.
Finally institutions haven't learned how to deal with today's political climate with the attempt for breaking news. Every Hypothesis is sold to the public as a new Theory... Then if that Hypothesis is shown false (as it is common in science) then the media who may have a political slant will go and say see Science is Wrong again, just like our political stance has predicted!
Science for the most part is quite work, collaborating with like minded people, with checks and balances to try to filter out strong egos. But it has gone commercial so these checks and balances are weaken as strong egos will win out.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
On individual days and in individual studies the science can be protected, but you will never completely remove even unintentional bias.
Willful misrepresentation of the facts to satisfy an agenda will continue as long as humans are involved in the experimentation or in the compilation of the results.
Happiness in intelligent people is the rarest thing I know.
Ernest Hemingway
There is no vaccine for the common cold, not even for the "unusually vulnerable." There are over 200 different cold viruses. As a kid, you get lots of different colds, as you get older, you get fewer because you've already been exposed to a large cross-section of them. The next generation is going to have much bigger problems because they won't have been exposed to many of them when they were young - kids with colds are not allowed in day cares so nobody else gets exposed, nobody lets their kids play in the mud any more, everything has to be sanitized (like good old soap and water isn't good enough - you have to have an antibacterial soap).
"Transparent" is a shit show that trades on every stereotype going. A man in drag is NOT a transsexual.
Actually there is not a problem with science there is a problem with biomedical research which the author of the article keeps confusing with all science despite actually referring to fields such as particle physics which does not have this problem. That's not to say that we do not have mistakes but these tend to get caught quickly and retracted e.g. faster than light neutrinos.
Except for medical research, I'd say most of science is the same way as particle physics: the odd mistakes which tend to get caught quickly. I don't hear of frequent retractions or contradictions by chemists, mathematicians, computer scientists, geologists or even non-medical biologists like you do frequently for medical studies. In fact it is incredibly ironic that an article written by a medical researcher criticizing the poor practices in his field is so inaccurately and carelessly written. This aptly illustrates at least part of their problem.
The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue.
That has ALWAYS been true. In fact just about the only way to make a name for yourself in science is to show that someone else is wrong about something. Einstein is famous because he showed how Newton was wrong. We put forward hypothesis, test them and (in what should be a surprise to no one) most of them ultimately turn out to be wrong or defective in some way. As a general rule that is both acceptable (to a point) and expected.
Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.
Again, why the notion that any of this is somehow new?
Can bad scientific practices be fixed? Part of the problem is that no-one is incentivized to be right. Instead, scientists are incentivized to be productive and innovative.
Bullshit they aren't incentivized to be right. Being right is hugely incentivized. The problem is that it is hard to be right about something that is actually complicated and meaningful. So we have to break big problems up into little problems and most of those aren't consequential and many are going to turn out to be wrong or dead ends. Not every bit of science is going to be of world altering importance. Some people are doing some shady things to earn a paycheck and stay in the game but they tend to get found out in due time. Science is remarkably effective in weeding out bad data over time.
Measles vaccine effectiveness is one that is specifically in doubt.
Having looked at this problem, I note that before and after the measles vaccine was introduced, we saw a three order of magnitude drop in US measles cases with similar declines in other countries, correlating with the introduction of measles vaccines in those countries. There's just too much of an effect to hand wave away with the assertion that the world no longer practices measles parties as much as it used to or with the other assertions you make.
Also, lab tests were developed and began being introduced at the same time as the vaccines that only verify 100/25,0000 of suspected cases. A suspect case of measles is not a case of measles. It is not even a diagnosis of measles. It is a case where doctor is covering their ass for a measles-like illness by ordering the test. There is no reason today to expect a "suspected case of measles" in the developed world to have a high likelihood of being a case of measles, especially with the extremely rare incidence of measles. There is no actual evidence here that doctors have a high likelihood of misdiagnosing measles.
You know, this stuff has been explained to you before and yet you continue with your erroneous assertions. When are you going to listen to reason?
One issue with academia is that all research must be the stated hypothesis is confirmed, i.e. a negative hypothesis result is not considered valuable. Even though the elimination of a degree of freedom from consideration for further study is one of the cornerstones of science. Instead, everyone must make something new and groundbreaking.
"Who are you?" "No one of consequence." "I must know." "Get used to disappointment."