Positive Bias Could Erode Public Trust In Science

Feelings are more important than science by Gothmolly · 2012-05-11 00:51 · Score: 5, Insightful

Right? isn't that what American schools and TV have been teaching for the last 30 years? Nerds aren't cool - facts are open to interpretation - everyone is special - you can eat more than you grow... When you have a society rewarding irrationality, what do you expect? Rigorous science?

--
I want to delete my account but Slashdot doesn't allow it.

Re:Feelings are more important than science by r1348 · 2012-05-11 01:13 · Score: 3, Insightful

We still are, it's just the definition of "fit" has shifted since when we started creating the environment we live in.
Also, I personally don't miss the "good old times when we were all starving".
Re:Feelings are more important than science by AwesomeMcgee · 2012-05-11 01:19 · Score: 5, Insightful

I wonder how many of these "positive bias" results come from the fact that if you publish results that disagree with the bias of those who are paying for the study, they'll probably ensure it's never published and you'll find yourself no longer running studies on their dollars.

In the tech industry we all deal with non-technical managers who drive the technical direction and often times define the message to the clients. Does science suffer the same unskilled managerial types pushing scientists to interpret results in a particular way perhaps?

I have a hard time believing a professional scientist doesn't know how to apply the scientific method, but then again incompetence is rampant in every other industry I guess, why not the scientific one..
Re:Feelings are more important than science by SirGarlon · 2012-05-11 01:29 · Score: 4, Interesting

This is a dilemma that is really goes to the heart of the philosophy of government. If the majority is irrational, is it better to give them self-determination and accept they will make frequent bad decision, or have the enlightened few rule them and impose better-informed decisions upon them?
Hint: there is no correct answer. I am not an historian but as far as I know this debate between a pure democracy and some form of republic goes back to Rome and Greece.
What is kind of weird is that the two major parties in America have developed into philosophies that are kind of opposite their names: the Democrats favor the paternalistic nanny state governed by the enlightened few (what I would call a "republic"), and the Republicans favor the ignorant mob ("democracy").
As an aside, when America was a young nation many of her leaders advocated public education as a way to narrow the gap between the elite and the general population. That does not seem to be working out real well, though.

--
[Sir Garlon] is the marvellest knight that is now living, for he destroyeth many good knights, for he goeth invisible.
Re:Feelings are more important than science by ArcherB · 2012-05-11 01:31 · Score: 3, Insightful

Right? isn't that what American schools and TV have been teaching for the last 30 years? Nerds aren't cool - facts are open to interpretation - everyone is special - you can eat more than you grow... When you have a society rewarding irrationality, what do you expect? Rigorous science?
Considering that I'm done growing, if I didn't eat more than I grow, I'd die of starvation.

--
There is no "I disagree" mod for a reason. Flamebait, Troll, and Overrated are not substitutes.
Re:Feelings are more important than science by Sique · 2012-05-11 01:37 · Score: 3, Interesting

About every living creates the environment it lives in. Some in very obvious ways (ants, bees, beavers, moles...), some pretty subtle (grazing animals of the open plains tend to hinder the growing of shrubs and trees and thus keep the plain open). Same goes for plants, which change the immediate environment in a way to hinder concurrent species by shadowing the ground, changing water levels and chemical properties of the soil, and fend off enemies.

--
.sig: Sique *sigh*
Re:Feelings are more important than science by Rakishi · 2012-05-11 01:56 · Score: 4, Insightful

Here's the thing, there are no enlightened few. There's just a few equally irrational people whose irrationality makes them think they are rational and all knowing.
Re:Feelings are more important than science by Dragon+Bait · 2012-05-11 02:08 · Score: 2

Right? isn't that what American schools and TV have been teaching for the last 30 years? Nerds aren't cool - facts are open to interpretation - everyone is special - you can eat more than you grow... When you have a society rewarding irrationality, what do you expect? Rigorous science?
Considering that I'm done growing, if I didn't eat more than I grow, I'd die of starvation.
Interesting interpretation of what GP said. Not in context, but interesting. Obviously he meant eat more than you need to grow and sustain yourself, i.e., get fat -- and of course we can argue that getting fat is growing, just in the wrong direction; but again, that would miss his point.
Re:Feelings are more important than science by danbuter · 2012-05-11 02:14 · Score: 4, Insightful

Sadly, in many cases, it's other scientists that are causing the problems. Back when I was in college, my Geology professor was trying to publish a paper that would have invalidated the results of an older study. Unfortunately for him, the major Geology magazines all used a similar pool of professors who were "experts" on that particular topic. One of those reviewers was the geologist whose work was being overturned. Let's just say that my professor's work was shot down quite quickly. (He did get it published, but in a smaller magazine, that honestly has little impact in the field).
Re:Feelings are more important than science by next_ghost · 2012-05-11 02:15 · Score: 5, Informative

Positive bias works a little differently than you think. First, a few definitions. Positive result looks like this: "We tried X on Y and it works." Negative result looks like this: "We tried X on Y and it doesn't work." Which one looks more interesting? That's right, the positive one. Now remember that outside mathematics, science is stuck with probabilities. There's always a very small chance of false positives (and also false negatives, but those are less of a problem).
So let's suppose that we have an experiment which should come out negative with 99% certainty. There's 1% chance of false positive. Now let's have 100 scientists independently perform the experiment. Chances are that 1 scientist will get a false positive. This scientist will then publish a paper while the other 99 will give up and research something else without publishing because of the impression that negative results are not interesting.
This is how positive bias works. Those 99 negative outcomes need to be reported to show that the one false positive is a false positive but they aren't reported. It's not some kind of intentional fraud but merely a consequence of overt obsession with original and sensational results dictated by those who pay for research. It doesn't matter what the research actually means, only that it can be phrased as "We tried X on Y and it works."
Re:Feelings are more important than science by uncqual · 2012-05-11 02:23 · Score: 2, Funny

I also often eat more than I grow... but then my circumference grows, so perhaps it all balances out.

--
Why is there an "insightful" mod and why isn't it "-1"? If I wanted insight, I wouldn't be reading /.
Re:Feelings are more important than science by Baloroth · 2012-05-11 02:28 · Score: 2

Indeed. I was fortunate enough to go to a private (highly regarded) high school. One of my teachers (who was also the principal) liked to say: "The purpose of high school is to train students' bullshit detectors." Unfortunately, most teachers don't think that way, they just think their job is to cram a bunch of information down students throats, which of course they promptly forget the first day on summer vacation.
Obviously, there are sets of basic information everyone should know, but the most important skill, which is how to think and analyze information, is often completely overlooked. And what with all the sensationalist media reporting (which has always been popular, but with the Internet has grown much more so), that skill is necessary if you are to end up actually having your own opinion, and not simply the opinion of a set of people (liberal, conservative, green, communist, w/e) which sounds cool or true to you. Since most people don't have the skill to actually consider the truth behind an opinion, you end up with a society that is extremely divided, but completely unable to really justify their own opinions. This is for both sides of any debate in the US: 99% of the people on both sides really cannot justify their own position.

--
"None can love freedom heartily, but good men; the rest love not freedom, but license." --John Milton
Re:Feelings are more important than science by OeLeWaPpErKe · 2012-05-11 02:35 · Score: 3, Insightful

And humans do ? I must say that while a certain form of organisation is present in cities, only very few cities looked like any global design was done at all in them. When it comes to organisation essentially done by individual humans : animals have plenty of "local" organisation like humans ... take the tunnel structure below molehills for example. This is aside from huge projects like the hoover dam. But most things humans build are much more like the massive organic changes animals cause just as much as humans do.
Besides, humans are not unique in having big infrastructure projects. Ants is the most common example of an animal that organises itself into building huge structures on cue. So do apes. Nothing quite the scale of Hoover dam of course, but the difference is a scale difference, not intent or organisation or even intelligence.
Re:Feelings are more important than science by radtea · 2012-05-11 02:50 · Score: 5, Interesting

During most of my career in pure physics I got negative results, which were always hell to publish. One of the things you'd hear a lot was, "Don't worry, a negative result is just as good as a positive result!"
At some point I started telling friends and colleagues who got positive results, "Don't worry, a positive result is just as good as a negative result!" Which is false, as proven by the fact that no one but me ever said it.
Negative results are hard to get published, but far more common than positive results. Furthermore, on the road to any positive result there are going to be lots of negatives: even today, working in an area where true positives are much more common, I try to put a section in every paper entitled something like "Things That Did Not Work So Well", because any experiment or computation or theory is likely to involve some dead ends that seemed like a good idea at the time, and if scientists don't report on them they will continue to seem like good ideas to people who haven't tried them, who will then waste effort on trying them, and fail to publish them when they don't work...

--
Blasphemy is a human right. Blasphemophobia kills.
Re:Feelings are more important than science by jpate · 2012-05-11 02:56 · Score: 3, Interesting

This is how positive bias works. Those 99 negative outcomes need to be reported to show that the one false positive is a false positive but they aren't reported.
I just want to point out that it's not quite so straightforward. A null result for an effect is not, in and of itself, negative evidence for that effect, it's just a lack of evidence for that effect. It's always possible that a different set of materials, a larger sample size, an additional control, more sophisticated stats, or any number of methodological modifications would succeed in finding an effect. 99 null results with bad materials are not evidence against even a small number (not one!) of positive results with good materials. Null results are under-reported because they are much more ambiguous, not (only) because they are harder to sensationalize.
Re:Feelings are more important than science by crmarvin42 · 2012-05-11 02:58 · Score: 2

It's no mistake that a large portion of students that go to college tend to be moderate to liberal. Many of the teachers and professors, whether knowingly or unknowingly, teach from a liberal standpoint and thus this is what children are taught.

I have to admit that my experience is similar. I remember simply mentioning my views on abortion on a bus during a field trip in college (I was NOT volunteering, and did try to dodge the question for the sake of everyones sanity during the hour long drive), and promptly spent the next 45 min defending my opinion, not only against the students (my peers) but the professor, with his inherent, and unavoidable position as THE authority figure present (and all of the psychological baggage that comes with it). It's subtle and I can't imagine it was intentional, but there you go. I was under immense pressure to change my opinion, with liberal professor applying much of the pressure.

--
Bureaucracy expands to meet the needs of the expanding bureaucracy.-Oscar Wilde
Re:Feelings are more important than science by grep_rocks · 2012-05-11 03:20 · Score: 3, Insightful

Mod me down or whatever but I think you really got it wrong - one party is for an elite or aristocracy based on financial wealth, they spend a lot of money on ads TV networks etc. to get a majority to vote in favor of the aristocracy, lower taxes on the rich less restrictions on the use of capital, less labor laws, less environmental regulation - i.e. freeehdum! the other party is a bit more disorganized and is probably more easily defined as the not-moneyed elite, but they still have to produce leaders and they tend to be more technocratic - so for example one party puts an oil executive as head of the dept of energy, the other put in a nobel prize winning physicist
Re:Feelings are more important than science by matthewv789 · 2012-05-11 03:35 · Score: 5, Interesting

Yup, that's the crux of the problem. While it may be true, as others say below, that publication bias against negative results occurs in all fields (such as physics) regardless of study funding, what we are seeing now is the influence of pharmaceutical industry funding in the clinical trials used for FDA approval of drugs (that is, a company funding the trial of its own drug).
Specifically, drug studies funded by pharmaceutical companies are four times more likely to show a positive benefit than ones funded by neutral sources. This is a problem because nearly two-thirds of clinical trials used for FDA approval are now industry-funded.
Re:Feelings are more important than science by vtcodger · 2012-05-11 03:46 · Score: 2

Actually, it is a bit worse than that. Let us assume that eating Twinkies has no affect whatsoever on toenail cancer (TNC) rates. Let us assume that 20 groups set out to measure the affect of Twinkies on TNC. Since "proof" levels are traditionally set at p=.0.05 levels of significance, it is quite possible that not one, but two groups will get "significant" results. One group proves that Twinkies cause TNC. The other that Twinkies prevent TNC. Both groups will try to publish. They will likely succeed. The remaining groups move on to some other worthy effort.
(For the non-North American readers, Twinkies are an artificial pastry. They have a shelf life probably measured in millenia and are thought to be indestructible. Taste? Taste tests show that most people think that the product tastes better than the cellophane wrapper. Their manufacturer has filed for banruptcy, but the Twinkie franchise is probably considered to be an asset rather than a liability ).

--
You can't see ANYTHING from a car, You've got to get out of the goddamned contraption and walk...Edward Abbey
Re:Feelings are more important than science by Belial6 · 2012-05-11 04:09 · Score: 3, Interesting

The 'facts are open to interpretation seems to run like this: 1) Statements are either facts or opinions. 2) Facts are statements that are correct. 3) Thus all other statements are opinions. 4) Opinions can't be 'wrong' because they are not facts, and are inherently subjective. 5) Thus all Opinions are correct. 6) Statements that are correct are Facts. 7) Opinions are facts. This leads to: 1) Red is blue. 2) Red is blue is incorrect. 3) Since red is not blue, "Red is blue" is an opinion. 4) "Red is blue" can't be wrong because it is an opinion. 5) Thus "Red is blue" is correct. 6) Since "Red is blue" is correct, it is a fact. 7) Red is blue is a correct fact.
Re:Feelings are more important than science by ThatsNotPudding · 2012-05-11 05:16 · Score: 2

Let's just say that my professor's work was shot down quite quickly. (He did get it published, but in a smaller magazine, that honestly has little impact in the field).
Juggs?

Obvious Complex System by Anonymous Coward · 2012-05-11 00:57 · Score: 5, Interesting

Positive Bias is another word for Group Think. I guess it could also mean deception

Re:Obvious Complex System by ballpoint · 2012-05-11 01:52 · Score: 2

Positive Bias is another word for Group Think. I guess it could also mean deception
Replying so parent gets noticed despite a downmod 'Overrated' from score zero.
Please read the link as the behavior it exposes is highly relevant to the article.

--
Flourescent (adj): smelling like ground wheat.

There types of articles are moronic. by atlasdropperofworlds · 2012-05-11 00:58 · Score: 4, Interesting

There are "studies", and then there is observation, modelling, prediction, model testing which is this thing called science. "Studies" are bullshit. Scientific research functions as it should. I believe the OP's article is just a chunck of sensationalist BS, or utterly ignorant of what science is (and is not).

Re:There types of articles are moronic. by jeffmeden · 2012-05-11 01:02 · Score: 3, Insightful

There are "studies", and then there is observation, modelling, prediction, model testing which is this thing called science. "Studies" are bullshit. Scientific research functions as it should. I believe the OP's article is just a chunck of sensationalist BS, or utterly ignorant of what science is (and is not).
You forgot to mention that it is yet another piece of published work that suffers from positive bias...
Re:There types of articles are moronic. by fearofcarpet · 2012-05-11 01:38 · Score: 4, Interesting

There are "studies", and then there is observation, modelling, prediction, model testing which is this thing called science. "Studies" are bullshit. Scientific research functions as it should. I believe the OP's article is just a chunck of sensationalist BS, or utterly ignorant of what science is (and is not).
That is not really what TFA is talking about. Daniel Sarewitz is re-phrasing a long-known problem with "studies," as you call them, which is that complex systems are--by definition--too complex to study as a whole. I am a physical scientist, which means that I typically make or measure something in a well-controlled experiment and then change variables in order to test a hypothesis. I can basically publish a paper that says "we tried really, really hard to find it, but it wasn't there." In the life sciences, they are trying to answer vague cause-effect questions like "does this drug affect a particular type of tumor more than a placebo." Thus researchers in those fields have to create models in which they can control variables. He gives the example of mouse models, which are obviously imperfect models for human physiology. How imperfect is the question. The creeping phenomenon that he is addressing is the tendency to relax the standards for what counts as positive evidence--and I'm grossly oversimplifying--by waving your hands around about how mouse models are imperfect, but that there is definitely "a statistically significant trend." The root cause is simply the ridiculous amount of pressure that life science researchers are under to publish, which requires results, because their methodology is standardized. Those poor bastards can spend eight years on a PhD project that goes nowhere or burn four years of their tenure clock figuring out that their experimental design was flawed. *Poof* no funding, no tenure, no degree, time to consider a new career. That sort of potential downside creates the sort of forced-optimism that TFA describes.

--
Actually, I wrote my thesis on life experience.
Re:There types of articles are moronic. by Will.Woodhull · 2012-05-11 01:50 · Score: 2

Both studies and science are affected by bias inducing forces, such as 'publish or perish' policies of institutions and grant availibilty from stakeholders with an economic interest in the results. Elsevier, et al, raise similar problems with their control of the distribution of knowledge pipelines.
About a hundred years ago traffic problems became so bad that government had to step in and legislate which side of the road drivers had to use, and who had right of way at intersections. It might be that similar legislation is needed to curb the biasing influences on research today.
It is a mistake to think that this is about the science itself. This is about controlling the environment in which the science is done so that we benefit from good science rather than science steered by whoever has the deep pockets to pay for it. The laboratory needs to be isolated from outside economic and political influences, and those influences have found too many ways to get around the barriers that used to work, sort of, in a kind of "it is insulated enough that sometimes good work can be done" way.

--
Will
Re:There types of articles are moronic. by Dusty101 · 2012-05-11 05:00 · Score: 2

Minor correction - the quote actually comes from Rutherford:
"All science is either physics or stamp collecting".
He was subsequently awarded the Nobel prize for Chemistry (1908). Presumably by some perversely vengeful chemists.

Wait, what? by bmo · 2012-05-11 01:01 · Score: 2, Insightful

'Nothing will corrode public trust more than a creeping awareness that scientists are unable to live up to the standards that they have set for themselves,' he adds.

No, the corrosion of public trust is the incessant idiocy coming from Fox and other Murdoch properties exclaiming "oh those silly scientists got it wrong again!" when the story is about a refinement of a model or something.

Scientists are losing the credibility war because scientists are not PR flacks and are unable to counteract the "we don't have to report actual news, we got a court order saying we don't" assholes at Fox.

There is a concerted effort to discredit scientific research no matter what it is.

--
BMO

Re:Wait, what? by Anonymous Coward · 2012-05-11 01:10 · Score: 3, Insightful

Oh yes, blame it all on fox, the ultimate evil in the universe. It has nothing to do with studies being published making lavish claims which are then later proved false, or so wildly overblown that it's almost embarrassing. Of course, the conduct of the scientists themselves couldn't possibly be at fault and it must all be an even republican conspiracy.
Grow up, pull your head out of your ass, and realize that fox and republicans aren't the only source of evil in the world.
Re:Wait, what? by gandhi_2 · 2012-05-11 01:16 · Score: 4, Insightful

This is a story about actual bias in scientists which is affecting the quality of their research.
And you completely gloss over that to take issue with Fox, which is no more ore less biased than msnbc, et al...
Look, people seek an echo chamber. "News" companies of all types just supply the demand.
I suppose YOU don't see a problem with some news organizations taking biased scientific output and unquestioningly running with it as though it were the concrete truth for ever more.

--
THL phish sticks
Re:Wait, what? by AlecC · 2012-05-11 01:26 · Score: 3, Informative

The trouble is that most such "X causes cancer" statements come from the media themselves, recklessly shortening research results saying "consumption of X correlates with a positive increase in cancer", where the nature of the correlation is unknown and the increase is very small. So it is all to often not an accurate prediction of the research. Particularly, there can often be a chinese whispers effect, where the researcher publishes a paper, the University PR department publishes a precis edited for PR purposes, a popular science journal then reports with its own bias, and it is then taken by mainstream media and truncated again.
A particular example, as often reported b y Ben Goldacre, is the Daily Mail, which seems to summarise everything into causing or curing cancer, and will report the same substance on both sides within days.

--
Consciousness is an illusion caused by an excess of self consciousness.
Re:Wait, what? by jo_ham · 2012-05-11 01:26 · Score: 4, Insightful

angry much?
Damn right scientists are angry. The pervasive anti-intellectualism and well-funded attacks on science to suit political ends are getting ridiculous. Positive bias in peer review is a problem that needs to be addressed, but it is all part of science - studies that cannot be replicated are examined in detail and their models either rejected or refined to be tested again. Just because something is published does not make it "the truth" - it means it is a set of conclusions based on experiments that were done by a particular team of scientists. If it's not repeatable then future publications will say so - that's how the back and forth and refining of models happens. Things don't get held back until the issue is "settled" then get published, it simply doesn't work like that.
Where things start to fall down (and where the anti-science folk with an agenda have such an easy time) is that it can be hard for the layman to sort out what to trust in a scientific publication, since the well-hammered, repeated-by-many-groups stuff is sitting alongside single publications claiming XYZ based on PQR from a single data set in a single research group. It's so easy to wade in there and say "look at this! see! scientists can't agree! it's all a big con to get more money! they'll say anything to further their agenda!".
Again, I'm not dismissing the problems of positive bias - it's a factor of peer review and the way human beings approach the reporting of science, but the job is made much harder by a barely-science-literate blogosphere working full steam to discredit anything they can to make it look like there's some sort of global conspiracy of scientists working against the general public in whatever field the propaganda machines are working against. Climate change is clearly the biggest one at the moment, but it's not restricted to that. There's almost anything related to things that challenge fossil fuels and energy research towards energy independence, then there's anything in the biology field relating to stem cells, vaccinations, pharmaceuticals, etc.
The rise of public opinion that science is somehow something to be regarded with great suspicion and that scientists are actively working against the public good is not only troubling, it's highly counterproductive. It's time we started getting angry about it, it's just almost impossible to fight back - scientists are not equipped to do so against a highly organised and well funded group of individuals who do that sort of character assassination and propaganda for a living.
Re:Wait, what? by thegreatemu · 2012-05-11 01:31 · Score: 5, Insightful

While I agree that models are frequently refined, leading to new results, there is a disturbing trend I see, not having to do with positive bias necessarily, but with uncertainty estimation.
One thing that I've found incredibly hard to beat into undergrads taking my physics lab courses is that getting your uncertainties (or error bars) right is far more important than getting the right central value. This is because uncertainties are the only way that two experiments can be compared against each other, or the only way to compare experiment to theory. If I have two models of climate change, one of which predicts a temperature rise of 3 C ± 5% and another that predicts 4 C ± 7%, those results are in large disagreement, whereas two studies that predict 20 C ± 15% and 40 C plusmn 35% are in much closer agreement.
But I see it seems much more frequently, especially in fields like astronomy, too little thought goes into the systematic uncertainties, and you'll get 4 experiments measuring the same thing with results that cannot be reconciled if you take their statistics at face value. This was a huge problem with many of the early global warming predictions as well; every year a new estimation would come out that was completely incompatible with the previous one. Yes, these models are insanely complicated, and it's damn hard to understand all the systematics. And of course you can't put in error bars for plain old mistakes. But do it too many times, and people begin to lose any faith that your estimates can be relied on for anything.
This is the problem I see; not necessarily bias toward a positive result, but a bias toward underestimating the uncertainty of your measurement, which I suppose could be different sides of the same coin. (E.g., a result of 2 ± 0.1 is a positive result; a result of 2 ± 5 is not!).
Re:Wait, what? by AngryDeuce · 2012-05-11 02:09 · Score: 2

"This is absolutely, unalterably correct and anybody entertaining even reasonable skepticism is an IDIOT"
Nobody has a problem with reasonable skepticism. It's unreasonable skepticism, such as that you often hear in the far-right wing media, that people are getting sick of.
The anti-vaccination assholes, and the people that just refuse to believe that mankind is having an effect on the climate, are two good examples of where you see a lot of unreasonable skepticism, but there are plenty others...
I knew the far-right had officially gone full retarded when I had someone tell me climate change was a myth because Al Gore. Seriously, that was the reason why it was a myth: Al Gore. Give me a fucking break.
Re:Wait, what? by joeboomer628 · 2012-05-11 02:20 · Score: 2

One of the reasons that I read Slashdot and other internet information aggregates is because I distrust all traditional news media outlets. At least I get to hear various sides of arguments and if I am informed enough about a subject I can usually discern the truth. In theory journalists are unbiased. Anyone care to evaluate what happens in practice. One of the conditions of employment for New York Times employees that are involved in producing news content, is that they abstain from making political financial contributions. This is to protect the papers image as an unbiased reporter of facts. Two employees who were subject to the rule were listed in contributor public disclosure documents of a political candidate. Their "punishment" was a requirement to ask for their money back. Lack of accountability is the root cause of false reporting due to bias.

--
JoeR
Re:Wait, what? by ArcherB · 2012-05-11 02:22 · Score: 4, Informative

You should thank God that FoxNews exists
Fox is the only "news" organization, ever, that went to court in the US to prove that they don't have to report news.
You are a complete and utter moron if you give whatever they say any credibility at all. This is not ad hominem. This is their own claim that whatever they report does not have to be based on any facts at all. None. Zero. Nada. They can make shit up out of whole cloth if they want. They have a Ruling saying they can.
--
BMO
How many news organizations report false stories? Here's a hint; ALL OF THEM. It happens. We are human. It's USUALLY a mistake.
The difference is when FoxNews makes a mistake, they are sued by someone like yourself, only with extra money laying around, trying to silence the opposition and run FoxNews out of business. Did anyone sue to strip the license of NBC when they released a modified 911 call in the Trayvon Martin shooting? Did CBSNews have to go to court to defend their right to report stories that turn out to be false after that blatantly false Bush National Guard story?
Why is FoxNews different?
Also, the court case you are referring to was not about FoxNews, but a local Fox affiliate. The local affiliate was sued by a former reporter who didn't like the fact that they were made to include Monsanto's side of the story, which they claimed to be false. The part you are referring to where the court said the news organizations were allowed to lie did not come from FoxNews or the court, but a group called Project Censored, which from what I can tell is an anti-government, anti-media, anti-corporation organization. Your kinda people

--
There is no "I disagree" mod for a reason. Flamebait, Troll, and Overrated are not substitutes.
Re:Wait, what? by bmo · 2012-05-11 02:31 · Score: 3, Insightful

I would suggest it all started with Al Gore and Global Warming
You would be wrong and not only that, but without a sense of history at all. I can just point out Senator William Proxmire's (D, btw) "golden fleece award" given to the Aspen Movie Map with regards to "hurr, we don't understand the tech so we must be getting scammed" to Sarah Palin's bitching about silly "fruit fly experiments...hurr, we don't need those" - which comes from a long line of idiots decrying basic science, because their minds are *that* blinkered.
Never mind that the fruit fly experiments are all about genetics because fruit flies have lifespans of ... fruit flies so it makes heritability easier to study. Never mind the fact that the Aspen Movie Map was groundbreaking and you can now look back at it and say "Gee, I wonder where Google got their idea for Street View." But no, you don't hear about that. You hear on Fox that fruit fly studies are just wasting taxpayer money. Because Sarah Palin said so. Because she's such an expert in genetics. *spit*
Then the whole Climategate thing that showed the primary movers of AGW to be complete jerks and thoroughly unlikeable people.
So? It was found to be nothing more than egos. The science itself wasn't discredited, and has only strengthened since then. And I have voiced my opinions here about how badly I thought that it was being handled and my own strong skepticism about AGW in previous years. But you know what, I have recently (as in the past couple of years) found that the anti-AGW crowd to be increasingly full of *real* integrity problems and conflicts of interest.
Back to basic science:
Here's a clue for you and your buddies: You don't get applied science and engineering and fancy new products without basic science.
--
BMO
Re:Wait, what? by jo_ham · 2012-05-11 02:31 · Score: 5, Insightful

And there's jerks like you who claim that anyone who dare look askance at your work are "anti-intellectual" are are too stupid to sort out what to trust in a scientific publication (are you saying that scientific publications, Nature, et. al. have untrustworthy material in it?)
Ah, classic twisting of my words. I'm talking about anti-intellectualism as a movement. Those doing it are extremely smart - that's what makes them good at it. They're good at duping people into following their cause. I am not personally calling individual people stupid unless they actually demonstrate that they are.
And yes, I'm absolutely saying that scientific publications contain untrustworthy information, including the big hitters like Science and Nature - their size and prestige is no assurance of infallibility, and in fact can work against them since people are often reluctant to question them. That's the nature of scientific publishing - until results have been replicated, single-source experiments and models need to be looked at with extreme skepticism.
Last year I performed some work that disproved a piece of published literature (in a non-controversial area of chemistry). I didn't set out to disprove it - I set out to see if I could replicate the results and I determined that the published paper was incorrect. My own conclusions, method and data set were published in response, with some discussion on why the previous paper was drawing incorrect conclusions (mainly an issue with experimental control). My situation is one that is repeated constantly - it's how science works. The stuff that can't be replicated is corrected, the stuff that is replicated becomes more solid.
It's not the layman's fault that they don;t understand some of the intricacies of how peer review and scientific publishing and research works. They're not stupid for not getting that in the same way that I'm not stupid for not understanding the first thing about programming - it's simply not my area of expertise. Where the stupidity *does* arise, however, is when people start to distrust scientists out of hand because they're being told to do so by certain media outlets or special interests. It happened with vaccinations due to a corrupt doctor manipulating a very weak study with the ultimate aim to push a competing vaccine made by a company that paid him off, but it backfired spectacularly - far from getting the competing vaccine popular, people rejected vaccination entirely against their own interests, putting their own and everyone else's kids at more risk. It's this sort of media frenzy and associated public panic and distrust of science (even now, people refuse to believe scientists on the issue, despite the original study being totally debunked and Wakefield himself being struck off the medical register et and the whole thing exposed as a sham).
That's the sort of thing I'm talking about here. We saw it with the MMR vaccine, we see it with nuclear power, we see it with stem cell research, we see it with GM foods (and there *are* some legitimate issues to be raised there, being drowned out by typical media hysteria), we see it with climate science - again, there are legitimate issues to be raised and discussed on a topic that is *gigantic* in scope in the scientific community, but it's being drowned in so much media hysteria and political propaganda that it's almost impossible to get anything done.

Of course, equating people who don't drink your Kool Aid to those who deny the Holocaust really helps your cause to be seen as our nights in shining armor.
Where did I say that? You're dangerously close to Godwining the thread by trying to imply that I brought that up when I did no such thing. The hyperbole serves no one, it only makes your arguments look weak.
I'm not looking to be anyone's "night [sic] in shining armor", nor are most scientists. We just work on the science in our field and go where that leads us. If we wanted to be knights rescuing people I'd have joined the fire service or something. I became a scientist to ultimately help mankind and further our collective knowledge, but I'm no superhero or white knight.
Re:Wait, what? by jo_ham · 2012-05-11 02:38 · Score: 4, Informative

"Scientists" are not forcing anyone to do anything - that's what politicians do. We just do the science. Now, if we end up discovering, for example, that CFCs are damaging the ozone layer, or that PbEt4 in gasoline is poisoning the air then we bring that to the attention of people who can make a decision on what to do about that. We can certainly offer suggestions of what to do about it - stop using CFCs (and develop alternatives) and stop using PbEt4 in gasoline (and work on alternatives) etc, but ultimately sometimes changes have to be made that people aren't going to like.
We don't enjoy "telling people what to do" or set about to discover things that will allow us to dictate to people what to do - ideally we want to find solutions to things that have a minimal impact on people's lives.
We're not in this to control people - we just do science. We don't want to "force" people to do anything, but sometimes what we discover necessitates change in order to benefit everyone as a whole (like removing lead from gasoline, or switching to non-CFC propellants and refrigerants). That's just life I'm afraid.
Re:Wait, what? by FrootLoops · 2012-05-11 02:40 · Score: 2

While the rest of your post strikes me as rather crazy, the last sentence leads to a decent "case study" on the misuse of science by fundamentalists. Research is a theme in the anti-gay movement. Both the Family Research Council (FRC) [a leading anti-gay group] and the National Association for Research and Therapy of Homosexuality (NARTH) [a leading ex-gay group] have it in their names. The anti-gay people who reference studies usually focus on health and other social problems. Topics include higher incidence of HIV and various STDs, various intestinal parasites, anal cancer, domestic violence and murder, promiscuity, mental health problems and suicide, whether it's a choice or genetic, illegal drug use, and pedophilia.
Here's an article about the American College of Pediatricians [the comparatively tiny anti-gay offshoot of the mainstream American Academy of Pediatrics] distorting some research:

The ACP also claimed that the longer you can keep kids from identifying as gay, the less likely they are to kill themselves. ...
In this case, Remafedi says, the ACP missed the larger point: Kids who come out at a younger age are more likely to kill themselves because they are less able to deal with the stigma and isolation of being gay. If anything, the research shows the need for more support.
"It's obvious that they didn't even read my research," Remafedi says. "I mean, they spelled my name wrong every time they cited it."
It's easy to put up a front of respectability. Here's a nice example from the Family Research Council: an hour long lecture against gay marriage. It's offered by the "FRC University Library", which is almost as respectable-sounding as the American College of Pediatricians. The format is a traditional lecture given by a middle-aged guy in a suit. If you actually listen to him, he rambles incessantly and when he makes a point his reasoning is often highly flawed (for instance, listen to his discussion of the interracial marriage analogy around 48 minutes in; the circular reasoning is just sad). But if you just look at the trappings and believe his conclusions, you might get the impression that he was a trustworthy source to base your own views on.
Re:Wait, what? by MightyMartian · 2012-05-11 03:18 · Score: 2

First of all, scientists aren't forcing anyone. They do not have that power. Governments do. Beyond that, history indicates that people do not make the right choices all on their own, and quite often will give in to selfishness and shortsightedness. Look at the North Atlantic cod fishery. For decades scientists warned that it was going to get wiped out, but market forces and the short-term self interest of the fishermen won out, until, that is, the population collapsed due to centuries of overfishing. Leaving the fishermen to their own devices didn't work, it backfired and a major industry of the east coast of North America was all but wiped out.
You have this idea somehow that politics can trump reality. It can't. If we're going to get hit by a tsunami, it doesn't matter one little bloody bit that some organized group stands up and says "It's all liberal lies!"

--
The world's burning. Moped Jesus spotted on I50. Details at 11.
Re:Wait, what? by MightyMartian · 2012-05-11 03:24 · Score: 3, Insightful

Questioning is one thing, but wholesale declarations that the large majority of climatologists are liars or fools is another. What's more it's an old tactic developed by the Creationists in their attacks on evolution; declarations of a cabal of biologists out to hide the truth blah blah blah.
You act as if consensus is a bad thing. Do you think evolution is false because the overwhelming majority of biologists accept it? Do you think General Relativity and Quantum Mechanics is wrong because the overwhelming majority of physicists accept it? I'll wager you don't, but because AGW says "We keep vomiting vast amounts of CO2 into the atmosphere we're going to screw things up royally", which interferes with your short-term self interest, well, it must be false.

--
The world's burning. Moped Jesus spotted on I50. Details at 11.
Re:Wait, what? by jo_ham · 2012-05-11 03:35 · Score: 2

1. http://dictionary.reference.com/browse/scientific+method
2. Reproducibility means exactly that - if you follow the method set out in the paper can you reproduce the results that the paper claims? There's no "system" for testing reproducibility because there doesn't need to be one - you simply do what the authors of the paper did and either it works or it doesn't. If you haven't been given enough information to reproduce the paper's results then the paper is invalid and needs to be corrected.
3. Studies are a particular tool of science, and are one of the most abused, misunderstood and mistrusted tools we have, for all of the reasons you mentioned and more. A huge number of studies are conducted poorly for a large number of reasons. Analysing and critiquing the study itself is large part of peer review.
4. This is just nonsense rambling. I'm sure there are some "career development for PhD's" that is not "science" but it hardly describes the majority of science being doing that is funded by means other than the taxpayer. This just reads like an anti-science talking point with nothing to substantiate it. You know if you're reaching for character attacks on scientists themselves you've run out of arguments.
5. This is definitely true. The media's reporting and (often wilful, most times ignorant) misunderstanding of what scientific studies are reporting is rife and it is one of the main reasons science is so misunderstood by the general public. Quoting scientists out of context, or just plain making stuff up is also widespread. "Scientists say....", "some experts believe...." etc etc.

Also, consider this, can you trust the findings of a scientist without a moral compass?
That's a bit of a non-sequitur. You can replace "scientist" in that sentence with almost any profession. Lawyer, politician, police officer, doctor etc... How does it relate to the issue under discussion, unless you're claiming that *all* scientists have no moral compass?
Re:Wait, what? by Uberbah · 2012-05-11 06:12 · Score: 2

Are you illiterate? The story wasn't lies. They allowed Monsanto to respond to the allegations levied against them. The reporter who wrote the story against Monsanto called Monsanto's response lies and refused to report it. See, that's the difference. FoxNews allows the accused to respond. They report both sides of the story.
Are you? Monsanto objected to the story, and the reporters were fired. Fox successfully argued in court that it had no responsibility to tell the truth. The parent is correct, and you are spouting right wing propaganda.
As usual.

How many news organizations report false stories? Here's a hint; ALL OF THEM.
Here's a hint: we all see your lame false equivalence and lack of proportion. It's like saying that a guy 50 pounds overweight (NBC) and a guy 70 pounds overweight (CNN) are on the same level as a guy 600 pounds overweight (Fox) because they're all obese.
How many Fox reporters were fired for showing summer footage of an old Glenn Beck rally and pretending it was new, so it would look like more people showed up?

data point by Anonymous Coward · 2012-05-11 01:01 · Score: 5, Insightful

I received my PhD in physics, and the thesis was measuring a number, in which I measured zero within the error bar. Not particularly interesting, but valid science. My wife was in a PhD program in Biology, she also did valid science, novel measurement technique, came up with an uninteresting result, therefore was not able to publish, therefore was unable to graduate. It would have been extremely simple to fudge the result to a 2-3 sigma result 'hinting' at an interesting answer, which would have gotten published. I think certain sciences have gotten to a point where they have forgotten that if you do valid work in a novel way, then that is science and you should not be punished for the conclusion of the measurement. Most measurements you do of the natural world should probably end up being unsurprising, and thus uninteresting, but you don't graduate or get tenure with those kinds of results. I think this is the mechanism for the positive bias. That is why I do not take results from certain branches of science at face value.

Re:data point by spiffmastercow · 2012-05-11 01:13 · Score: 4, Insightful

This. Until we start giving PhDs for finding expected results and/or verifying the results of others, we're going to have this problem of research 'exageration'.
Re:data point by Anonymous Coward · 2012-05-11 01:27 · Score: 2, Insightful

Original commenter here:
It shouldn't be in the highest tier journals, but should be able to be published somewhere so people can find the technique and that someone had done the measurement, with result x. And if the idea and methodology is sound, it shouldn't prevent someone from getting a PhD. But it does. And it filters out a lot of people who would choose to publish results that do not further their own career, as well as filtering in people willing to do things like playing with how they cut their data until the results start looking interesting. (it does not even require faking data, just someone willing to stop looking for ways to cut out 'bad' data when the results start looking interesting.)
Re:data point by AwesomeMcgee · 2012-05-11 01:29 · Score: 3, Insightful

If it's valid good science it speaks to the competency of the individual who practiced it, if we aren't graduating these folks, then we're encouraging graduation of incompetent or sensationalist scientists.
Re:data point by Hatta · 2012-05-11 01:44 · Score: 3, Interesting

Which is exactly what we need to do. We need an entire independent arm of the sciences dedicated to confirming established results. This is something that needs to occur separate from the pressure to come up with novel results to please grant reviewers.

--
Give me Classic Slashdot or give me death!
Re:data point by fearofcarpet · 2012-05-11 01:51 · Score: 3, Interesting

I spent some time in the Chemistry department at a fancy ivy leave school that seemed designed to crush the spirits of biologists. While we were off drinking beer and playing volleyball against the Physics department, they were toiling night and day on multi-year projects that may or may not generate publishable results. The sick thing is that the outcome was totally decoupled from the abilities of the researchers--it was more like a test of stamina and luck. I saw talented, brilliant people turn into bitter husks. Some gave up on research and wound up teaching at private colleges, others left science completely--and not into fields that used any of their expertise. It seems like a lottery where those that get lucky (or stab the most backs) go on to academic positions at top-tens and the rest end up in rocking back and forth in the corner mumbling to themselves all day.

--
Actually, I wrote my thesis on life experience.
Re:data point by martas · 2012-05-11 04:29 · Score: 2

Actually there doesn't need to be an independent arm of any kind. All you need to do is get the government to allocate funding for grants dedicated to verification, of course with strict criteria regarding competence of the researcher in experimental protocol and statistics. And publish verification results in university or government run open-access journals.

--
weinersmith

Of course it is a problem by tp1024 · 2012-05-11 01:02 · Score: 4, Insightful

The solution lies in a reformation of research finance that is not focussed on how many papers X published compared to Y, but also takes into account whether they are consequential or not and if they actually comply with at least basic scientific attributes such as repeatibility, verifiablity, falsifiability, accessibility of all data and all conducted research, as well as actually conducted verification of research by independent third parties.

There should also be an outright condemnation of data mining, where data bases are checked only for the existence of attributes and correlations that happen to affirm the researchers opinion and leave all others untouched.

Fields like economics, medicine and climate have long since deteriorated to mere cargo cults due to those failings.

Re:Of course it is a problem by tp1024 · 2012-05-11 01:28 · Score: 2

Even if there were such lists, they are not what research funding is based on.
What funding is based on is which journal published your article and how many citations it received in other papers - whether the author citing your paper has actually so much as read (much less made use of) your paper or not. Just being popular with peers helps, because you can always find some excuse or other to cite a paper even if it is semi-relevant at best.
Re:Of course it is a problem by ceoyoyo · 2012-05-11 01:44 · Score: 3, Interesting

"also takes into account whether they are consequential or not"
That makes the problem worse. We need to evaluate papers based on whether they are good science or not, and not publish the ones that are bad science. Currently the "negative results" which are actually inconclusive results, are not published because they are, well, inconclusive. Unfortunately a lot of the "positive" results are also inconclusive, but they ARE published. The solution is not to publish more "negative" results, it's to stop publishing the flawed "positive" ones.

Replication is king... by Life2Short · 2012-05-11 01:02 · Score: 2

Some sort of redirection towards findings that can be verified by independent labs would seem to be an improvement on the current system. But that would require a focus non science as a system and less of the "great researcher" emphasis we see today.

Positive bias is the wrong term. by Karmashock · 2012-05-11 01:12 · Score: 5, Informative

They're overstating precision. Which rather then a forgivable error is an elementary mistake no trained scientist should ever make.

A VERY basic concept they teach at the lowest level of science education is the distinction between accuracy and precision. This is science 101.

Accuracy is whether or not a given conclusion is correct.
Precision is to the degree of specificity.

Typically you run into problems on complex subjects because they overstate the precision of their data or their ability analyze the data.

This can boil down to simple thinks like significant digits.

For example, I'm measuring volume to two significant digits in a giant data set with thousands of measurements. When and if I average those numbers the final average can't have more then two significant digits. That sounds elementary but you see this error made on some big studies. You'll have a situation where something is being measured in a crude sense by many sources and then in the analysis a much higher degree of specificity is implied.

Often that degree of specificity is required to make certain conclusions which is why they break the rule. This is lazy and a breach of scientific ethics. What they need to do is collect the data all over again this time to the level of specificity they need.

Simply saying its too hard to collect the data properly so they're going to make assumptions is not reasonable or ethical. I suppose you could do it so long as you kept an asterisk next to the data and the findings to make it very clear throughout that the conclusion is a guess and not in any way empirical science since at some point people were guesstimating results.

--
I've decided to stop wasting my time responding to AC trolls/sockpuppets... so if you want a response from me... login.

Re:Positive bias is the wrong term. by medcalf · 2012-05-11 01:38 · Score: 2

That's one of the problems I have with global warming. The precision of the best made, best sites thermometers is about a degree. One common class is +/- 5 degrees. Yet these are being used to derive results in tenths of a degree. Satellite proxy measures help, but that's a very short record in climate-significant time frames. Is the Earth warming? Almost certainly. Is that outside the norm? Hell, we don't even know if it's outside the margin of error.

--
-- Two men say they're Jesus. One of them must be wrong. - Dire Straits
Re:Positive bias is the wrong term. by ceoyoyo · 2012-05-11 01:50 · Score: 5, Informative

"When and if I average those numbers the final average can't have more then two significant digits."
Yes, the average can have more precision than the individual measurements. That's actually kind of the point of an average. It can't improve accuracy though, for the most common definitions of accuracy.
Your definitions of accuracy and precision are sort of right, but also sort of misleading. And the way you use specificity is incorrect. But as you correctly point out, a lot of working scientists are a bit fuzzy on all these concepts too.
Re:Positive bias is the wrong term. by Karmashock · 2012-05-11 01:55 · Score: 2

Well, it's funny when they do that with proxy data. They use samples of biomass in sediment from 10,000 years ago and pretend to derive a .01 temperature difference. You can't get that from pine cones, ice cores, and fossilized vegetation.
The real problem with all this stuff is that the data was never collected for this purpose. The standards and precision were for crop forecasts and whether or not people should go to the beach. It was never collected to be compared to a tenth of a degree across continents and throughout centuries.
Even the satellite data has problems because it's calibrated. The sats don't know what temperature it is on earth without calibration. Basically, you ask the sat what temp it is in a place where you know the temp and from that you infer what all the other temps are in it's readings.
Where things get squirrelly is that you can have two way calibration. That is data is used to calibrate sat data and then that sat data is used to filter the ground stations. So rather then everything being based on a wide data set everything suddenly becomes based on the calibration data.
And what's troubling is that the calibration data for the sats is not fixed. It changes. If you use the old calibration data for new readings most of the sat data reports cooler temperatures. Its only after calibration that it reports higher figures.
Possibly this is all legitimate. But it would be very easy to bias the calibration data slightly to show a warming trend.
Ultimately, the solution is to collect a lot more data, make that data publicly available with no filtration, and then hand that off to another group of scientists to analyze it.
There is too much theory crafting and too little data collection.

--
I've decided to stop wasting my time responding to AC trolls/sockpuppets... so if you want a response from me... login.

Apophenia by concealment · 2012-05-11 01:16 · Score: 3, Insightful

When you have a hammer, everything looks like a nail.

Anything can become a religion, as a result. We're less critical of our data when that happens, and we "nudge" it into place.

The problem is not "science" per se but our social approach to it.

Re:Wow! I guess Science HAS become a religion by AwesomeMcgee · 2012-05-11 01:23 · Score: 5, Insightful

Science is a method dingbat, anyone who puts faith in a scientist however is practicing demagoguery.

Practice science, not demagoguery.

Science is about money now by hshana · 2012-05-11 01:24 · Score: 2

Like just about everything else in this world, science is about money. And how do you get money in science? By finding and/or hyping the next leap forward. Being successful in science is all about getting grants. You don't get tenure without bringing in grant money, you don't get grant money without publishing in the best journals, you don't publish in the best journals without finding the next leap. Your typical PhD finishes school in their late 20's, probably with significant school loan debt. He or she then gets a postdoc where they can barely afford to live in the city with the prestigious school that they think they need to further their career. At the same, it's probably time to think about starting a family (especially if you are a woman). And as a postdoc, the pressure to publish is even greater in a more compressed time frame. There is so much pressure there financially, emotionally and mentally, that is it is no wonder that some people cave and take shortcuts and fudge results. And then, if you do make it to a tenure track position, you don't do much science any more. Instead you spend all your time writing grants and churning through postdocs, who may or may not be fudging their results to get a recommendation to get a better position...

Positive bias in engineering research by l00sr · 2012-05-11 01:24 · Score: 2, Insightful

In engineering research, there is definitely a positive bias; in fact, negative results are rarely published at all. This is both because negative results have less sex appeal than positive results and because peer reviewers are trained to outright reject publications without positive results. Although there is huge pressure to publish positive results, I'm not aware of systemic fraud in the literature. What does happen, however, is roughly this: 1) researcher gets great idea. 2) researcher tries idea. 3) idea fails to produce state-of-the-art results. 4) researcher adds hacks and kludges to marginally improve performance. 5) repeat steps 2-5. So, what you get in the end are journals filled with "positive results" that mean nothing and a bunch of "scientists" who make a living doing things that do not really resemble science at all.

Re:Positive bias in engineering research by ceoyoyo · 2012-05-11 01:54 · Score: 2

I bet what you think are negative results are actually inconclusive results. LOTS of people, including many of the ones writing papers about positive publication bias, make that mistake. An insignificant p-value is NOT a negative result. It's an inconclusive one. In order to actually show a negative result you have to do more work, and delve into the (usually very simple) stats that few people know. Most studies don't have the power to show actual negative results, but in my experience if you actually do the other work you get published. It's almost a given because the reviewers never see that sort of thoroughness.

Re:Wow! I guess Science HAS become a religion by samjam · 2012-05-11 01:29 · Score: 2

I've not seen it put more concisely than that yet - well done.

In relation to your second line: Practice of science is more engineering that science -
In theory there is no difference between theory and practice, but in practice there is.

--
blog.sam.liddicott.com

Re: by Reibisch · 2012-05-11 01:30 · Score: 3, Interesting

And ignores the fact that once published, there's no reliable corrective mechanism to propagate those results down beyond a standard literature search. I'm posting as AC because quite a few years ago I published results that I believed at the time to be correct, but were shown to be wrong in a subsequent paper. Despite this, I'm *still* being cited in new papers while the paper that refuted mine is seldom cited. Science isn't some infallible field. We make mistakes; Sometimes those mistakes are accidental, sometimes they're sloppy, and yes, sometimes they're even intentional. That doesn't reduce the validity of science, but it requires us to be more vigilant.

Obligatory XKCD by codegen · 2012-05-11 01:30 · Score: 2

The case of the green jelly beans.. http://xkcd.com/882/

--
Atlas stands on the earth and carries the celestial sphere on his shoulders.

Re: by Reibisch · 2012-05-11 01:31 · Score: 3, Funny

Crap. I only thought I was posting as AC. Doh! :)

Re:Not in Climate Science! by benjfowler · 2012-05-11 01:34 · Score: 2

LOL.

"First they ignore you, then they ridicule you, then they fight you, then you win." -- Mahatma Gandhi

It happens in Slashdot too... by elsurexiste · 2012-05-11 01:44 · Score: 4, Interesting

I remember a few days ago someone submitted a story about piracy for "The Avengers" being low compared to potential profits from them. A few high-ranked comments were like "This is yet another proof that [insert common /. parlance here]". I saw very few comments that stated the most plausible reason: a camcorded action film, with crappy audio and a shaking image, can't compete against the real thing. I thought the same thing: confirmation bias.

People do it all the time. If something can somehow support their views (specially if they don't RTFA) they'll use it as yet more confirmation. "I still don't get why this piece of evidence is discarded by everyone else! They must be delusional or have bad intentions". For example, I imagine this article will be used as evidence for: lack of funding, falling standards in the US, the demise of education, lack of scientific reasoning (maybe they'll even extend it to scientists themselves), and other common /. utterances. I wonder how many of them will actually say what I found out after RTFA...

So, everyone is playing the same game, and scientists are no exception. But hey, that study has numbers on it. At least you can try to replicate the findings, if only the entry barrier wasn't so high: these tests are *hugely* expensive. More collaboration may be a good idea. Shared laurels are better than none, right?

P.S., a nice article on confirmation bias (and other goodies) here.

--
I rarely respond to comments. Also, don't ask for clarifications: a brain and Google are faster, believe me!

Science comes when results are confirmed by Geoffrey.landis · 2012-05-11 01:44 · Score: 5, Insightful

Actually, science is stll working; the real trouble comes with the publicity of the science.

You should never believe the results of any single study. Every scientist knows this; or should know this. Science comes when results are confirmed, not when somebody publishes the first paper. The real work of science just starts when somebody publishes a study saying "we show that x has the effect y." That initial paper really is no more than "here's a place to start looking." However, newspapers want to publish news, and they need to publish whatever's hot and interesting and being done today, not "well, scientist z had his team take a look at the xy phenomenon to see if there was anything interesting there, and they couldn't really find anything there, although maybe some other research lab might have different results."

And, I suppose that somebody should post a link to the obligatory xkcd: http://xkcd.com/882/

--
http://www.geoffreylandis.com

Re:Science comes when results are confirmed by silentcoder · 2012-05-11 02:02 · Score: 5, Interesting

There's a brilliant line on this in "The science of discworld" - I won't pretend I can quote it 100% accurately off the top of my head but it goes something like this:
"In the media you will often read that a certain scientist is trying to prove a theory. Maybe it's because journalists are trained in journalism and don't know how science works or maybe it's because journalists are trained in journalism and don't care how science works - but a good scientist never tries to prove her theory, a good scientist tries her best to disprove her theory before somebody else does it for her, failing to disprove it is what makes a theory trustworthy."

--
Unicode killed the ASCII-art *
Re:Science comes when results are confirmed by slew · 2012-05-11 03:07 · Score: 3, Interesting

I think the real problem today, is that jounalists think they can be stars out of the gate. In the old days, you started out as a fact-checker, before you could get even get a word that you wrote published (usually anonymously at first). By the time you got a "by-line" you have come up through the trenches and seen all of the behind the scenes mistakes that the "star" writers made. Today you have a blog and no editor. Not only quality journalism goes out the window, but these new so-called journalists don't learn the consequences for some of their habits, because they haven't seen others make them (and haven't been motivated like a fact checker to find the problems).
Of course this "star-at-birth" issue isn't just problem with journalists, but many professions (e.g., scientists, chefs, programmers, etc)...
Re:Science comes when results are confirmed by rgbatduke · 2012-05-11 03:35 · Score: 5, Insightful

Actually, you need to read "The Black Swan" by N. N. Taleb. Science that tries to confirm a theory is already infected with confirmation bias. There are a pile of examples that demonstrate the fallacy of confirmatory inference. Taleb uses a variant of Bertrand Russell's -- a turkey might reasonably infer, based on his daily experience, that humans exist for the sole purpose of feeding him, caring for him, providing for his every need. This might go on for day after day, increasing the turkey's degree of belief in his hypothesis of a good and beneficent humanity filled with love of turkeys, right up to the day that -- ulp -- something unexpected happens.

I'm not a Popperite, rather a Jaynes-Cox-Bayesian, but nevertheless it is important to avoid confounding the relative strength of positive and negative evidence. Absence of evidence is not the same as evidence of absence, yet we almost invariably confound the two.

Taleb damn skippy agrees with you about publicity, however, and the near-criminality of publicity and reporting of science. A newspaper necessarily takes a scientific result or observation and transforms it two ways: First of all, it creates a narrative. It isn't just "a tornado hit Houston", but "a tornado hit Houston, possibly caused by Anthropogenic Global Warming" with the subtext "this isn't an act of nature, random an unpredictable, but is instead our fault". Aztec priests couldn't have come up with a better excuse for ripping the still beating hearts out of a stream of slaves and war captives. Second, it necessarily reduces the complexity of the result to no more than three variables, ideally one. It "Platonifies" it (according to Taleb) -- wraps it up in a pretty, easy to understand package that makes it more predictable, less random than it really was. Global warming is a much simpler "cause" than "A cold front overrunning a warm wet surface layer of air near the ground, creating turbulent rolls that break off and terminate on the ground, sustained and driven by the thermal difference, and it is a better story too.

Sadly, as you point out, real science is all too often (and should be) scientist z looked at something and didn't find much. But what they failed to find and how they looked is actually often as or more important than a study that claims to find something, especially when the latter uses questionable methodology to try to prove something, cherrypicks data (for the same purpose), ignores silent evidence (ditto) etc. Medical science is permeated with this. Nobody gets famous, or rich, or even a job, for looking for a cure for cancer and not finding one. This too is addressed by Taleb. Great book.

rgb

--
Even when the experts all agree, they may well be mistaken. --- Bertrand Russell.
Re:Science comes when results are confirmed by Belial6 · 2012-05-11 03:40 · Score: 4, Informative

Bad "journalism" started WAY before blogs.

Re:Wow! I guess Science HAS become a religion by BobMcD · 2012-05-11 01:50 · Score: 4, Interesting

Saying 'science is a method' is like saying 'Christ was a Jew'. True, but it doesn't change what happened.

Science was an idea designed to seek empirical truth. To find things in such a way that those who followed after could find them again. Then people got a hold of it and started using it as a means to control one another.

Christ (even from the atheist point of view, so bear with me) had a simple message of love being service to your fellow man. Then people got a hold of it and we get monstrosities like the Crusades.

That's where the 'HAS become' part of the above phrase kicks in...

Re:R&D by ceoyoyo · 2012-05-11 01:55 · Score: 2

It's just you. Corporate R&D produces products. Sometimes incremental engineering improvements. Not science.

Corporations USED to do some decent science, even more or less basic science. But not anymore.

The myth of positive bias by ceoyoyo · 2012-05-11 02:13 · Score: 2

Okay, it may not be exactly a myth. We can't tell. I strongly suspect there is actually a negative publication bias.

What most people think are "negative" results are actually inconclusive. A non-significant p-value is NOT a negative result. That misunderstanding is very widespread, and leads to lots of high level mistakes. Half of the neuroscience papers published in top journals including Nature the last two years that could make a mistake based on that fallacy, did. And neuroscience didn't seem to be particularly worse than most other fields.

A non-significant p-value is just that - not significant. Inconclusive. Getting an actual negative result is considerably more work than getting a positive one. You need to figure out what the minimum effect size you're interested in is (you should do that for positive results too, but almost everyone just uses zero for that) and show that your confidence intervals do not include it. As Ionnadis points out, you really should consider the power of your study as well (also for positive results), and take a stab at estimating the priors too.

If you go and do all that, and also do a quality experiment, in my experience you actually have a pretty good chance of getting published, because a) it's clear to the reviewers you've done a really thorough job (any idiot can run some data through a t-test and get a p-value, negative results are harder) and to show a negative result your study is probably much higher powered than a positive result one, meaning an impressively big p-value.

The problem is not that there's a positive publication bias, it's that most scientists don't know how to show negative results so there are very few negative papers around.

Re:R&D by geekoid · 2012-05-11 02:13 · Score: 2

I think that's a classic keyhole problem.

Private companies have a shit ton of marketing and PR.
90% of all projects in large companies fail. 10% succeed. The marketing never mentions the failure.

In the government, 90% of all project are successfully, 10% fail. Because the PR(news media) only focuses on the negative, the success don't get talked about.

This is the same thing.

.

--
The Kruger Dunning explains most post on /. http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect

Re: by jbcksfrt · 2012-05-11 02:17 · Score: 2

If you have a personal web site or something similar, it may be a good idea on your part to make that clear and link to the refuting paper. It would take some guts on your part, of course, because it could negatively affect your career, but it would probably be the right thing to do.

it's a problem, here's how to fix it by Goldsmith · 2012-05-11 02:45 · Score: 2

I'm a scientist. It's a big problem.

Here's how you fix it:
The metric for success for both researchers and their government funding sources is published papers. It's hard to change the cultural view of the scientific community, but it's easy to change government metrics. Paper publishing as a metric is easy to track between programs, but has had a terrible impact on scientific culture. It's also led to a large bias in how the government decides what areas to fund. If your metric is paper publishing and you're looking at energy issues, do you fund a sub-field with a historic high paper publication rate, a moderate paper publication rate or a low publication rate? It's fine for us here to say we'd fund the best research, but a government program manager may lose his job for picking a field with the lower publishing rate.

Other metrics such as how many other researchers use some results or whether a practical implementation of some new technique is developed will be harder to judge and take a longer time to evaluate, but would at least give us an honest assessment of the quality of government funded research. Tie future funding to what our broader society is looking for out of science, and eventually the scientific culture will follow.

troubles doing proper science by bzipitidoo · 2012-05-11 03:13 · Score: 2

There's nothing wrong with science itself. But there are many barriers against doing science correctly. I can't say for sure if the barriers have gotten worse in recent decades. But I think yes, it has become harder to do good science. Why?

The Space Race and Bobby Fischer's day in the sun are long gone. They were good contests, and well-liked, with very positive outcomes, especially when compared to the alternative of nuclear war. In a way, it makes up for science having discovered The Bomb. It was possible that no one would make it to the moon. We were not going to suffer big losses if that had been the outcome. As for Fischer, he looked more and more unstable as the years passed.

But one thing about The Bomb: it demonstrated the power of science, even if very darkly. After that, the Space Race demonstrated the power of science again.

And now? It's really hard to top The Bomb and The Moon. But, by some measures we have. We have the Internet, which has been, among other things, a virtual nuke to the traditional business model of Big Media. Perhaps that's too subtle for the more cretinous among us.

That's what I think is a big factor in the current state of science. We're still in the letdown following the big high of the moon landing, and the awesomeness that is the Internet has not been appreciated enough.

Some of the stretching and outright fraud is a symptom of that letdown. We're straining to meet impossible expectations. It's as if we have to top the Moon to impress the public. One of the more crudely obvious ways to do that is to visit Mars. Send a human to Mars. Except it can't be a mere visit, it would have to be a prelude to colonization. Can we do that? Not presently.

Is there any other reason to suppose that fraud in science is any worse than in the past? Maybe. I think also that science has become more accessible, which on the whole is good. But a bad consequence is that it's easier for quacks, cranks, and cons to fake it.

Finally, there is motivation for the increased fraud: bias against bad news. Right now, one of the loudest message scientists are giving is AGW. The frauds have seized on public dislike of that negative message. They've been riding high ever since Big Tobacco showed them the way with "Doubt is our product", and they've corrupted and captured the social conservatives. The Republican Party has fallen a long way from the sober fiscal pragmatism of the 1950s. In those days, they were the sane, careful counterbalance to the misty and foolish idealism of the Democrats. The book "The Republican War on Science" would have been laughable in the 1960s. Most "squares" were Republicans, while many Democrats were dirty, drug addled, brain rotted, uneducated hippies. No one would have asked if a Republican president was our worst ever, as Rolling Stone did. Now, the Republicans are the crazies in denial.

--
Intellectual Property is a monopolistic, selfish, and defective concept. It is "tyranny over the mind of man"

Absolutely by Will+Steinhelm · 2012-05-11 03:14 · Score: 2

I work in optics and computer vision, and investigate research papers on a regular basis. In the majority of the papers I read the methods proposed either only work for the specific example in the paper (some don't even do that), or only work under such "ideal" circumstances that they barely work under lab conditions, much less real world cases. There are a few gems, but you can tell that much of what is published is just written so someone gets that degree or that quota to reach tenure. Prior to working at this job, my trust in published research was much higher. Now my tendency is to treat it all as garbage until it proves itself otherwise.

Re:Its also about non-scientists expectations by cdrguru · 2012-05-11 03:24 · Score: 3, Insightful

Sorry, but you are asking for something that can be called the "everyone should be at the top" problem. Yes, it would be nice if everyone could understand science, scientific methods and what constitutes the difference between one result and a confirmed, peer-reviewed body of research.

It isn't going to happen that way.

What we need is to stop publishing unconfirmed, preliminary results in popular media. That by itself solves 90% of the problem. So who gets to decide when something is ready for 60 Minutes, Fox and Friends in the Morning or the Today Show? I don't know, but I know it isn't the news media and it should not be the original team reporting some unconfirmed results. Just a rule that says nothing gets copied out of scientific journals until it is marked as having been confirmed by at least one independent team would go a long, long way.

Re:Wow! I guess Science HAS become a religion by Gr8Apes · 2012-05-11 03:31 · Score: 2

Actually, he said "science is a method dingbat," whatever that means.

The importance of commas....

--
The cesspool just got a check and balance.

Either/Or by ThomasLB · 2012-05-11 03:47 · Score: 2

I think the real problem is the "all or nothing" approach the public tends to take.

"I found a study on climate change that is mistaken; therefor, every study on climate change is mistaken!" "I found a minor tenant of Darwinian Theory that has been modified; therefor, every theory of evolution must be a lie!"

Science has always been contentious. It's really hard to admit we're wrong.

A perfect example of what? by Anonymous Coward · 2012-05-11 03:49 · Score: 5, Informative

To me it seems a perfect example of how NON SCIENTISTS have made the expectations that scientists are not able to live up to.

The report was basically "Hey, guys, this looks odd, but we've corrected for everything we could think of. You may want to see if you can replicate this".

Nobody else managed to replicate.

MEANWHILE the original posters were being told how they had it wrong. They changed their process to accord for this and retained some extra difference, despite this. Another attempt by another team showed no effect, so the original scientists CONTINUED to see what could be the cause if not FTL neutrinos.

They then found that the 50ns difference can be accounted for if the connectors were not tight.

NOT, at that time, that this was the cause of the discrepancy, but that

a) the couplings were loose NOW, but they didn't check THEN, could have loosened since then
b) an effect of this is a slight delay, enough to remove the difference they'd seen

All absolutely fine and no "positive bias" AT ALL.

But what is the public saying about it? What were the press saying about it?

THAT debacle (and how it's ignored by the parties involved in it) is a perfect example of the problems science has with people who aren't scientists.

Don't conflate medical research with all science by Benfea · 2012-05-11 04:21 · Score: 2

Let's all try to be careful, here.

Medical research was a little late the science party, and they still have serious issues to work out. Some of these problems can't be helped (low sample sizes lead to higher p values, but higher sample sizes place more human beings at risk), others can (not being skeptical enough of research conducted by organizations with a financial stake in the outcomes).

Most of these problems are peculiar to medical research and should not be conflated with all of science. While we should hold the medical research community's feet to the fire over this, the reason we are hearing so much about this is that they themselves are talking about it and they themselves are conducting the meta-research that is producing all of these articles. Other branches of science have looked down on medical research for a long time, but at least they are now getting serious about addressing the problems.

The problem with "landmark" studies by tgibbs · 2012-05-11 04:39 · Score: 2

Something that most scientists know, but that is not widely appreciated by the public, is that "landmark" studies are particularly subject to positive bias. For a study to be acclaimed as a "landmark," it is the first report of a novel phenomenon, which by definition means that it has not yet been confirmed by other investigators, and moreover that it is in some sense unexpected--which means that there is not even much "indirect" evidence from other sources that it is correct. It is also likely to be submitted to one of the high-profile "newsy" journals, like Science or Nature, where submissions are not evaluated solely on the basis of whether the science is high quality, but are first screened (and usually by an editor, before it even gets specialist peer reviewers) on the basis of whether it is of "broad interest." Because these journals are widely read and cited , they have high impact factors, so even getting published in one of these journals is a feather in a researcher's cap. In contrast, a publication in a middle-rank journal does not give a major immediate boost to a scientist's career, but may have a long-term benefit--but only if it turns out to be correct. Being "scooped" (by having somebody else publish research leading to the same conclusion) will not prevent you from being published in a middle-rank journal, but it will knock you out of the newsy journals. So the alluring prospect of publication in a newsy journal may lead a scientist to be a bit less cautious than normal, and the risk of being "scooped" means that he/she may not take usual precautions (like seeing if somebody else in the lab can reproduce the result independently). High profile journals are stringently reviewed, but peer reviewers even in these highly-regarded journals necessarily take the authors at their word, unless there are blatant errors--nobody visits the authors' labs to look over the shoulders of the researchers or to verify that negative results have not been discarded. On the other hand, if you are planning to publish a result in a middle-rank journal, you are more likely to take the time to make sure that your conclusions will hold up over the long haul.

Working scientists know this, so they take these "landmark" studies with a grain of salt until they are independently confirmed. But the public can easily confuse "high profile" with "most reliable."

That is how proper science is done by Sycraft-fu · 2012-05-11 05:35 · Score: 2

And Feynman said as much in his famous "Cargo Cult Science" speech, which I encourage everyone to go read.

The problem is a lot of scientists DON'T do science that way these days. Bad science, "positive bias" as this article calls it, was rampant in the behavioural sciences when I was studying them. I basically never read a paper where they falsified their theory, or where they said things were inconclusive. They always found a way to hammer the results in to support of their theory, and none of them were ever willing to give me raw data (admittedly I only asked a few, but still).

So while journalists are part of the problem, some scientists are another. They can't ever have their theory be wrong, so they'll do whatever they can to "prove" it. Rather than trying to be the ultimate skeptic and work to find evidence to falsify it, they discard any of that and try to show how it might be true.

This is not a flaw in the system by wen1454 · 2012-05-11 06:13 · Score: 2

This is not a flaw in the system. This is a virtue of the system. Nobody wants to read 100s of papers about how bananas do not cause cancer. No scientist (at least no scientist that I know) believes that a paper proves anything. The paper is a finding worth sharing which must then be confirmed repeatedly by other scientists to be accepted as a fact.

Slashdot Mirror

Positive Bias Could Erode Public Trust In Science

92 of 408 comments (clear)