Positive Bias Could Erode Public Trust In Science

ananyo writes "Evidence is mounting that research is riddled with positive bias. Left unchecked, the problem could erode public trust, argues Dan Sarewitz, a science policy expert, in a comment piece in Nature. The piece cites a number of findings, including a 2005 paper by John Ioannidis that was one of the first to bring the problem to light ('Why Most Published Research Findings Are False'). More recently, researchers at Amgen were able to confirm the results of only six of 53 'landmark studies' in preclinical cancer research (interesting comments on publishing methodology). While the problem has been most evident in biomedical research, Sarewitz argues that systematic error is now prevalent in 'any field that seeks to predict the behavior of complex systems — economics, ecology, environmental science, epidemiology and so on.' 'Nothing will corrode public trust more than a creeping awareness that scientists are unable to live up to the standards that they have set for themselves,' he adds. Do Slashdot readers perceive positive bias to be a problem? And if so, what practical steps can be taken to put things right?"

Comments

Feelings are more important than science

[Gothmolly]

Right? isn't that what American schools and TV have been teaching for the last 30 years? Nerds aren't cool - facts are open to interpretation - everyone is special - you can eat more than you grow... When you have a society rewarding irrationality, what do you expect? Rigorous science?

Re:Feelings are more important than science

[r1348]

We still are, it's just the definition of "fit" has shifted since when we started creating the environment we live in.

Also, I personally don't miss the "good old times when we were all starving".

Re:Feelings are more important than science

[AwesomeMcgee]

I wonder how many of these "positive bias" results come from the fact that if you publish results that disagree with the bias of those who are paying for the study, they'll probably ensure it's never published and you'll find yourself no longer running studies on their dollars.

In the tech industry we all deal with non-technical managers who drive the technical direction and often times define the message to the clients. Does science suffer the same unskilled managerial types pushing scientists to interpret results in a particular way perhaps?

I have a hard time believing a professional scientist doesn't know how to apply the scientific method, but then again incompetence is rampant in every other industry I guess, why not the scientific one..

Re:Feelings are more important than science

[Anonymous Coward]

Right? isn't that what American schools and TV have been teaching for the last 30 years? Nerds aren't cool - facts are open to interpretation - everyone is special - you can eat more than you grow... When you have a society rewarding irrationality, what do you expect? Rigorous science?

Yes, yes I do

Re:Feelings are more important than science

[SirGarlon]

This is a dilemma that is really goes to the heart of the philosophy of government. If the majority is irrational, is it better to give them self-determination and accept they will make frequent bad decision, or have the enlightened few rule them and impose better-informed decisions upon them?

Hint: there is no correct answer. I am not an historian but as far as I know this debate between a pure democracy and some form of republic goes back to Rome and Greece.

What is kind of weird is that the two major parties in America have developed into philosophies that are kind of opposite their names: the Democrats favor the paternalistic nanny state governed by the enlightened few (what I would call a "republic"), and the Republicans favor the ignorant mob ("democracy").

As an aside, when America was a young nation many of her leaders advocated public education as a way to narrow the gap between the elite and the general population. That does not seem to be working out real well, though.

Re:Feelings are more important than science

[ArcherB]

Right? isn't that what American schools and TV have been teaching for the last 30 years? Nerds aren't cool - facts are open to interpretation - everyone is special - you can eat more than you grow... When you have a society rewarding irrationality, what do you expect? Rigorous science?

Considering that I'm done growing, if I didn't eat more than I grow, I'd die of starvation.

Re:Feelings are more important than science

[Sique]

About every living creates the environment it lives in. Some in very obvious ways (ants, bees, beavers, moles...), some pretty subtle (grazing animals of the open plains tend to hinder the growing of shrubs and trees and thus keep the plain open). Same goes for plants, which change the immediate environment in a way to hinder concurrent species by shadowing the ground, changing water levels and chemical properties of the soil, and fend off enemies.

Re:Feelings are more important than science

[r1348]

True, but most other species don't adapt their environment through thought, planned interventions but through instinctive, automatical reactions to the environment conditions. The complexity to which such process happens is not even comparable to human's.

Human knowledge is cultural, not genetical, thus way faster to adapt to a shift of conditions. To think about it, other than a fairly large brain and rather precise hands, we're pretty much defenseless compared to other animals. Yet, we thrive in almost all environments of our planet, and we're the only species that actually ventured beyond its limit.

Our evolutionary history is one of unprecedented success, so don't count me in for going back to hunter/gatherers.

Re:Feelings are more important than science

[Rakishi]

Here's the thing, there are no enlightened few. There's just a few equally irrational people whose irrationality makes them think they are rational and all knowing.

Re:Feelings are more important than science

[Dragon Bait]

Here's the thing, there are no enlightened few. There's just a few equally irrational people whose irrationality makes them think they are rational and all knowing.

Bad time to be out of mod-points. I'd mod you up.

Re:Feelings are more important than science

[Dragon Bait]

Right? isn't that what American schools and TV have been teaching for the last 30 years? Nerds aren't cool - facts are open to interpretation - everyone is special - you can eat more than you grow... When you have a society rewarding irrationality, what do you expect? Rigorous science?

Considering that I'm done growing, if I didn't eat more than I grow, I'd die of starvation.

Interesting interpretation of what GP said. Not in context, but interesting. Obviously he meant eat more than you need to grow and sustain yourself, i.e., get fat -- and of course we can argue that getting fat is growing, just in the wrong direction; but again, that would miss his point.

Re:Feelings are more important than science

[Vellmont]

I wonder how many of these "positive bias" results come from the fact that if you publish results that disagree with the bias of those who are paying for the study, they'll probably ensure it's never published and you'll find yourself no longer running studies on their dollars.

What? Where are you getting this idea? Scientists publish research that disagrees with the people who fund it all the time. No academic researcher would agree to be funded with stipulations on publishing.

Perhaps you're talking about internal studies done by such agents? Shell obviously isn't required to publish research on global warming.

Re:Feelings are more important than science

[danbuter]

Sadly, in many cases, it's other scientists that are causing the problems. Back when I was in college, my Geology professor was trying to publish a paper that would have invalidated the results of an older study. Unfortunately for him, the major Geology magazines all used a similar pool of professors who were "experts" on that particular topic. One of those reviewers was the geologist whose work was being overturned. Let's just say that my professor's work was shot down quite quickly. (He did get it published, but in a smaller magazine, that honestly has little impact in the field).

Re:Feelings are more important than science

[next_ghost]

Positive bias works a little differently than you think. First, a few definitions. Positive result looks like this: "We tried X on Y and it works." Negative result looks like this: "We tried X on Y and it doesn't work." Which one looks more interesting? That's right, the positive one. Now remember that outside mathematics, science is stuck with probabilities. There's always a very small chance of false positives (and also false negatives, but those are less of a problem).

So let's suppose that we have an experiment which should come out negative with 99% certainty. There's 1% chance of false positive. Now let's have 100 scientists independently perform the experiment. Chances are that 1 scientist will get a false positive. This scientist will then publish a paper while the other 99 will give up and research something else without publishing because of the impression that negative results are not interesting.

This is how positive bias works. Those 99 negative outcomes need to be reported to show that the one false positive is a false positive but they aren't reported. It's not some kind of intentional fraud but merely a consequence of overt obsession with original and sensational results dictated by those who pay for research. It doesn't matter what the research actually means, only that it can be phrased as "We tried X on Y and it works."

Re:Feelings are more important than science

[sohmc]

This is going to sound troll-y but I hope that /.-ers will look past the thesis and read the arguments...

Public education has done a lot, but to a point. I grew up going to private school when I was younger and then public school for middle school or high. If memory serves me correctly (which it may not), is that my views when younger tended to be more conservative than in my later years. Of course, in college, it was a bit of a wild target but eventually settled right of the middle.

It's no mistake that a large portion of students that go to college tend to be moderate to liberal. Many of the teachers and professors, whether knowingly or unknowingly, teach from a liberal standpoint and thus this is what children are taught. There are many history lessons I learned when I was in high school that turned out to be either flat wrong or horribly one-sided.

The lesson here is that students don't get the full story. But teachers teach as if they do. Teachers have to do this because they need to get through a lesson plan and can't spend weeks on one topic. It's understandable. Unfortunately, when all students hear is "Lincoln freed the slaves", they think that's all there is. Very few of them go deeper into the material because they have assignments, projects, papers, math problems, etc. And what kid wants to spend his time reading for fun about the Civil War when Pokemon is on!

Children are taught it's more important to be correct than it is to know the full story. They learn through social pressure it's cooler to know who the Jersey Shore has on next season than the top writs in front of the SCOTUS. You're considered a jock if you can name ERAs of various baseball players and not know the atomic number of helium.

While I'm sure there are kids that know all of these things, they are in the minority. And until this changes, we'll always have a gap between elite and the General Population.

Re:Feelings are more important than science

[uncqual]

I also often eat more than I grow... but then my circumference grows, so perhaps it all balances out.

Re:Feelings are more important than science

[Baloroth]

Indeed. I was fortunate enough to go to a private (highly regarded) high school. One of my teachers (who was also the principal) liked to say: "The purpose of high school is to train students' bullshit detectors." Unfortunately, most teachers don't think that way, they just think their job is to cram a bunch of information down students throats, which of course they promptly forget the first day on summer vacation.

Obviously, there are sets of basic information everyone should know, but the most important skill, which is how to think and analyze information, is often completely overlooked. And what with all the sensationalist media reporting (which has always been popular, but with the Internet has grown much more so), that skill is necessary if you are to end up actually having your own opinion, and not simply the opinion of a set of people (liberal, conservative, green, communist, w/e) which sounds cool or true to you. Since most people don't have the skill to actually consider the truth behind an opinion, you end up with a society that is extremely divided, but completely unable to really justify their own opinions. This is for both sides of any debate in the US: 99% of the people on both sides really cannot justify their own position.

Re:Feelings are more important than science

[OeLeWaPpErKe]

And humans do ? I must say that while a certain form of organisation is present in cities, only very few cities looked like any global design was done at all in them. When it comes to organisation essentially done by individual humans : animals have plenty of "local" organisation like humans ... take the tunnel structure below molehills for example. This is aside from huge projects like the hoover dam. But most things humans build are much more like the massive organic changes animals cause just as much as humans do.

Besides, humans are not unique in having big infrastructure projects. Ants is the most common example of an animal that organises itself into building huge structures on cue. So do apes. Nothing quite the scale of Hoover dam of course, but the difference is a scale difference, not intent or organisation or even intelligence.

Re:Feelings are more important than science

[fortfive]

I think there are enlightened "elite" folks, although I don't know how few or many. Identifying them, however, and getting them in power and preventing their corruption once there are fraught with perhaps insurmountable difficulties.

Re:Feelings are more important than science

[radtea]

During most of my career in pure physics I got negative results, which were always hell to publish. One of the things you'd hear a lot was, "Don't worry, a negative result is just as good as a positive result!"

At some point I started telling friends and colleagues who got positive results, "Don't worry, a positive result is just as good as a negative result!" Which is false, as proven by the fact that no one but me ever said it.

Negative results are hard to get published, but far more common than positive results. Furthermore, on the road to any positive result there are going to be lots of negatives: even today, working in an area where true positives are much more common, I try to put a section in every paper entitled something like "Things That Did Not Work So Well", because any experiment or computation or theory is likely to involve some dead ends that seemed like a good idea at the time, and if scientists don't report on them they will continue to seem like good ideas to people who haven't tried them, who will then waste effort on trying them, and fail to publish them when they don't work...

Re:Feelings are more important than science

[dzfoo]

Does nerd-bias count as "Positive" or "Negative" bias?

(I think you kinda prove the parent's point.)

        -dZ.

Re:Feelings are more important than science

[jpate]

This is how positive bias works. Those 99 negative outcomes need to be reported to show that the one false positive is a false positive but they aren't reported.

I just want to point out that it's not quite so straightforward. A null result for an effect is not, in and of itself, negative evidence for that effect, it's just a lack of evidence for that effect. It's always possible that a different set of materials, a larger sample size, an additional control, more sophisticated stats, or any number of methodological modifications would succeed in finding an effect. 99 null results with bad materials are not evidence against even a small number (not one!) of positive results with good materials. Null results are under-reported because they are much more ambiguous, not (only) because they are harder to sensationalize.

Re:Feelings are more important than science

[SirGarlon]

For what it's worth, I totally agree. The question becomes, "whose irrationality is best." I chose not to raise that point in my post because it could distract from the philosophical dilemma, which I think remains intact even if you concede that no one is truly wise.

Re:Feelings are more important than science

[crmarvin42]

It's no mistake that a large portion of students that go to college tend to be moderate to liberal. Many of the teachers and professors, whether knowingly or unknowingly, teach from a liberal standpoint and thus this is what children are taught.

I have to admit that my experience is similar. I remember simply mentioning my views on abortion on a bus during a field trip in college (I was NOT volunteering, and did try to dodge the question for the sake of everyones sanity during the hour long drive), and promptly spent the next 45 min defending my opinion, not only against the students (my peers) but the professor, with his inherent, and unavoidable position as THE authority figure present (and all of the psychological baggage that comes with it). It's subtle and I can't imagine it was intentional, but there you go. I was under immense pressure to change my opinion, with liberal professor applying much of the pressure.

Re:Feelings are more important than science

[pak9rabid]

As an aside, when America was a young nation many of her leaders advocated public education as a way to narrow the gap between the elite and the general population. That does not seem to be working out real well, though.

You can lead a horse to water...

Re:Feelings are more important than science

[Joce640k]

The problem isn't the science, the problem is the newspapers reporting any old science paper as if it were the absolute scientific truth as believed by the majority of scientists.

Combine this with the fact that the ignorant journalists enjoy making scientists look silly in exactly the same way that jocks like picking on nerds at high school and the results are quite predictable.

If I was king I'd make them all live a few weeks without access to any of the products developed by those scientists. Unfortunately I'm not.

Re:Feelings are more important than science

[Anonymous Coward]

While true, that's an oversemplification. I would rather be governed by somebody that is trying to tackle, say, unemployment based on our best current scientific understanding of economics, rather than somebody that screams 'they took 'r jerbs!'. The more rational person will still draw the wrong conclusions and make irrational mistakes. But which one would you want in power?

Re:Feelings are more important than science

[Anonymous Coward]

A good number of my teachers in high school felt that way as well...unfortunately, I think that many students failed to grasp that. Granted, there were plenty who did, but it sorta makes me feel sad that so many just didn't seem to get it - it could have been a useful skill for them to have. Unfortunately, as well, the teachers did not hold to the same philosophy in college for the most part. It was definitely all about cramming knowledge into heads. There's not much you can do about that though. You just move on and surround yourself with people that can think for themselves. Otherwise, you just have a bunch of "yes-men" as friends. Who wants that?

Re:Feelings are more important than science

[grep_rocks]

Mod me down or whatever but I think you really got it wrong - one party is for an elite or aristocracy based on financial wealth, they spend a lot of money on ads TV networks etc. to get a majority to vote in favor of the aristocracy, lower taxes on the rich less restrictions on the use of capital, less labor laws, less environmental regulation - i.e. freeehdum! the other party is a bit more disorganized and is probably more easily defined as the not-moneyed elite, but they still have to produce leaders and they tend to be more technocratic - so for example one party puts an oil executive as head of the dept of energy, the other put in a nobel prize winning physicist

Re:Feelings are more important than science

[mdarksbane]

I would be much happier if we just didn't put anyone in power, rather than trying to pick the smarter monkey.

Re:Feelings are more important than science

[matthewv789]

Yup, that's the crux of the problem. While it may be true, as others say below, that publication bias against negative results occurs in all fields (such as physics) regardless of study funding, what we are seeing now is the influence of pharmaceutical industry funding in the clinical trials used for FDA approval of drugs (that is, a company funding the trial of its own drug).

Specifically, drug studies funded by pharmaceutical companies are four times more likely to show a positive benefit than ones funded by neutral sources. This is a problem because nearly two-thirds of clinical trials used for FDA approval are now industry-funded.

Re:Feelings are more important than science

[LordLimecat]

facts are open to interpretation

Seems to me youre begging the question: If there is positive bias, these "facts" are indeed open to interpretation.

Re:Feelings are more important than science

[tomthegeek]

That's not what my mom told me!

Re:Feelings are more important than science

[vtcodger]

Actually, it is a bit worse than that. Let us assume that eating Twinkies has no affect whatsoever on toenail cancer (TNC) rates. Let us assume that 20 groups set out to measure the affect of Twinkies on TNC. Since "proof" levels are traditionally set at p=.0.05 levels of significance, it is quite possible that not one, but two groups will get "significant" results. One group proves that Twinkies cause TNC. The other that Twinkies prevent TNC. Both groups will try to publish. They will likely succeed. The remaining groups move on to some other worthy effort.

(For the non-North American readers, Twinkies are an artificial pastry. They have a shelf life probably measured in millenia and are thought to be indestructible. Taste? Taste tests show that most people think that the product tastes better than the cellophane wrapper. Their manufacturer has filed for banruptcy, but the Twinkie franchise is probably considered to be an asset rather than a liability ).

Re:Feelings are more important than science

[sustik]

And that is why publishing only that they tried X and worked on Y is not publishable (or should not be and it is not as I am aware) in a top journal. You need an explanation. If the explanation is lacking but the result is truly surprising or revolutionary (cold fusion, cure of up-to-then uncurable condition, faster than light travel, etc.) then you repeat it maybe 5-10 more times? Until that is completed, you may publish a tech report putting your stamp on this work.

Re:Feelings are more important than science

[Anonymous Coward]

"Everyone is special - you can eat more than you grow..."

Surely this is a cynical reference to entitlement? We deserve to eat even though we are not producing our food as individuals. We deserve to eat even though we are producing too many mouths as a populace. All that is missing is a modest proposal...

Re:Feelings are more important than science

[phantomfive]

Don't worry, a positive result is just as good as a negative result!"

lol that is hilarious.

I try to put a section in every paper entitled something like "Things That Did Not Work So Well"

And that is an excellent idea.

Re:Feelings are more important than science

[Belial6]

The 'facts are open to interpretation seems to run like this: 1) Statements are either facts or opinions. 2) Facts are statements that are correct. 3) Thus all other statements are opinions. 4) Opinions can't be 'wrong' because they are not facts, and are inherently subjective. 5) Thus all Opinions are correct. 6) Statements that are correct are Facts. 7) Opinions are facts. This leads to: 1) Red is blue. 2) Red is blue is incorrect. 3) Since red is not blue, "Red is blue" is an opinion. 4) "Red is blue" can't be wrong because it is an opinion. 5) Thus "Red is blue" is correct. 6) Since "Red is blue" is correct, it is a fact. 7) Red is blue is a correct fact.

Re:Feelings are more important than science

[TheGoodNamesWereGone]

Right? isn't that what American schools and TV have been teaching for the last 30 years? Nerds aren't cool - facts are open to interpretation - everyone is special - you can eat more than you grow... When you have a society rewarding irrationality, what do you expect? Rigorous science?

Shit, and me with no mod points. Well said.

Re:Feelings are more important than science

[Belial6]

More accurately, Twinkies are a pastry that, due to their popularity over such a long time, have become a household name in the US. Thus, hipsters like make up urban myths about them. Things like their longevity ( actually 25 days ) . Their 'artificial' status. And, the idea that they are not extremely delicious. I don't usually reference Snopes, but their writeup on Twinkies is pretty good. http://www.snopes.com/food/ingredient/twinkies.asp

Re:Feelings are more important than science

[ThatsNotPudding]

Let's just say that my professor's work was shot down quite quickly. (He did get it published, but in a smaller magazine, that honestly has little impact in the field).

Juggs?

Re:Feelings are more important than science

[GrandTeddyBearOfDoom]

It's reasonably well known that, in the case of clinical trials of psychiatric medications, that there are often positive correlations between the success of a drug in a trial and the corporation funding the trial. And the funding body in these cases has a veto on the trial. (See Moncrieff's books for more and the actual references... I've lent my copies to friends so don't have them to hand.) Most likely this goes on all the time in medical science due to the amount of money at stake. In the case of other sciences, one would expect evolutionary pressures to promote what the funding bodies want to see, regardless of explicit intent. The funding bodies' pressure to publish, and the need for academics to conform to a certain extent in order to preserve their career path is what has caused this bias to build up over the years.

Re:Feelings are more important than science

[Anonymous Coward]

Right? isn't that what American schools and TV have been teaching for the last 30 years? Nerds aren't cool - facts are open to interpretation - everyone is special - you can eat more than you grow... When you have a society rewarding irrationality, what do you expect? Rigorous science?

Consider the science, politics and emotions behind the "global warming" and "green" movements. Need I say more?

Re:Feelings are more important than science

[orgelspieler]

Nothing quite the scale of Hoover dam of course...

Are you sure about that? Worldwide mega colony

Re:Feelings are more important than science

[DarkIye]

He is implying that that is what we are heading for, yes.

Re:Feelings are more important than science

[Anonymous Coward]

Well it is difficult to justify forcing a human being to carry a fertilized egg to term.

Re:Feelings are more important than science

[Anonymous Coward]

>advocated public education as a way to narrow the gap

actually it worked pretty well until we stared dumbing down public education and cutting funding for it. the level of ignorance in this country truly frightens me.

Re:Feelings are more important than science

[Cro Magnon]

Considering that I'm done growing, if I didn't eat more than I grow, I'd die of starvation.

I'm still growing. Just not vertically.

Re:Feelings are more important than science

[r1348]

"and had probably been spread and maintained by human travel"

Re:Feelings are more important than science

[Anonymous Coward]

Abortion as a topic sucks, because it's a completely entrenched position on both sides, and one neither side can budge from.

The problem is that it's one of those few cases where the result of a metaphysical question "when during gestation does a human cell or collection of human cells become a human?" has very real physical and legal consequences. Furthermore, the stakes are high on both sides.

Once it's a human, killing it by choice is murder. Certain actions may be able to reduce this to something slightly lower, but it still constitutes ending the life of another human.

Until it's a human, preventing abortion amounts to denying medical care, or perhaps even slavery or rape (for certain definitions.)

It's impossible to intelligently talk about the physical and legal question without really considering the metaphysical question, but nobody wants to do that. Even when they do, it's not usually a reasoned argument. One side says "at conception" and the other side says "at birth." I don't think either side is really willing to carry their views out to their full logical implications.

Based primarily on people's actions (NOT their words), I think most people's answer to the metaphysical question is "somewhere in-between." Most women don't grieve a heavy period 1 week late, but most will grieve a fetal demise discovered at the 20-week ultrasound. The tipping point is probably a lot closer to conception than it is to that ultrasound, which is something most "Pro-Choice" activists would deny.

All of this comes to: The only thing to do about abortion is to keep your mouth shut. If you disagree with the crowd, keep your mouth shut. If you agree with the crowd, keep your mouth shut anyway, lest the crowd change their mind. For heaven's sake, don't admit to having or having prevented an abortion, as you'll be crucified by the other side.

To tie this back to the original topic, abortion views are an extreme form of what is an overall societal problem. Namely: we're uncomfortable with uncertainty. This is similar to the positive science bias described. No one wants an "I don't know" view, and it's difficult to sell drugs or make public policy based on that. Thus, we prefer strong views that make decision-making easy.

Re:Feelings are more important than science

[exploder]

I read "grow" as "produce", too.

Re:Feelings are more important than science

[Anonymous Coward]

This post is too rational to have come from a human.

What planet are you from?

Re:Feelings are more important than science

[Anonymous Coward]

A republic only slows and moderates the effects of direct democracy.

There is a correct answer: always let people choose for themselves!

Re:Feelings are more important than science

[pnot]

During most of my career in pure physics I got negative results, which were always hell to publish... if scientists don't report on them they will continue to seem like good ideas to people who haven't tried them, who will then waste effort on trying them, and fail to publish them when they don't work...

I agree entirely. It's infuriating to think of the wasted effort of thousands of scientists worldwide repeating each other's mistakes, just because negative results often aren't considered publishable. I think there's growing awareness of this, though. A physicist friend recently pointed me at the Journal of Unsolved Questions ( http://junq.info/ ) which is attempting to correct this bias. His advice was along these lines: ‘It's a great place to publish interesting negative results, because they've just started so there's not a lot of competition for publication, but as soon as the world figures out what a good idea this is, their impact factor's going to start taking off’. Worth a shot, I suppose.

Re:Feelings are more important than science

[haruchai]

How exactly is Juggs a "smaller" magazine?

Re:Feelings are more important than science

[uninformedLuddite]

When people talk about public education in the USA I always like to post this quote from John Taylor Gatto

In 1882, fifth graders read these authors in their Appleton School Reader: William Shakespeare, Henry Thoreau, George Washington, Sir Walter Scott, Mark Twain, Benjamin Franklin, Oliver Wendell Holmes, John Bunyan, Daniel Webster, Samuel Johnson, Lewis Carroll, Thomas Jefferson, Ralph Waldo Emerson, and others like them. In 1995, a student teacher of fifth graders in Minneapolis wrote to the local newspaper, "I was told children are not to be expected to spell the following words correctly: back, big, call, came, can, day, did, dog, down, get, good, have, he, home, if, in, is, it, like, little, man, morning, mother, my, night, off, out, over, people, play, ran, said, saw, she, some, soon, their, them, there, time, two, too, up, us, very, water, we, went, where, when, will, would, etc. Is this nuts?"

Re:Feelings are more important than science

[OeLeWaPpErKe]

So ? Humans use animals to help them build things. Or at least we did for a long while.

Re:Feelings are more important than science

[Savantissimo]

"I try to put a section in every paper entitled something like "Things That Did Not Work So Well", because any experiment or computation or theory is likely to involve some dead ends that seemed like a good idea at the time..."

Things That Did Not Work So Well:
Accidentally applying coffee to the superconducting field magnets seemed to have a negative effect on both. Application to theorists, whether internally or externally, tended to increase volubility without notably increasing comprehensibility; while a positive qualitative effect was noted on ingestion by experimentalists, particularly graduate students.

Re:Feelings are more important than science

[LunaticTippy]

That is the most sinister snopes article I have ever seen. They mention the manufacturer's claim that it takes 45 seconds to explode a twinkie in a microwave. That will cause millions of curious people to microwave a twinkie until it explodes, wreaking havoc across the globe and goosing twinkie sales.

Re:Feelings are more important than science

[r1348]

Are you suggesting that ants purposedly board trans-oceanic cargo ships?

Re:Feelings are more important than science

[Anonymous Coward]

That depends entirely on your definition of purposefully. They're certainly not doing it accidentally. If you mean that they plan out on a map where there going, then no, they don't. But they definitely board trans-oceanic ships with the intention of travelling a long way, yes.

Phrasing

[Anonymous Coward]

Well you could have phrased that friendlier. Maybe positive bias only needs a little spin to leave the bad rep. behind?

Re:Phrasing

[NatasRevol]

Yeah, that'll fix everything.

Give it a media kit & time on late night shows.

Wow! I guess Science HAS become a religion

[Anonymous Coward]

Nuff' said

Re:Wow! I guess Science HAS become a religion

[AwesomeMcgee]

Science is a method dingbat, anyone who puts faith in a scientist however is practicing demagoguery.

Practice science, not demagoguery.

Re:Wow! I guess Science HAS become a religion

[samjam]

I've not seen it put more concisely than that yet - well done.

In relation to your second line: Practice of science is more engineering that science -
In theory there is no difference between theory and practice, but in practice there is.

Re:Wow! I guess Science HAS become a religion

[BobMcD]

Saying 'science is a method' is like saying 'Christ was a Jew'. True, but it doesn't change what happened.

Science was an idea designed to seek empirical truth. To find things in such a way that those who followed after could find them again. Then people got a hold of it and started using it as a means to control one another.

Christ (even from the atheist point of view, so bear with me) had a simple message of love being service to your fellow man. Then people got a hold of it and we get monstrosities like the Crusades.

That's where the 'HAS become' part of the above phrase kicks in...

Re:Wow! I guess Science HAS become a religion

[fortfive]

I approve of this message. I was going to say, "although perhaps religion wasn't the best analogy," until I realized that it was. Science today, as opposed to say 30 years ago, appears to suffer from religiosity, or more precisely, orthodoxy. Orthodoxy excludes what does not fit within the prescribed framework, thus limiting itself and limiting progress. That process is useful in some contexts, but science is not among them.

Re:Wow! I guess Science HAS become a religion

[dzfoo]

Actually, he said "science is a method dingbat," whatever that means.

Re:Wow! I guess Science HAS become a religion

[Gr8Apes]

Actually, he said "science is a method dingbat," whatever that means.

The importance of commas....

Re:Wow! I guess Science HAS become a religion

[phantomfive]

Science was an idea designed to seek empirical truth.

lol would you say that this is another way of saying, "science is a method designed to seek empirical truth?"

Re:Wow! I guess Science HAS become a religion

[BobMcD]

I'm speaking toward the change from the original concept to the application today.

As in, it was dreamed up as a pure thing, but that's no longer the case.

Re:Wow! I guess Science HAS become a religion

[phantomfive]

It was never a pure thing. Today it's probably closer than ever.

Obvious Complex System

[Anonymous Coward]

Positive Bias is another word for Group Think. I guess it could also mean deception

Re:Obvious Complex System

[ballpoint]

Positive Bias is another word for Group Think. I guess it could also mean deception

Replying so parent gets noticed despite a downmod 'Overrated' from score zero.
Please read the link as the behavior it exposes is highly relevant to the article.

Re:Obvious Complex System

[kenboldt]

Agreed. If I had mod points I would certainly be modding the parent up.

[sarc] But, let's not let facts get in the way of a good ol' apocalyptic scare story... [/sarc]

Re:Obvious Complex System

[Coriolis]

For balance, please read consider the opposing view on the history of McIntyre, Briffa and the Yamal data. And no, the two terms are not synonymous (although group think is a type of positive bias).

Re:Obvious Complex System

[Anonymous Coward]

Gavin seems to follow a different Scientific Method than I do.

The irony is of course that the demonstration that a regional reconstruction is valid takes effort, and needs to be properly documented. That requires a paper in the technical literature and the only way for Briffa et al to now defend themselves against McIntyre’s accusations is to publish that paper (which one can guarantee will have different results to what McIntyre has thrown together).

Re:Obvious Complex System

[Savantissimo]

The response just makes it even worse. Lies piled on lies, spin on spin. This appears to me a clear case of deliberate scientific fraud and orchestrated coverup.

There types of articles are moronic.

[atlasdropperofworlds]

There are "studies", and then there is observation, modelling, prediction, model testing which is this thing called science. "Studies" are bullshit. Scientific research functions as it should. I believe the OP's article is just a chunck of sensationalist BS, or utterly ignorant of what science is (and is not).

Re:There types of articles are moronic.

[jeffmeden]

There are "studies", and then there is observation, modelling, prediction, model testing which is this thing called science. "Studies" are bullshit. Scientific research functions as it should. I believe the OP's article is just a chunck of sensationalist BS, or utterly ignorant of what science is (and is not).

You forgot to mention that it is yet another piece of published work that suffers from positive bias...

Re:There types of articles are moronic.

[fearofcarpet]

There are "studies", and then there is observation, modelling, prediction, model testing which is this thing called science. "Studies" are bullshit. Scientific research functions as it should. I believe the OP's article is just a chunck of sensationalist BS, or utterly ignorant of what science is (and is not).

That is not really what TFA is talking about. Daniel Sarewitz is re-phrasing a long-known problem with "studies," as you call them, which is that complex systems are--by definition--too complex to study as a whole. I am a physical scientist, which means that I typically make or measure something in a well-controlled experiment and then change variables in order to test a hypothesis. I can basically publish a paper that says "we tried really, really hard to find it, but it wasn't there." In the life sciences, they are trying to answer vague cause-effect questions like "does this drug affect a particular type of tumor more than a placebo." Thus researchers in those fields have to create models in which they can control variables. He gives the example of mouse models, which are obviously imperfect models for human physiology. How imperfect is the question. The creeping phenomenon that he is addressing is the tendency to relax the standards for what counts as positive evidence--and I'm grossly oversimplifying--by waving your hands around about how mouse models are imperfect, but that there is definitely "a statistically significant trend." The root cause is simply the ridiculous amount of pressure that life science researchers are under to publish, which requires results, because their methodology is standardized. Those poor bastards can spend eight years on a PhD project that goes nowhere or burn four years of their tenure clock figuring out that their experimental design was flawed. *Poof* no funding, no tenure, no degree, time to consider a new career. That sort of potential downside creates the sort of forced-optimism that TFA describes.

Re:There types of articles are moronic.

[ceoyoyo]

We generally call observation, modelling, prediction and model testing "studies."

The problem is that a lot of the observation is flawed, the modelling based on that observation may be flawed, the predictions are unreliable and the model testing is insufficient.

The basic problem is not any kind of bias, it's that the majority of working scientists don't know how to do adequate stats. It's quite simple to fix. Most scientists could probably learn the majority of what they need to know in an afternoon seminar focusing on the actual problem.

Re:There types of articles are moronic.

[Will.Woodhull]

Both studies and science are affected by bias inducing forces, such as 'publish or perish' policies of institutions and grant availibilty from stakeholders with an economic interest in the results. Elsevier, et al, raise similar problems with their control of the distribution of knowledge pipelines.

About a hundred years ago traffic problems became so bad that government had to step in and legislate which side of the road drivers had to use, and who had right of way at intersections. It might be that similar legislation is needed to curb the biasing influences on research today.

It is a mistake to think that this is about the science itself. This is about controlling the environment in which the science is done so that we benefit from good science rather than science steered by whoever has the deep pockets to pay for it. The laboratory needs to be isolated from outside economic and political influences, and those influences have found too many ways to get around the barriers that used to work, sort of, in a kind of "it is insulated enough that sometimes good work can be done" way.

Re:There types of articles are moronic.

[atlasdropperofworlds]

I don't agree. There is a distinction between proper science and these so-called "studies". Physics, for example, is a science that is properly executed and routes out bias in experimentation. Chemistry is the same. Biology has some facets that are susceptible, but as Einstein once said, "In science there is only physics. The rest is stamp collecting."

Re:There types of articles are moronic.

[HornWumpus]

When English roads were being built people carried swords. Most people being right handed they passed on the left, so swords were at the ready.

When American roads were being built people carried guns. Most people being right handed they passed on the right, so rifles were at the ready.

Government didn't have to tell them.

Re:There types of articles are moronic.

[Dusty101]

Minor correction - the quote actually comes from Rutherford:

"All science is either physics or stamp collecting".

He was subsequently awarded the Nobel prize for Chemistry (1908). Presumably by some perversely vengeful chemists.

Re:There types of articles are moronic.

[Anonymous Coward]

I'm sorry but either you are insulated in a real science such as physics or completely unaware.

I have my phd in physics and did my graduate work studying polymer theory. I currently work in a biology lab because that is were the funding is. I have observed (and fought against) publishing bad data and the statistical abuse of data. I have raised my voice every time I think the evidence isn't good enough or we are overselling what we have. The papers still get published in per reviewed journals.

I have biological friends who brag about the statistical significance of their models - with only 5 repeats. I have friends who brag about 'proving' their model, even though that is impossible and their data is iffy. I have read papers in good journals where the data conflicts with the conclusion. I have had to explain that a model must predict something that is measurable other wise the model is useless.

In my experience, there are major problems in the field of biology and biomedical sciences. I just pray that I've just had bad experiences, but having worked at some big name universities, I don't think so.

Re:There types of articles are moronic.

[Will.Woodhull]

Yeah, yeah, heard this before. And said it better: more succinctly yet covering the broad spectrum of scientific purity in a few easy to understand drawings.

So other than the snob value of "physics is better", what does this have to contribute to the current discussion? Nothing that I can see. If all there was was physics, our fastest mode of travel would be wind powered sails and the fastest way to transmit messages would be line of sight bonfires and semaphore towers.

Re:There types of articles are moronic.

[atlasdropperofworlds]

By reading, you would have seen that I said the article in question considers your typical study to be science. I hope I don't need to offer further explanation.

Re:There types of articles are moronic.

[Will.Woodhull]

If the readers are failing to understand whatever it is that the writer wants to communicate, the problem lies in the writer's scope of action, not with the readers.

Perhaps you intended to communicate something that was different from the meaning of the words that you used. That does happen. The solution is to learn to write better. Become better with your choice of words, more clear in your argument, and so forth. You will not learn this in science classes, but it is taught in higher level writing and language classes. Go ask an English teacher.

Or, horror of horrors, perhaps the words you use in your head when you think about the subject are too fuzzy or too tied up in some personal meaning to let you describe reality to yourself in an approximately correct way. That happens, too. You bring your own little packet of biases and misconceptions with you wherever you go, and you project them onto the world around you; you should never trust what you see because you can never be sure how much you are seeing is objective reality and how much are reflections of your personal biases. Dealing with this problem is also outside the realm of science classes. There are methods of getting this under control that can be learned, but you need about a decade of experience as an adult as prerequisite before beginning to learn how to think critically. Learning how to use your mind to critically evaluate the way you use your mind is very difficult and requires levels of emotional maturity and self confidence that are simply not present before the onset of early middle age.

There is one important clue about self-deception from reflected biases: as the biases arise from within the mind rather than from external stimuli, they typically present themselves more clearly than what is really going on. So when something is clearly obvious to you, that should be taken as an indication that it is probably your own bias reflected back upon you rather than the reality Out There. Do not trust yourself when you are sure of yourself, because you are probably oh so wrong.

Re:There types of articles are moronic.

[Will.Woodhull]

I should add that the scientific method, when fully applied with demonstrably repeatable experimental results, is a very effective tool for getting around the problem of reflected bias. With regard to the experimental results, only. Unfortunately the reflected bias problem comes into play again as soon as one begins to think about the implications of the results. And it is in full sway in discussions about how science should be done.

In fact, typically young scientists are in a worse position than liberal arts or business majors with regard to dealing with their own biases. Science majors tend to be so focused on the techniques of developing and testing abstract thought models that they do not get much exposure to multicultural conflict mediation (present in the liberal arts schools) or managing conflicts of interest between stakeholders (present in business schools). Liberal arts and business majors almost always bump into some issue or other that will force them to examine their innate biases; it is built into the courses. But not so with science majors. It is much easier to graduate a class of science majors who are highly trained but have little education than it would be to do so with a liberal arts or business program.

Re:There types of articles are moronic.

[atlasdropperofworlds]

I'm not as verbose as you may like. I prefer to be laconic. Fewer words *usually* begets fewer misinterpretations.

Re:There types of articles are moronic.

[atlasdropperofworlds]

This is exactly what I was saying: Studies alone don't have enough rigor to be called 'science'. I partly blame whoever the OP was for his choice of words when titling this discussion "Positive bias could erode public trust in science", because he chose a phrase that is symantecly ambiguous. It's easy to apply the positive bias to this concept of studies. A better title would have been "Positive bias of common studies could erode public trust of science" - not great but a definite improvement. Even /. doesn't seem to be able to get away from sensationalism, or at the very least, poor titles and descriptions.

Re:There types of articles are moronic.

[atlasdropperofworlds]

Following the scientific method exposes bias over time. There is no quick fix, but eventually the models have to make other predictions to graduate their way to theory. Sooner or later, the biased ones get found out.

There is a whole separate problem where over-eager journalists misrepresent early experimental findings. That type of behaviour probably does more damage than positive bias alone.

Re:There types of articles are moronic.

[ceoyoyo]

There's an unfortunate habit of a lot of people, not just journalists, to regard journals as a record of Truth. They're not. Lots of things get published, and SHOULD get published that aren't very reliable. The authors should be up front about their unreliability of course. When you see something interesting you publish it so other people can independently look at it. Part of the job of a scientist is to read the literature and evaluate it's state, and the state of individual papers.

Re:There types of articles are moronic.

[Will.Woodhull]

Verbosity is not the issue. Succinct is good. Laconic-- not so much.

Develop a larger vocabulary, learn to use it properly, and learn how to think critically and correct your own biases.

Re:There types of articles are moronic.

[atlasdropperofworlds]

Do you now feel you've sufficiently added to the discussion?

Re:There types of articles are moronic.

[Will.Woodhull]

No. I feel like I should have stopped several posts ago.

I recognize that in these circumstances there is very little likelihood that the person I am addressing will benefit from my efforts. But as this is a public forum, there is the possibility of a silent audience that might find something of value in the continued discussion, so sometimes it is worth the effort to throw a comment or two against what is clearly a stone wall. However this thread has gone well beyond the point where any third party with any sense is likely to follow, so for the last few volleys it has probably been just you and me. I should have dropped out a while ago. But I do not always use good judgment.

So goodbye now; you win; I concede; the game is yours; please award yourself however many points you think you deserve; give your ego an extra stroke from me, gratis. It has not been enjoyable nor enlightening for me, but it has shown me I need to exercise better judgment about terminating this kind of thread. So thank you for that.

Have a good day.

Re:There types of articles are moronic.

[atlasdropperofworlds]

Yes. Looking down the nose of your notoriety at those who disagree with you is never a good idea.

Wait, what?

[bmo]

'Nothing will corrode public trust more than a creeping awareness that scientists are unable to live up to the standards that they have set for themselves,' he adds.

No, the corrosion of public trust is the incessant idiocy coming from Fox and other Murdoch properties exclaiming "oh those silly scientists got it wrong again!" when the story is about a refinement of a model or something.

Scientists are losing the credibility war because scientists are not PR flacks and are unable to counteract the "we don't have to report actual news, we got a court order saying we don't" assholes at Fox.

There is a concerted effort to discredit scientific research no matter what it is.

--
BMO

Re:Wait, what?

[Anonymous Coward]

There is a concerted effort to discredit scientific research no matter what it is.

--

Now now, you know that's not true. Any scientific research that can be distorted and manipulated to suit Fox and other right-wing agendas is given entirely too much credit.

Like say, if there was some badly conducted study showing that white men were more intelligent than white women, black men, Asian women. They'd try to prove they were smarter than Asian men or even Jewish Men, but they'd rather just exploit prejudices against them. Still, such a study will be trumpeted as if it validated every prejudice they had and made to show why they need to discriminate more!

And they're probably still looking for "scientific" evidence that gays are more likely to cause Global Warming.

Re:Wait, what?

[SteelKidney]

Or the hordes of bloggers, forum commenters, and internet shills claiming that "This is absolutely, unalterably correct and anybody entertaining even reasonable skepticism is an IDIOT".

Re:Wait, what?

[Anonymous Coward]

Oh yes, blame it all on fox, the ultimate evil in the universe. It has nothing to do with studies being published making lavish claims which are then later proved false, or so wildly overblown that it's almost embarrassing. Of course, the conduct of the scientists themselves couldn't possibly be at fault and it must all be an even republican conspiracy.

Grow up, pull your head out of your ass, and realize that fox and republicans aren't the only source of evil in the world.

Re:Wait, what?

[tp1024]

Not being an American, I don't know what FOX is putting out. But if they are pointing out that any arbitrary number of "X causes cancer", "Y prevents cancer" and "Z causes heart disease" studies are bullshit, then it is not merely their perfect legal right to do so, but they are also delivering an accurate describtion of such "research".

Just because somebody is a murderer and thus a bad man, doesn't mean he also picked your pocket, ran a red light and parked his car in front of your garage.

Re:Wait, what?

[gandhi_2]

This is a story about actual bias in scientists which is affecting the quality of their research.

And you completely gloss over that to take issue with Fox, which is no more ore less biased than msnbc, et al...
Look, people seek an echo chamber. "News" companies of all types just supply the demand.

I suppose YOU don't see a problem with some news organizations taking biased scientific output and unquestioningly running with it as though it were the concrete truth for ever more.

Re:Wait, what?

[Anonymous Coward]

Right

Re:Wait, what?

[AlecC]

The trouble is that most such "X causes cancer" statements come from the media themselves, recklessly shortening research results saying "consumption of X correlates with a positive increase in cancer", where the nature of the correlation is unknown and the increase is very small. So it is all to often not an accurate prediction of the research. Particularly, there can often be a chinese whispers effect, where the researcher publishes a paper, the University PR department publishes a precis edited for PR purposes, a popular science journal then reports with its own bias, and it is then taken by mainstream media and truncated again.

A particular example, as often reported b y Ben Goldacre, is the Daily Mail, which seems to summarise everything into causing or curing cancer, and will report the same substance on both sides within days.

Re:Wait, what?

[jo_ham]

angry much?

Damn right scientists are angry. The pervasive anti-intellectualism and well-funded attacks on science to suit political ends are getting ridiculous. Positive bias in peer review is a problem that needs to be addressed, but it is all part of science - studies that cannot be replicated are examined in detail and their models either rejected or refined to be tested again. Just because something is published does not make it "the truth" - it means it is a set of conclusions based on experiments that were done by a particular team of scientists. If it's not repeatable then future publications will say so - that's how the back and forth and refining of models happens. Things don't get held back until the issue is "settled" then get published, it simply doesn't work like that.

Where things start to fall down (and where the anti-science folk with an agenda have such an easy time) is that it can be hard for the layman to sort out what to trust in a scientific publication, since the well-hammered, repeated-by-many-groups stuff is sitting alongside single publications claiming XYZ based on PQR from a single data set in a single research group. It's so easy to wade in there and say "look at this! see! scientists can't agree! it's all a big con to get more money! they'll say anything to further their agenda!".

Again, I'm not dismissing the problems of positive bias - it's a factor of peer review and the way human beings approach the reporting of science, but the job is made much harder by a barely-science-literate blogosphere working full steam to discredit anything they can to make it look like there's some sort of global conspiracy of scientists working against the general public in whatever field the propaganda machines are working against. Climate change is clearly the biggest one at the moment, but it's not restricted to that. There's almost anything related to things that challenge fossil fuels and energy research towards energy independence, then there's anything in the biology field relating to stem cells, vaccinations, pharmaceuticals, etc.

The rise of public opinion that science is somehow something to be regarded with great suspicion and that scientists are actively working against the public good is not only troubling, it's highly counterproductive. It's time we started getting angry about it, it's just almost impossible to fight back - scientists are not equipped to do so against a highly organised and well funded group of individuals who do that sort of character assassination and propaganda for a living.

Re:Wait, what?

[thegreatemu]

While I agree that models are frequently refined, leading to new results, there is a disturbing trend I see, not having to do with positive bias necessarily, but with uncertainty estimation.

One thing that I've found incredibly hard to beat into undergrads taking my physics lab courses is that getting your uncertainties (or error bars) right is far more important than getting the right central value. This is because uncertainties are the only way that two experiments can be compared against each other, or the only way to compare experiment to theory. If I have two models of climate change, one of which predicts a temperature rise of 3 C ± 5% and another that predicts 4 C ± 7%, those results are in large disagreement, whereas two studies that predict 20 C ± 15% and 40 C plusmn 35% are in much closer agreement.

But I see it seems much more frequently, especially in fields like astronomy, too little thought goes into the systematic uncertainties, and you'll get 4 experiments measuring the same thing with results that cannot be reconciled if you take their statistics at face value. This was a huge problem with many of the early global warming predictions as well; every year a new estimation would come out that was completely incompatible with the previous one. Yes, these models are insanely complicated, and it's damn hard to understand all the systematics. And of course you can't put in error bars for plain old mistakes. But do it too many times, and people begin to lose any faith that your estimates can be relied on for anything.

This is the problem I see; not necessarily bias toward a positive result, but a bias toward underestimating the uncertainty of your measurement, which I suppose could be different sides of the same coin. (E.g., a result of 2 ± 0.1 is a positive result; a result of 2 ± 5 is not!).

Re:Wait, what?

[invid]

In order to understand what this bias means, people have to understand the scientific process, and many people don't. Such people would be more convinced by scientists pontificating on high that they are always right, the way religious leaders do.

Re:Wait, what?

[bmo]

You mean like the anti-vaxers?

Two years ago last February, the Lancet retracted "Doctor" Wakefield's "study" and you still see this shit.

--
BMO

Re:Wait, what?

[Anonymous Coward]

No they have been found to be deliberately lying, and that it is in fact part of their corporate policy. They have however not been convicted since they argued their channel "Fox News" is not a news network but a news-themed entertainment channel, so they don't have to report factual news. That is the loophole that allows them to have news-like programming that spreads obvious falsehoods.

Re:Wait, what?

[ArcherB]

'Nothing will corrode public trust more than a creeping awareness that scientists are unable to live up to the standards that they have set for themselves,' he adds.

No, the corrosion of public trust is the incessant idiocy coming from Fox and other Murdoch properties exclaiming "oh those silly scientists got it wrong again!" when the story is about a refinement of a model or something.

Scientists are losing the credibility war because scientists are not PR flacks and are unable to counteract the "we don't have to report actual news, we got a court order saying we don't" assholes at Fox.

There is a concerted effort to discredit scientific research no matter what it is.

--
BMO

God forbid you get more than one side of any story. You should thank God that FoxNews exists. They are the only news organization that actually reports on things that might make the current government look bad. For example, yesterday there was a report out about stuff Mitt Romney did in high school, 45 years ago. Can anyone tell me what grades Obama got in college 20 years ago? FoxNews is the only organization that even looks for these answers while the rest of the media is busy showing Obama swat at flies, say that he supports gay marriage (again), report the good news that gas prices have dropped a few cents (after going up a dollar), and reporting the winners of the latest "Dancing with the Stars".

The job of the press is to look for information, report what it finds, and act as a government watchdog. FoxNews is the only news organization doing that job. FoxNews is not Pravda. They'll actually report some of things you don't want reported. I'm sorry if you are mad that I'm getting information you don't want me to know, but I'd prefer to make up my own mind instead of being told what to believe without any counter argument.

Just because they report stuff you don't like does not mean that they should be silenced. There is that whole Bill of Rights thing that is supposed to protect them. Do you agree with the First Amendment or not?

Re:Wait, what?

[Anonymous Coward]

Grow up, pull your head out of your ass, and realize that fox and republicans aren't the only source of evil in the world.

But they are.

Re:Wait, what?

[Alomex]

to take issue with Fox, which is no more ore less biased than msnbc, et al...

Sorry, but Fox is the only news organization that has explicitly told its reporters that it is ok to lie.

This "everyone does it" business is just not true anymore. It used to be, but for the last decade or so Republicans and their supporters have taken the lead in uncivil discourse.

Re:Wait, what?

[ArcherB]

Not being an American, I don't know what FOX is putting out.

FoxNews is the only American news organization that will report issues from the right's perspective. Of course, they will also report from the left, as seen yesterday when Shepherd Smith, one of the most popular anchors at Fox, praised Obama for his stance on gay marriage. The difference is that every other news organization in the US will only report issues from the left. On most networks you will hear that the Keystone Pipeline was cancelled because it would harm the environment. You won't hear that the report claiming that was one of many, most of which said that there would be no problem or that there are plans that would move the pipeline to a less sensitive area. You will hear repeatedly how the GDP grew x.y% last quarter, which was better than expected, but you won't hear, a month later, that those numbers have been revised and that the economy actually performed worse than expected.

The difference is that FoxNews will report both sides of these issues, including the right. The problem is that because FoxNews includes the right's view when no one else does, this puts FoxNews far to the right of every other news organization. This is seen by bias from all that are left of center and they fell that FoxNews must be destroyed or discredited (FauxNews).

Personally, I find it rather scary when one side wants to completely silence or destroy all who disagree with them. You would think these people would learn from history.

Re:Wait, what?

[bmo]

You should thank God that FoxNews exists

Fox is the only "news" organization, ever, that went to court in the US to prove that they don't have to report news.

You are a complete and utter moron if you give whatever they say any credibility at all. This is not ad hominem. This is their own claim that whatever they report does not have to be based on any facts at all. None. Zero. Nada. They can make shit up out of whole cloth if they want. They have a Ruling saying they can.

--
BMO

Re:Wait, what?

[ArcherB]

This is a story about actual bias in scientists which is affecting the quality of their research.

And you completely gloss over that to take issue with Fox, which is no more ore less biased than msnbc, et al...
Look, people seek an echo chamber. "News" companies of all types just supply the demand.

Not only that, he was modded (Insightful +5). It's like they didn't even see the "Offtopic" option.

I suppose YOU don't see a problem with some news organizations taking biased scientific output and unquestioningly running with it as though it were the concrete truth for ever more.

Only if he agrees with the report. Of course, he wouldn't agree with it if he didn't think was true, so he actually thinks that these reports are the truth. The problem is that he believes that anyone who disagrees with him must be wrong, and therefor should not allowed to report their findings because they are not the truth. It never dawns on him that he might actually be wrong.

Think of it this way. He believes that no "credible" scientist believes that there are problems with the idea that global warming is caused by man. This is because all scientists who disagree with that view, no matter how sound their research is, immediately loses credibility when they question the man-made GW idea.

Re:Wait, what?

[Anonymous Coward]

I would suggest it all started with Al Gore and Global Warming

He was the first to use Science as a cudgel against his political foes and advance his political agenda. He's the one that put Science in the fight.

That was then later compounded by the outrageous catastrophic predictions (sure...say they didn't "predict", but the words came out of their mouths and they didn't object when others took those words for a prediction) that didn't pan out. Isn't Florida supposed to be underwater by now?

Then the whole Climategate thing that showed the primary movers of AGW to be complete jerks and thoroughly unlikeable people.

And there's jerks like you who claim that anyone who dare look askance at your work are "anti-intellectual" are are too stupid to sort out what to trust in a scientific publication (are you saying that scientific publications, Nature, et. al. have untrustworthy material in it?)

Of course, equating people who don't drink your Kool Aid to those who deny the Holocaust really helps your cause to be seen as our nights in shining armor.

How about scientists climb down off their high horses, admit mistakes instead of trying to cover them up, and stop being pedantic, obnoxious jerks?

Re:Wait, what?

[AngryDeuce]

"This is absolutely, unalterably correct and anybody entertaining even reasonable skepticism is an IDIOT"

Nobody has a problem with reasonable skepticism. It's unreasonable skepticism, such as that you often hear in the far-right wing media, that people are getting sick of.

The anti-vaccination assholes, and the people that just refuse to believe that mankind is having an effect on the climate, are two good examples of where you see a lot of unreasonable skepticism, but there are plenty others...

I knew the far-right had officially gone full retarded when I had someone tell me climate change was a myth because Al Gore. Seriously, that was the reason why it was a myth: Al Gore. Give me a fucking break.

Re:Wait, what?

[Anonymous Coward]

I have you my mod points, and I hope other people mod you up as well.

Margins of error are critical to understanding research. I find it interesting that margins of error are (almost) always displayed when news organizations are talking about political polls, but when it comes to scientific research, it can't be found. Going along with the topic of climate change that you mentioned, I can't remember the last time I saw the margin of error reported anywhere in the news story itself. If I want to find it, I have to go find the original study and hope that it's in there.

It's true that the majority of people in the world (not just Americans; we're not a special kind of stupid, there is stupid everywhere) simply don't understand what this stuff means and why it is important.

I wish that the major news organizations would turn over all reporting on scientific stories to an actual scientist, who isn't pressured to create sensationalist headlines and will explain the study and what it actually means. That will never happen though...

Re:Wait, what?

[Anonymous Coward]

Fox News lies, but no one else does? I know of ABC News getting caught lying, NBC News several times (the Ford Explorer blowing up, the recent editing of the Zimmerman 911 call), CBS News (the fake docs about Bush in the National Guard).... But I don't know anything about Fox News getting caught straight out lying.

Re:Wait, what?

[Anonymous Coward]

No, the Anonymous Coward above you got it right. Fox News somehow managed to argue (successfully) to the Supreme Court that something calling *itself* a news show has no duty to tell the truth, and should be allowed to *knowingly* lie to its audience while claiming to report the truth. The disrespect for Fox News in rational circles has nothing to do with 'political leanings', or being "the only American news organization that will report issues from the rights' perspective". Its a simple lack of respect for someone who claims to report the News and Facts, but when called on *knowingly* lying to its audience claims to 'just be news-styled entertainment'.

The very idea that Fox 'News' is the only place reporting the 'right's perspective' is absurd on its face. News channels (the real ones) report the News based on the facts available. If those facts rarely line up with 'the right's perspective', then it's not a problem with the news, or the facts.

Re:Wait, what?

[Anonymous Coward]

Wait, let me get this straight. When pro-AGW folks scream that we are all going to be underwater in 20 years, push legislation through Congress, impose radical agendas in schools, look to put whole industries out of business all because of these eminent scientists predictions, we are not allowed to say "we told you so" when the predictions are eventually shown to be a pile of shit, like many folks have been saying all along. Oh yeah, and the folks who have been criticizing the BS predictions are treated like dog shit, and STILL the pro-AGW folks go about their business like the original predictions are still going to come true.

I think you need to reevaluate your approach to the subject. Maybe YOU should realize that you are being used for political gain, all on the backs of these biascientists whoare pushing an agenda over the science they are supposed to be conducting.

In other words, you are supposed to question everything, as long as you don't question ANYTHING involving the "Consensus" of AGW.

Re:Wait, what?

[joeboomer628]

One of the reasons that I read Slashdot and other internet information aggregates is because I distrust all traditional news media outlets. At least I get to hear various sides of arguments and if I am informed enough about a subject I can usually discern the truth. In theory journalists are unbiased. Anyone care to evaluate what happens in practice. One of the conditions of employment for New York Times employees that are involved in producing news content, is that they abstain from making political financial contributions. This is to protect the papers image as an unbiased reporter of facts. Two employees who were subject to the rule were listed in contributor public disclosure documents of a political candidate. Their "punishment" was a requirement to ask for their money back. Lack of accountability is the root cause of false reporting due to bias.

Re:Wait, what?

[ArcherB]

You should thank God that FoxNews exists

Fox is the only "news" organization, ever, that went to court in the US to prove that they don't have to report news.

You are a complete and utter moron if you give whatever they say any credibility at all. This is not ad hominem. This is their own claim that whatever they report does not have to be based on any facts at all. None. Zero. Nada. They can make shit up out of whole cloth if they want. They have a Ruling saying they can.

--
BMO

How many news organizations report false stories? Here's a hint; ALL OF THEM. It happens. We are human. It's USUALLY a mistake.

The difference is when FoxNews makes a mistake, they are sued by someone like yourself, only with extra money laying around, trying to silence the opposition and run FoxNews out of business. Did anyone sue to strip the license of NBC when they released a modified 911 call in the Trayvon Martin shooting? Did CBSNews have to go to court to defend their right to report stories that turn out to be false after that blatantly false Bush National Guard story?

Why is FoxNews different?

Also, the court case you are referring to was not about FoxNews, but a local Fox affiliate. The local affiliate was sued by a former reporter who didn't like the fact that they were made to include Monsanto's side of the story, which they claimed to be false. The part you are referring to where the court said the news organizations were allowed to lie did not come from FoxNews or the court, but a group called Project Censored, which from what I can tell is an anti-government, anti-media, anti-corporation organization. Your kinda people

Re:Wait, what?

[trout007]

I don't think it's an anti-intellectual sentament but anti-busybody. The problem comes when someone comes in the name of science to force you to do or not do something. It is the use of force people are resisting. If you remove the use of force you will have better results.

Re:Wait, what?

[Anonymous Coward]

Yes. Good old fox news! Last bastion of patriotism and truth....

As long as the patriot isn't liberal and the truth is what they want told.

We discredit it because Fox LIES all the time, just flat out lies and justifies your aberrant behavior.

MOST liberals are not actually trying to crush them.

I see them (and you) as the threat they are.

Re:Wait, what?

[bmo]

I would suggest it all started with Al Gore and Global Warming

You would be wrong and not only that, but without a sense of history at all. I can just point out Senator William Proxmire's (D, btw) "golden fleece award" given to the Aspen Movie Map with regards to "hurr, we don't understand the tech so we must be getting scammed" to Sarah Palin's bitching about silly "fruit fly experiments...hurr, we don't need those" - which comes from a long line of idiots decrying basic science, because their minds are *that* blinkered.

Never mind that the fruit fly experiments are all about genetics because fruit flies have lifespans of ... fruit flies so it makes heritability easier to study. Never mind the fact that the Aspen Movie Map was groundbreaking and you can now look back at it and say "Gee, I wonder where Google got their idea for Street View." But no, you don't hear about that. You hear on Fox that fruit fly studies are just wasting taxpayer money. Because Sarah Palin said so. Because she's such an expert in genetics. *spit*

Then the whole Climategate thing that showed the primary movers of AGW to be complete jerks and thoroughly unlikeable people.

So? It was found to be nothing more than egos. The science itself wasn't discredited, and has only strengthened since then. And I have voiced my opinions here about how badly I thought that it was being handled and my own strong skepticism about AGW in previous years. But you know what, I have recently (as in the past couple of years) found that the anti-AGW crowd to be increasingly full of *real* integrity problems and conflicts of interest.

Back to basic science:

Here's a clue for you and your buddies: You don't get applied science and engineering and fancy new products without basic science.

--
BMO

Re:Wait, what?

[jo_ham]

And there's jerks like you who claim that anyone who dare look askance at your work are "anti-intellectual" are are too stupid to sort out what to trust in a scientific publication (are you saying that scientific publications, Nature, et. al. have untrustworthy material in it?)

Ah, classic twisting of my words. I'm talking about anti-intellectualism as a movement. Those doing it are extremely smart - that's what makes them good at it. They're good at duping people into following their cause. I am not personally calling individual people stupid unless they actually demonstrate that they are.

And yes, I'm absolutely saying that scientific publications contain untrustworthy information, including the big hitters like Science and Nature - their size and prestige is no assurance of infallibility, and in fact can work against them since people are often reluctant to question them. That's the nature of scientific publishing - until results have been replicated, single-source experiments and models need to be looked at with extreme skepticism.

Last year I performed some work that disproved a piece of published literature (in a non-controversial area of chemistry). I didn't set out to disprove it - I set out to see if I could replicate the results and I determined that the published paper was incorrect. My own conclusions, method and data set were published in response, with some discussion on why the previous paper was drawing incorrect conclusions (mainly an issue with experimental control). My situation is one that is repeated constantly - it's how science works. The stuff that can't be replicated is corrected, the stuff that is replicated becomes more solid.

It's not the layman's fault that they don;t understand some of the intricacies of how peer review and scientific publishing and research works. They're not stupid for not getting that in the same way that I'm not stupid for not understanding the first thing about programming - it's simply not my area of expertise. Where the stupidity *does* arise, however, is when people start to distrust scientists out of hand because they're being told to do so by certain media outlets or special interests. It happened with vaccinations due to a corrupt doctor manipulating a very weak study with the ultimate aim to push a competing vaccine made by a company that paid him off, but it backfired spectacularly - far from getting the competing vaccine popular, people rejected vaccination entirely against their own interests, putting their own and everyone else's kids at more risk. It's this sort of media frenzy and associated public panic and distrust of science (even now, people refuse to believe scientists on the issue, despite the original study being totally debunked and Wakefield himself being struck off the medical register et and the whole thing exposed as a sham).

That's the sort of thing I'm talking about here. We saw it with the MMR vaccine, we see it with nuclear power, we see it with stem cell research, we see it with GM foods (and there *are* some legitimate issues to be raised there, being drowned out by typical media hysteria), we see it with climate science - again, there are legitimate issues to be raised and discussed on a topic that is *gigantic* in scope in the scientific community, but it's being drowned in so much media hysteria and political propaganda that it's almost impossible to get anything done.

Of course, equating people who don't drink your Kool Aid to those who deny the Holocaust really helps your cause to be seen as our nights in shining armor.

Where did I say that? You're dangerously close to Godwining the thread by trying to imply that I brought that up when I did no such thing. The hyperbole serves no one, it only makes your arguments look weak.

I'm not looking to be anyone's "night [sic] in shining armor", nor are most scientists. We just work on the science in our field and go where that leads us. If we wanted to be knights rescuing people I'd have joined the fire service or something. I became a scientist to ultimately help mankind and further our collective knowledge, but I'm no superhero or white knight.

Re:Wait, what?

[bmo]

How many news organizations report false stories? Here's a hint; ALL OF THEM. It happens. We are human. It's USUALLY a mistake

False equivalence is false and you are delusional and you have fallen for their trap that "everyone lies, so trust us, we don't lie" crap, when it's been proven in court that they lie.

I don't know what to tell you man.

but a local Fox affiliate.

But then Fox Corporate came out and trumpeted this as a big win.

So much for distancing themselves from lies.

--
BMO

Re:Wait, what?

[jo_ham]

"Scientists" are not forcing anyone to do anything - that's what politicians do. We just do the science. Now, if we end up discovering, for example, that CFCs are damaging the ozone layer, or that PbEt4 in gasoline is poisoning the air then we bring that to the attention of people who can make a decision on what to do about that. We can certainly offer suggestions of what to do about it - stop using CFCs (and develop alternatives) and stop using PbEt4 in gasoline (and work on alternatives) etc, but ultimately sometimes changes have to be made that people aren't going to like.

We don't enjoy "telling people what to do" or set about to discover things that will allow us to dictate to people what to do - ideally we want to find solutions to things that have a minimal impact on people's lives.

We're not in this to control people - we just do science. We don't want to "force" people to do anything, but sometimes what we discover necessitates change in order to benefit everyone as a whole (like removing lead from gasoline, or switching to non-CFC propellants and refrigerants). That's just life I'm afraid.

Re:Wait, what?

[FrootLoops]

While the rest of your post strikes me as rather crazy, the last sentence leads to a decent "case study" on the misuse of science by fundamentalists. Research is a theme in the anti-gay movement. Both the Family Research Council (FRC) [a leading anti-gay group] and the National Association for Research and Therapy of Homosexuality (NARTH) [a leading ex-gay group] have it in their names. The anti-gay people who reference studies usually focus on health and other social problems. Topics include higher incidence of HIV and various STDs, various intestinal parasites, anal cancer, domestic violence and murder, promiscuity, mental health problems and suicide, whether it's a choice or genetic, illegal drug use, and pedophilia.

Here's an article about the American College of Pediatricians [the comparatively tiny anti-gay offshoot of the mainstream American Academy of Pediatrics] distorting some research:

The ACP also claimed that the longer you can keep kids from identifying as gay, the less likely they are to kill themselves. ...
In this case, Remafedi says, the ACP missed the larger point: Kids who come out at a younger age are more likely to kill themselves because they are less able to deal with the stigma and isolation of being gay. If anything, the research shows the need for more support.

"It's obvious that they didn't even read my research," Remafedi says. "I mean, they spelled my name wrong every time they cited it."

It's easy to put up a front of respectability. Here's a nice example from the Family Research Council: an hour long lecture against gay marriage. It's offered by the "FRC University Library", which is almost as respectable-sounding as the American College of Pediatricians. The format is a traditional lecture given by a middle-aged guy in a suit. If you actually listen to him, he rambles incessantly and when he makes a point his reasoning is often highly flawed (for instance, listen to his discussion of the interracial marriage analogy around 48 minutes in; the circular reasoning is just sad). But if you just look at the trappings and believe his conclusions, you might get the impression that he was a trustworthy source to base your own views on.

Re:Wait, what?

[Anonymous Coward]

This is a story about actual bias in scientists which is affecting the quality of their research.

And you completely gloss over that to take issue with Fox, which is no more ore less biased than msnbc, et al...

Other comments on this thread and ten seconds with google will give you dozens of cases where Fox reports science is a grossly distorted way. DO you have any evidence that msnbc has done the same thing, to the same degree?

Look, people seek an echo chamber. "News" companies of all types just supply the demand.

I suppose YOU don't see a problem with some news organizations taking biased scientific output and unquestioningly running with it as though it were the concrete truth for ever more.

You seem to be arguing that blatant dishonesty from your team is okay, because you are sure everyone else would do it. In reality, they do not.

Re:Wait, what?

[ArcherB]

Are you illiterate? The story wasn't lies. They allowed Monsanto to respond to the allegations levied against them. The reporter who wrote the story against Monsanto called Monsanto's response lies and refused to report it. See, that's the difference. FoxNews allows the accused to respond. They report both sides of the story. Sometimes, one side is not correct. Most of the time, they are different sides to same coin, each omitting items that discredit their side. I don't know if Monsanto's response was a lie or not as the story was from 1997. But since it was about BGH, and BGH is still used today, and we are not all bovine-human hybrids, I assume that the story was full of biased, exaggerated claims about the dangers of BGH that have turned out not to be true.

False equivalence is false and you are delusional and you have fallen for their trap that "everyone lies, so trust us, we don't lie" crap, when it's been proven in court that they lie.

I didn't say that news organizations lie. I said they make mistakes. Although, in the cases of CBS and NBC, there is a lot of truth bending and mistakes going on. Wasn't it the NYTimes that said it was OK to report a fake story, as long as the conclusion was accurate? I think they used the term, "Fake, but accurate".

Again, if lies were told, it was by Monsanto. Accusing FoxNews of lying because they reported what Monsanto said would be like accusing every news organization when they showed Bill Clinton saying, "I did not have sexual relations with that woman, Ms. Lewinski." However, CBSNews did lie when they pushed that GWB National Guard story. Why are you not railing against CBS? Speaking of bias...

But then Fox Corporate came out and trumpeted this as a big win.

Citation?

So much for distancing themselves from lies.

This is from the Wiki page for one of the reporters:

WTVT did not run the report, and later argued in court that the report was not "breakthrough journalism." Wilson and Akre then claimed that Monsanto's actions constituted the news broadcast telling lies, while WTVT countered that it was looking only for fairness. According to Wilson and Akre, the two rewrote the report over 80 times over the course of 1997, and WTVT decided to exercise "its option to terminate their employment contracts without cause,"[5] and did not renew their contracts in 1998. WTVT later ran a report about Monsanto and rBGH in 1998, and the report included defenses from Monsanto

That's what "We Report, You Decide" is all about.

Re:Wait, what?

[Anonymous Coward]

1. Get a bunch of scientists in a room to define what "Science" is, you will not find a consensus. The most common thing you will hear is, reproducibility.
2. For publicly-funded/government-funded research there is no system in place for testing reproducibility. You cannot get published saying results of a prior study were not able to be reproduced.
3. Studies are not science because of #2, but also they do not have enough statistical power to mean anything. Correlation isn't causation. You can find correlations between almost all phenomena. Small size of N, not randomly sampled, means your observations are hokum.
4. Much of the activity that is called "Science" that is funded by tax payers is not science but career development for Ph.D.s. I couldn't afford to get a Ph.D., so I suspect most of them come from families of wealth, and then probably they have psychological issues of entitlement. (My opinion).
5. The media reporting of science studies gives science a bad name. The articles about studies inevitably draw conclusions not in the study, or they turn correlations into causation.

So you are angry that things that are called "Science" are not really science, and this distorts the public's view.

After spending years engaged in supporting activities called science, I believe that real science is getting swallowed up by misinformation, media, poor education, &c. Also, I think technology is much, much more valuable than science. The models of science will always be wrong. Newton was wrong, as Einstein revised him. Bohr's model of the atom was wrong. Einstein is wrong as we'll see one day. Whereas with technology, a good needle with a polio vaccine does miracles.

Also, consider this, can you trust the findings of a scientist without a moral compass?

Re:Wait, what?

[Anonymous Coward]

Yep, because despite their name being 'Fox News', they're only a 'news-styled entertainment' show, not a *real* news cast. That's their own claim by the way, not mine, and not some 'left-wing distortion'.

Re:Wait, what?

[MightyMartian]

Then ignore Al Gore. This constant bitching about him is ludicrous, and to my mind an intentional red herring. Al Gore is not a scientist, you don't need to hear anything he has to say to understand AGW. But, I guess, when you don't like what someone is saying, you find an easy strawman to knock down.

The overwhelming majority of experts in fields related to climatology accept that AGW is real. You can call that a high horse, but I call it at least suggestive that maybe they have something.

Re:Wait, what?

[ArcherB]

No, the Anonymous Coward above you got it right. Fox News somehow managed to argue (successfully) to the Supreme Court that something calling *itself* a news show has no duty to tell the truth, and should be allowed to *knowingly* lie to its audience while claiming to report the truth.

Speaking of "lies", your comment is incorrect. The story you are referring to can be found HERE. The story was about a local Fox affiliate, not FoxNews. The case never reached the Supreme Court. Fox never said it didn't have to tell the truth.

The case dealt with Fox including a response from Monsanto, which the REPORTER said was a lie. Fox said that they had an obligation to allow Monsanto to respond. That's not lies. That's fair reporting. Just because the reporter hated Monsanto doesn't mean they don't have the right to respond to allegations.

Re:Wait, what?

[MightyMartian]

First of all, scientists aren't forcing anyone. They do not have that power. Governments do. Beyond that, history indicates that people do not make the right choices all on their own, and quite often will give in to selfishness and shortsightedness. Look at the North Atlantic cod fishery. For decades scientists warned that it was going to get wiped out, but market forces and the short-term self interest of the fishermen won out, until, that is, the population collapsed due to centuries of overfishing. Leaving the fishermen to their own devices didn't work, it backfired and a major industry of the east coast of North America was all but wiped out.

You have this idea somehow that politics can trump reality. It can't. If we're going to get hit by a tsunami, it doesn't matter one little bloody bit that some organized group stands up and says "It's all liberal lies!"

Re:Wait, what?

[MightyMartian]

I doubt you've ever been within fifty feet of a scientist.

Re:Wait, what?

[acscott]

I doubt your doubt. (Cannot believe I am replying to a troll.)

Re:Wait, what?

[MightyMartian]

Questioning is one thing, but wholesale declarations that the large majority of climatologists are liars or fools is another. What's more it's an old tactic developed by the Creationists in their attacks on evolution; declarations of a cabal of biologists out to hide the truth blah blah blah.

You act as if consensus is a bad thing. Do you think evolution is false because the overwhelming majority of biologists accept it? Do you think General Relativity and Quantum Mechanics is wrong because the overwhelming majority of physicists accept it? I'll wager you don't, but because AGW says "We keep vomiting vast amounts of CO2 into the atmosphere we're going to screw things up royally", which interferes with your short-term self interest, well, it must be false.

Re:Wait, what?

[MightyMartian]

Hardly a troll. Having spent some time around scientists, they're nothing like you describe. The trolling here is being done by you.

Re:Wait, what?

[jo_ham]

1. http://dictionary.reference.com/browse/scientific+method

2. Reproducibility means exactly that - if you follow the method set out in the paper can you reproduce the results that the paper claims? There's no "system" for testing reproducibility because there doesn't need to be one - you simply do what the authors of the paper did and either it works or it doesn't. If you haven't been given enough information to reproduce the paper's results then the paper is invalid and needs to be corrected.

3. Studies are a particular tool of science, and are one of the most abused, misunderstood and mistrusted tools we have, for all of the reasons you mentioned and more. A huge number of studies are conducted poorly for a large number of reasons. Analysing and critiquing the study itself is large part of peer review.

4. This is just nonsense rambling. I'm sure there are some "career development for PhD's" that is not "science" but it hardly describes the majority of science being doing that is funded by means other than the taxpayer. This just reads like an anti-science talking point with nothing to substantiate it. You know if you're reaching for character attacks on scientists themselves you've run out of arguments.

5. This is definitely true. The media's reporting and (often wilful, most times ignorant) misunderstanding of what scientific studies are reporting is rife and it is one of the main reasons science is so misunderstood by the general public. Quoting scientists out of context, or just plain making stuff up is also widespread. "Scientists say....", "some experts believe...." etc etc.

Also, consider this, can you trust the findings of a scientist without a moral compass?

That's a bit of a non-sequitur. You can replace "scientist" in that sentence with almost any profession. Lawyer, politician, police officer, doctor etc... How does it relate to the issue under discussion, unless you're claiming that *all* scientists have no moral compass?

Re:Wait, what?

[belthize]

Since there's no +100 insightful mod option I'll just post a simple 'thank you for this post'.

That goes for your GP post as well.

Re:Wait, what?

[HornWumpus]

How many channels where showing 5 year old pictures of Martin? How is that not lying? Edited 911 tapes? How is that not lying? Pretending that 'stand your ground' is even an issue? How is that not lying?

Who was it that took a proportional typed (produced by Word) document as being produced in 1968? How many people fell on their swords trying to defend these lies? 'Fake but accurate'?

They all lie. Most at the direction of the DNC.

Re:Wait, what?

[acscott]

Thank you for a reasonable reply. On #1 maybe this quote will clarify my point; "The consequence of these debates is that there is no universal agreement as to what constitutes the "scientific method"" http://en.wikipedia.org/wiki/History_of_scientific_method#cite_note-69 On #4, this is just from my direct experience, and I am too much of a coward right now to share. And a hint and warning how much idealism in science must be guarded and protected from the vagaries inherent in humans getting involved. :) Your responses are fair. On the moral compass question, it is a bit mischievous, hopefully just showing how important verification is.

Re:Wait, what?

[martas]

Finally. Had to scroll to the bottom of the page to see by far the most important relevant problem mentioned. You are absolutely correct, there is a serious need to increase statistical competence in all scientific fields. Statistics is, in a way, the foundation of science (since science depends entirely on empirical verification, which, as you point out, always includes uncertainty). And yet, a statistics professor I know who used to collaborate with CERN tells horror stories of them using some kind of bullshit Bayesian methods or something for estimation... An embarrassment, if you ask me.

Re:Wait, what?

[Belial6]

Actually FOX as a whole is just shock television. It just so happens that they generally get more viewers by making crazy right political statements in their 'news' shows. Conversely, they get more views by making crazy left political statement in their adult cartoons.

This may because crazy right tends to think that direct confrontation means you are correct, and crazy left tends to think that snarky 'jokes' means you are correct.

Re:Wait, what?

[interkin3tic]

To add to that, the public is going to stop trusting science based on published clinical trials they can't access anyway?

What's that going to look like? People going to go Amish?

Re:Wait, what?

[Anonymous Coward]

And you know this because you have recorded every conversation that has occured in every other news company everywhere?
Move along troll.

Re:Wait, what?

[Anonymous Coward]

That's what "We Report, You Decide" is all about.

That's what they want you to believe.

Me? I see their professions of such unbiased for the deception they really are. I've never met an honest man who put half as much effort as a liar into insisting they were telling the truth.

I know, I know, you want to believe Fox is the truth-bearer, picked on by all those mean liberals.

But it just ain't so, Joe.

Re:Wait, what?

[bmo]

I'm not going to get into an argument about minutia with you.

Because you fail to see the forest for the trees.

I'm done here. Fox lies. Period. If you don't like my opinion, deal with it.

--
BMO

Re:Wait, what?

[Black Parrot]

I knew the far-right had officially gone full retarded when I had someone tell me climate change was a myth because Al Gore. Seriously, that was the reason why it was a myth: Al Gore. Give me a fucking break.

ISTM that conservatives tend to focus on people rather than on facts. Evolution deniers somehow think they can score a point by discrediting Darwin personally. They are especially keen on real or perceived change of heart, e.g. the (false) story of Darwin's deathbed conversion, the story we had here a week or two back about some scientist saying that global warming hasn't been as radical as he had predicted, or Antony Flew buying in to some intelligent design argument a few years ago.

And they're really keen on authority figures - so long as that figure agrees with them.

They also project that mentality onto everyone else; they can't comprehend why I don't care a fig about what Darwin or Flew thought or said.

Re:Wait, what?

[Black Parrot]

Not being an American [...] delivering an accurate describtion [...]

In English it's "description", but I think I like your spelling better.

Re:Wait, what?

[Black Parrot]

FoxNews is the only American news organization that will report issues from the right's perspective. [...] The difference is that every other news organization in the US will only report issues from the left.

Yeah, "liberal" US media.

You should look at the ratio of head counts of Republicans vs. Democrats, conservatives vs. liberals, even men vs. women, that they consistently have on their Sunday morning talking head shows.

Re:Wait, what?

[tp1024]

It wasn't me who wrote this, it were my fingers ... touch typing does strange things to your spelling, you know?

Re:Wait, what?

[Black Parrot]

But I don't know anything about Fox News getting caught straight out lying.

If you care, progressive bloggers report on it all the time. It makes dull reading once you've figured out that FOX habitually misrepresents the facts, but it may be worthwhile reading if somehow you haven't figured that out yet.

They also have a habit of "accidentally" misidentifying Republican politicians as Democrats during the first few days that a scandal hits the news.

Of course, if you saw their map that labeled Iraq as "Egypt" during the Egyptian unrest, you might be willing to dismiss FOX as a collection of clueless amateurs rather than as right-wing propagandists.

Re:Wait, what?

[Black Parrot]

This is because all scientists who disagree with that view, no matter how sound their research is, immediately loses credibility when they question the man-made GW idea.

Maybe you would like to direct us to some of that "sound" research.

Re:Wait, what?

[oldhack]

It's jihadis like you who further degrade science.

Re:Wait, what?

[Uberbah]

Are you illiterate? The story wasn't lies. They allowed Monsanto to respond to the allegations levied against them. The reporter who wrote the story against Monsanto called Monsanto's response lies and refused to report it. See, that's the difference. FoxNews allows the accused to respond. They report both sides of the story.

Are you? Monsanto objected to the story, and the reporters were fired. Fox successfully argued in court that it had no responsibility to tell the truth. The parent is correct, and you are spouting right wing propaganda.

As usual.

How many news organizations report false stories? Here's a hint; ALL OF THEM.

Here's a hint: we all see your lame false equivalence and lack of proportion. It's like saying that a guy 50 pounds overweight (NBC) and a guy 70 pounds overweight (CNN) are on the same level as a guy 600 pounds overweight (Fox) because they're all obese.

How many Fox reporters were fired for showing summer footage of an old Glenn Beck rally and pretending it was new, so it would look like more people showed up?

Re:Wait, what?

[Uberbah]

How many channels where showing 5 year old pictures of Martin? How is that not lying?

Were they pictures of some other kid put in his place? Then were was the lie?

Edited 911 tapes? How is that not lying?

And the people responsible were promptly fired when NBC found out. How many people at Fox were fired for airing Brietbarts edited videos?

Who was it that took a proportional typed (produced by Word) document as being produced in 1968? How many people fell on their swords trying to defend these lies? 'Fake but accurate'?

Guard memos have never been proven to have been forgeries. Never. Typewriters possessed by the military at the time were capable of that level of typography. And more importantly, the memos WERE verified - for the accuracy of their content. As in: even if the letters were forged, someone forged the truth.

And then there's the fact that the Rather standards has never ever ever been applied to anyone but Dan Rather - who was fired, along with the producer. How many typography experts does Fox News employ again? How many Fox reporters were fired for airing the Acorn hit video again?

Re:Wait, what?

[ArcherB]

I'm done here. Fox lies. Period. If you don't like my opinion, deal with it.

And that's the problem. You are trying to debate facts with opinion.

You don't care when CBS or NBC or anyone tells blatant lies because they are not FoxNews. However, you accuse FoxNews of lying because someone else said the court decision allows lying on a particular court case. Again, I showed that this was not FoxNews lying, but they were accused of lying because they let an accused company respond to accusations. Of course, those bringing the accusations would call the response lies.

Of course, that's only one case. There may be cases of FoxNews lying, but you have failed to produce any. I'm certain there are cases where a story may have been misreported or details fudged, but that's human error, no lying. Still, with no proof that FoxNews lies when compared to two proven instances where other networks either blatantly lied or intentionally fudged the data to change the story, you still stick by your story that it's FoxNews who deserves the liar label.

The whole point of TFA is bias. You have shown a fine example of it. You hated FoxNews before you heard the story about them lying so you believed the story to justify your hatred. When that story has been proven false, you refuse to believe it because your hatred will no longer be justified. This is what bias causes. You are covering your ears, screaming, "Fox lies. Period." and refusing to accept any evidence to the contrary. It has actually overridden your logic circuits. You have allowed you emotion to trump your common sense. You have valued your opinion over facts. That is bias!

Re:Wait, what?

[ArcherB]

I know, I know, you want to believe Fox is the truth-bearer, picked on by all those mean liberals.

Says the mean liberal picking on FoxNews.

That's rich!

Re:Wait, what?

[HornWumpus]

Thank you for providing an example of dishonest apologetics.

Rather was allowed to retire. He should have been fired. It's pure bullshit to suggest that Airforce secretaries were using typesetting equipment to type memos. Only partisan hacks will claim it they were not caught red handed forging a document.

Finally Brietbart released the unedited videos. They were just as bad as the edited ones. Perhaps worse. Particularly the Acorn videos.

Re:Wait, what?

[marcosdumay]

I suppose YOU don't see a problem with some news organizations taking biased scientific output and unquestioningly running with it as though it were the concrete truth for ever more.

The GP just wrote an entire post complaining about nothing but this, but ok, I guess he may have no problem with that.

Re:Wait, what?

[Uberbah]

This is a story about actual bias in scientists which is affecting the quality of their research.

The hell it is - this "story" is straight up concern trolling. Even if there is some bias in scientific experiences and journals - on what planet does the American public follow them closely enough to turn around and distrust science in general? The American public where the majority can't even name a single Supreme Court justice?

And you completely gloss over that to take issue with Fox, which is no more ore less biased than msnbc, et al...

The hell they aren't. How many stories did NBC or CNN run on Wienergate by identifying Rep. Wiener as a Republican? Dan Rather was fired by CBS, the NBC producer that edited the George Zimmerman tapes was fired - how many reporters at Fox were fired for mixing up footage at Glenn Beck rallies or for showing Brietbart's hit video on Acorn?

Re:Wait, what?

[Anonymous Coward]

Don't bother with ArcherB. He posts a lot of the kind of idiocy you'd expect of someone who was successfully driven mad by the constant stream of disinformation from Fox. I don't tend to pay much attention to who is saying what, but his username stands out as having been above a lot of wingnut regurgitation.

Re:Wait, what?

[Anonymous Coward]

>in my opinion, Fox lies.

FTFY

Re:Wait, what?

[Anonymous Coward]

excellent, salient points. +1 to you sir.

Re:Wait, what?

[Anonymous Coward]

On the Colbert Report last night, he mentioned how Republicans were producing fake candidates in Democratic primaries to force the party to spend money conducting them, and well, here's the dialogue:

"Republicans running as Democrats to force primaries is perfectly legitimate. It's just like when you call a time-out in Football by dressing as the other team's coach"

"But for some reason, that simple Civics Lesson was lost on the real Democrats" ...video here explaining it...

"Oh really, anyone who is fake is automatically a fraud? That's bigotry! I can only imagine what the Revered Martin Bloother King would say about that!"

Re:Wait, what?

[Anonymous Coward]

climate change was a myth because Al Gore. Seriously, that was the reason why it was a myth: Al Gore. Give me a fucking break.

Indeed a bad reason (the climate has always changed, it cannot be a myth). However, I was in the audience at Web 2.0 Summit in 2008 when Al Gore proclaimed that the arctic would be ice free in the summer within five years.

I'll reserve judgement until next year, but that's one bad looking prediction atm.

Re:Wait, what?

[Anonymous Coward]

I was in the audience at Web 2.0 Summit in 2008 when Al Gore proclaimed that the arctic would be ice free in the summer within five years.

Did Gore actually use the word "would" or did he say "could"? Can you cite the actual quote?

Re:Wait, what?

[Anonymous Coward]

I did manage to find a video of his presentation once when in a similar argument so yes, that should be possible. I have a reminder set to make something fun out of it next summer.

(I am of the opinion that Gore fully believed everything he said about the climate back then. Unfortunately, that was mostly the positive bias from the article talking)

Re:Wait, what?

[Savantissimo]

What, you're a physics teacher?! You should know that it is absolutely mathematically invalid to say "x degrees Celsius plus or minus y percent". Celsius is not a ratio scale. You have to use Kelvin to talk about ratios or percentages. That is why it is "3 degrees C" but "276.15 Kelvin". The Celsius scale does not have a proper zero point, Kelvin does.

3C = 276.15K,
276.15K ±5% = [262.34K, 289.96K] = [-9.8575C, 16.8075C], a range of 26.665K or C

4C = 277.15K,
277.15K ±7% = [257.7495K, 296.5505K] = [-15.4005C, 23.4005C], a range of 38.801K or C

The first range is entirely within the second range.

Re:Wait, what?

[Uberbah]

They're facts, deal with them. The memos were never proven to be forgeries. The memos were authenticated for the accuracy of their content. Thanks for providing another example of a right wing hack who sticks to his storyline no matter the facts.

Finally Brietbart released the unedited videos.

Obvious non-response is obvious. And dishonest.

How many Fox reporters were fired for airing the edited video?

Re:Wait, what?

[SteelKidney]

"This is absolutely, unalterably correct and anybody entertaining even reasonable skepticism is an IDIOT"

Nobody has a problem with reasonable skepticism.

I don't think we use the same internet. :) (Sorry about the thread Necromancy. I had totally forgotten I'd said anything.)

Re:Wait, what?

[Anonymous Coward]

The story wasn't lies. They allowed Monsanto to respond to the allegations levied against them. The reporter who wrote the story against Monsanto called Monsanto's response lies and refused to report it.

Wrong. Allowing liars to use your network to lie is not a form of being balanced. It's lying.

FoxNews is doing this whole Post Modernist bullshit where there is not objective truth to a story and a set of verifiable facts which necessarily excludes other "facts" from being possible. Instead, it's just one side against another. Thus their motto "we report, you decide". Reporting known lies as possible truths is not a form of truth-telling, it's a form of lying.

Akre and Wilson sued the Fox station and on August 18, 2000, a Florida jury unanimously decided that Akre was wrongfully fired by Fox Television when she refused to broadcast (in the jury's words) âoea false, distorted or slanted storyâ about the widespread use of BGH in dairy cows. They further maintained that she deserved protection under Florida's whistle blower law. Akre was awarded a $425,000 settlement. Inexplicably, however, the court decided that Steve Wilson, her partner in the case, was ruled not wronged by the same actions taken by FOX. FOX appealed the case, and on February 14, 2003 the Florida Second District Court of Appeals unanimously overturned the settlement awarded to Akre. The Court held that Akreâ(TM)s threat to report the stationâ(TM)s actions to the FCC did not deserve protection under Floridaâ(TM)s whistle blower statute, because Floridaâ(TM)s whistle blower law states that an employer must violate an adopted âoelaw, rule, or regulation." In a stunningly narrow interpretation of FCC rules, the Florida Appeals court claimed that the FCC policy against falsification of the news does not rise to the level of a "law, rule, or regulation," it was simply a "policy." Therefore, it is up to the station whether or not it wants to report honestly.

FoxNews is easily the most corrupt news organization in the world today. Their top executives are under arrest in Britain. Another one of their papers, The News of the World self destructed in the UK from the sheer amount of lawbreaking it was engaged in. http://en.wikipedia.org/wiki/News_of_the_World So the way you're presenting the specifics of the case under discussion is either grotesquely misinformed and pathologically incurious about the actual facts of the case or you're a liar for FoxNews paid to show up on public forums such as this one forums to defend them. I can't pretend to know which is the case.

http://www.huffingtonpost.com/2012/03/13/rebekah-brooks-husband-arrested-phone-hacking-probe_n_1340961.html

data point

[Anonymous Coward]

I received my PhD in physics, and the thesis was measuring a number, in which I measured zero within the error bar. Not particularly interesting, but valid science. My wife was in a PhD program in Biology, she also did valid science, novel measurement technique, came up with an uninteresting result, therefore was not able to publish, therefore was unable to graduate. It would have been extremely simple to fudge the result to a 2-3 sigma result 'hinting' at an interesting answer, which would have gotten published. I think certain sciences have gotten to a point where they have forgotten that if you do valid work in a novel way, then that is science and you should not be punished for the conclusion of the measurement. Most measurements you do of the natural world should probably end up being unsurprising, and thus uninteresting, but you don't graduate or get tenure with those kinds of results. I think this is the mechanism for the positive bias. That is why I do not take results from certain branches of science at face value.

Re:data point

[spiffmastercow]

This. Until we start giving PhDs for finding expected results and/or verifying the results of others, we're going to have this problem of research 'exageration'.

Re:data point

[jholyhead]

It's valid science, but unless it conveys important information to other people, why would other people be interested in reading it in a journal?

No one remembers the 100s of ways Edison found not to make a lightbulb - even though they were each as important as his one successful attempt.

Re:data point

[Anonymous Coward]

Perhaps she could publish in JASNH: http://www.jasnh.com/

Re:data point

[Anonymous Coward]

It's also hard to publish failures, unless they are spectacular. If research doesn't pan out, most shelve it and move on, leaving the next poor bastard to go down the same dead end in the same dead way instead of avoiding it or trying it in a different way.

Re:data point

[Anonymous Coward]

Original commenter here:

It shouldn't be in the highest tier journals, but should be able to be published somewhere so people can find the technique and that someone had done the measurement, with result x. And if the idea and methodology is sound, it shouldn't prevent someone from getting a PhD. But it does. And it filters out a lot of people who would choose to publish results that do not further their own career, as well as filtering in people willing to do things like playing with how they cut their data until the results start looking interesting. (it does not even require faking data, just someone willing to stop looking for ways to cut out 'bad' data when the results start looking interesting.)

Re:data point

[Anonymous Coward]

So, how does a layperson like myself figure out what studies are valid? Is there a BS detector cheat sheet to help us figure out what we can dismiss? What are these "certain" branches?

Re:data point

[AwesomeMcgee]

If it's valid good science it speaks to the competency of the individual who practiced it, if we aren't graduating these folks, then we're encouraging graduation of incompetent or sensationalist scientists.

Re:data point

[Anonymous Coward]

While I agree that negative results could receive more attention, your story is lacking two key elements.

First, was your wife involved in a single project for the entire duration of her PhD? That's tremendously risky, because it means that if the project delivers a negative result, then she has nothing to show for 5+ years of work. It's also strange, because I don't understand how someone could spend their entire PhD doing a single thing unless they had other obligations which didn't leave them time for a PhD (suggesting they shouldn't be doing one in the first place), or procrastinated a lot.

Second, where was your wife's advisor? An advisor's major responsibility is to make sure his/her students don't screw up by choosing ridiculously difficult topics (like a Millenium Prize problem in mathematics), things that show little promise (that other groups have tried and failed, and the student doesn't know because the result wasn't published) or (to a lesser degree) things that have little to no impact.

Re:data point

[jholyhead]

Agreed.

The science should be of a publishable quality, but that doesn't always mean the result is publishable. AC's wife got screwed.

Re:data point

[Hatta]

Which is exactly what we need to do. We need an entire independent arm of the sciences dedicated to confirming established results. This is something that needs to occur separate from the pressure to come up with novel results to please grant reviewers.

Re:data point

[fearofcarpet]

I spent some time in the Chemistry department at a fancy ivy leave school that seemed designed to crush the spirits of biologists. While we were off drinking beer and playing volleyball against the Physics department, they were toiling night and day on multi-year projects that may or may not generate publishable results. The sick thing is that the outcome was totally decoupled from the abilities of the researchers--it was more like a test of stamina and luck. I saw talented, brilliant people turn into bitter husks. Some gave up on research and wound up teaching at private colleges, others left science completely--and not into fields that used any of their expertise. It seems like a lottery where those that get lucky (or stab the most backs) go on to academic positions at top-tens and the rest end up in rocking back and forth in the corner mumbling to themselves all day.

Re:data point

[Anonymous Coward]

Yes, the primary problem was the advisor.

Everybody entering a PhD program thinks they might possibly be secretly a super-genius. The mistake they make is to choose an advisor with a extremely ambitious and interesting project. The most important advise I would give to a PhD candidate first coming in is to choose an advisor with a track record of graduating students, and who is basically nice. The project is secondary. Unfortunately, people just coming in to a grad program don't know how to judge professors and the difficulty of a project until it is too late.

But the fact you think a negative result should still end a PhD program is interesting. It means you think that a person should not try to choose one very complex but novel project, and assuming it is done correctly and was potentially interesting (the project does have to be approved by a committee of professors before starting, after all), should be a valid project regardless of the end result. But that is the prevailing thought in a lot of the sciences right now, one that I think eventually results in positive biases. And is why I give the cynical advice to ignore how interesting the potential project is.

Re:data point

[Anonymous Coward]

"My wife was in a PhD program in Biology, she also did valid science, novel measurement technique, came up with an uninteresting result, therefore was not able to publish, therefore was unable to graduate."

It can't be that simple. I've sat on a PhD thesis committee where everything you describe happened, and the exam went fine and the graduation happened. It was a great thesis on an idea that in the end turned out not to work. Whether the work got published or not also wasn't an issue that determined pass/fail of the thesis. All that mattered was whether it was novel and rigorously implemented science. Publication is certainly encouraged, but not a requirement to get a PhD in the programs I'm familiar with.

I did one publication for my own, but it was only a chapter of a thesis that was otherwise unpublished.

Re:data point

[Xanes]

One option is to read the comment section for literature posted in blogs or forums frequented by active scientists. The discussions more often than not revolve around the quality of the paper and a critique of the techniques used. As a word of warning, it would still be difficult to have a full understanding of what is going on without learning how the science is being done and having a general foundation in the field of study (bio, chem, physics, etc.).

Re:data point

[dzfoo]

So what you are proposing is, to lower the bar for PhD graduation by accepting publication of uninteresting research of common-place phenomena. Gotcha!

That'll go well, I'm sure. It certainly won't be abuse by every would-have-been drop-out to "predict" the boiling point of water, or the acceleration due to gravity, or what happens when you paint a room a different colour (hint: it no longer retains the previous colour, surprise!).

It's OK, you see, because they acquired their measurements in a novel way, with Legos, or macaroni, or something.

      -dZ.

Re:data point

[martas]

Actually there doesn't need to be an independent arm of any kind. All you need to do is get the government to allocate funding for grants dedicated to verification, of course with strict criteria regarding competence of the researcher in experimental protocol and statistics. And publish verification results in university or government run open-access journals.

Re:data point

[Anonymous Coward]

NIST sort of does that. They also do a lot of new science too.

http://www.nist.gov/index.html

Re:data point

[Anonymous Coward]

I think you missed the point. The technique itself was interesting and worthy of publication.

"she also did valid science, novel measurement technique, came up with an uninteresting result"

Re:data point

[Anonymous Coward]

What's up people using "This." as a sentence? A new fad? I've seen it here only in the past year or so. It's ambiguous since it can mean the previous post (agreeing with it) or the current post (disagreeing with or expanding on the previous). Maybe it's just me, but I think it's stupid. Why not "I agree/disagree." or "Yes/no." (even fewer characters to type)?

Re:data point

[Anonymous Coward]

My wife was in a PhD program in Biology, she also did valid science, novel measurement technique, came up with an uninteresting result, therefore was not able to publish, therefore was unable to graduate.

Your wife's advisor was an ass, and/or there's more to the story.

Re:data point

[Anonymous Coward]

THIS.

Re:data point

[Anonymous Coward]

But the fact you think a negative result should still end a PhD program is interesting. It means you think that a person should not try to choose one very complex but novel project, and assuming it is done correctly and was potentially interesting (the project does have to be approved by a committee of professors before starting, after all), should be a valid project regardless of the end result. But that is the prevailing thought in a lot of the sciences right now, one that I think eventually results in positive biases. And is why I give the cynical advice to ignore how interesting the potential project is.

As a recently PhD graduate, I think every PhD candidate should have multiple projects. Some complex, novel and risky, and some safe projects. An advisor's responsibility is to make sure a good student graduates. Which means he should never kill a student's ambition to do difficult things, yet he should be wise enough to make sure the student has backup plans when some of the complex projects fail (and they often will).

Your comment above describes my feelings towards the safe projects. I agree that my strategy helps promote positive biases, but it's a reaction to how reviewers think in my field. After 6 years of research and having read hundreds of papers in my field, I don't recall ever seeing a published negative result.

Re:data point

[drinkypoo]

It seems like a lottery where those that get lucky (or stab the most backs) go on to academic positions at top-tens and the rest end up in rocking back and forth in the corner mumbling to themselves all day.

Yes, just like real life. I used to think "weeder" courses were a bad idea, too. But seriously, science is frustrating. If you can't handle frustration, maybe you should make french fries. You chemists' employers can just buy regulation to force people to buy their products.

Re:data point

[Anonymous Coward]

We need an entire independent arm of the sciences dedicated to confirming established results.

Who's going to pay for that?

Of course it is a problem

[tp1024]

The solution lies in a reformation of research finance that is not focussed on how many papers X published compared to Y, but also takes into account whether they are consequential or not and if they actually comply with at least basic scientific attributes such as repeatibility, verifiablity, falsifiability, accessibility of all data and all conducted research, as well as actually conducted verification of research by independent third parties.

There should also be an outright condemnation of data mining, where data bases are checked only for the existence of attributes and correlations that happen to affirm the researchers opinion and leave all others untouched.

Fields like economics, medicine and climate have long since deteriorated to mere cargo cults due to those failings.

Re:Of course it is a problem

[Anonymous Coward]

Medicine is a cargo cult, really? You're one of those guys that refuses to get his children vaccinated too?

Re:Of course it is a problem

[Anonymous Coward]

Each article and journal has an "impact factor", a metric that quantifies the number of times a source is referenced. This rewards individuals with solid work. Whether or not all institutions weight such things appropriately is another matter.

Re:Of course it is a problem

[rwv]

So would you have scientists publish fewer original research papers and more papers that attempt to reaffirm or disaffirm research that's been published by their peers? I'd be surprised if there isn't *some* resource that links research publications with a list of secondary papers that "Support the Same Conclusion" and "Refute the Conclusion". Given that scientists are looking to publish (churn out) a lot of papers... it seems low-hanging fruit would be studying and trying to repeat the Research/Conclusion of a peer.

On the other hand, I'm an engineer and not a scientist so I take a practical view of applying what's coming out of the scientific community... and the most practical view truly is ignoring 95% of it and letting the "really good stuff" percolate out.

Re:Of course it is a problem

[tp1024]

I'm one of those guys who is fed up with "studies" that "indicate" that X prevents Y just after another study showed that X causes Y. I'm one of those guys who read the news and knows that studies showing that some new pill is usually tested for effectiveness not against accepted treatments but merely against placebos.

Vaccination is well established and has a firm scientific grounding that can be replicated any number of ways, be it epidemiology, in-vivo or in-vitro experiments

Re:Of course it is a problem

[codegen]

Impact factors are also misleading. They consider positive and negative references the exact same. They also don't measure industry impact.

Re:Of course it is a problem

[Anonymous Coward]

Medicine is a cargo cult, really? You're one of those guys that refuses to get his children vaccinated too?

Maybe s/he is just familiar with the increasing body of evidence that MOST published medical research is false.

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/

Or perhaps s/he is familiar with the rife corruption in basic research and financial pressure to confirm certain results, whether true or not, because those results will benefit certain biomedical companies.

Re:Of course it is a problem

[tp1024]

Even if there were such lists, they are not what research funding is based on.

What funding is based on is which journal published your article and how many citations it received in other papers - whether the author citing your paper has actually so much as read (much less made use of) your paper or not. Just being popular with peers helps, because you can always find some excuse or other to cite a paper even if it is semi-relevant at best.

Re:Of course it is a problem

[slimme]

Data mining is indeed a very mediocre scientific activity. Correlation on itself means nothing at all. If you want to proof something the correlation should be 100% and you should be able to explain why the correlation exists and replicate it in controlled experiments. The problem is that those slam dunk scientific discoveries are all or mostly allready found. And nowadays the poor scientists need to find something to bolster their path to glory.

Good science could be: find a correlation an proof the causality. But a lot of studies stop at the correlation. That's what fills newspapers nowadays. 'You get fat from diet coke since most people that drink diet coke are fat'.

Some scientist try to eliminate all other reasons and then decide that their causality is the only one that explains the correlation. But in effect they say: those things correlate and I 'the superintelligent scientist with multiple PhD's' cannot find another explanation and that is why my explanation must be true.

For background you should listen regularly to 'more or less: behind the stats' http://www.bbc.co.uk/programmes/b006qshd
You can listen to podcasts that are interesting and fun to listen to. And some of the older ones are absolutely great. They gave me great insight in the workings of media (and science).

The only downside is that if your girlfriend tells you something she heard on the radio and you answer her: correlation is not causality, she gets upset.

Re:Of course it is a problem

[ceoyoyo]

"also takes into account whether they are consequential or not"

That makes the problem worse. We need to evaluate papers based on whether they are good science or not, and not publish the ones that are bad science. Currently the "negative results" which are actually inconclusive results, are not published because they are, well, inconclusive. Unfortunately a lot of the "positive" results are also inconclusive, but they ARE published. The solution is not to publish more "negative" results, it's to stop publishing the flawed "positive" ones.

Re:Of course it is a problem

[olau]

Good science could be: find a correlation an proof the causality. But a lot of studies stop at the correlation. That's what fills newspapers nowadays. 'You get fat from diet coke since most people that drink diet coke are fat'.

You cannot prove things. You can only show that reasonable alternative explanations appear to be false.

Some scientist try to eliminate all other reasons and then decide that their causality is the only one that explains the correlation. But in effect they say: those things correlate and I 'the superintelligent scientist with multiple PhD's' cannot find another explanation and that is why my explanation must be true.

Now you're being inconsequential. First you don't want people to publish interesting findings before they have an explanation, next you say that if they just come up with their own explanations, they need to shut up.

I think one thing you're forgetting is bad reporting. On the scientific side, someone finds something interesting, publishes it, there's discussion, it is found to be not really that relevant anyway, case is closed. In the public eye, however, the interesting tidbit is announced, misquoted grossly, taken out of context several times, end of story. Nobody reports on the later dismissal by the scientific community. People still think it's true.

Re:Of course it is a problem

[marcosdumay]

The problem is that entire areas of knowledge only ever conclude something by the repetition of inconclusive studies. Those areas include stuff like macroeconomics and psicology, as also stuff like particle physics.

Re:Of course it is a problem

[ceoyoyo]

No, you don't ever conclude anything by repeating inconclusive studies, except maybe that you're doing something wrong. Read the linked paper by Ionnidis. He's over the top and misrepresents some things to make his point, but he's not wrong - doing many inconclusive studies actually INCREASES the chance that your conclusion is wrong. It DECREASES your knowledge. Especially when those inconclusive studies are misrepresented as conclusive.

Economics and psychology have varying standards, but the standards of discovery in particle physics are very high. Particle physicists don't do a lot of repetition of inconclusive studies, and they don't make any serious conclusions based on such things.

Replication is king...

[Life2Short]

Some sort of redirection towards findings that can be verified by independent labs would seem to be an improvement on the current system. But that would require a focus non science as a system and less of the "great researcher" emphasis we see today.

Artifact of specialisation. Part of the problem.

[fleeped]

The more specialised scientists become, the more difficult a proper and thorough peer review is to do. That's the 'innocent' side. And then, you have money and politics..

Trust is the least of the problems

[Hentes]

It also erodes science.

This isn't a new problem

[Anonymous Coward]

I thought positive bias was just one of those things part of the human brain that we just have to deal with. I mean, this isn't some kind of new thing. Look at all the miracles that have been attributed to God over the course of human history. I doubt that old research was any better. Cognitive science has just made us more aware of positive bias.

Sure it does !

[Anonymous Coward]

If what is reported by scientist is wrong, regardless of whether it is too optimistic or too pessimistic, then the public has to be losing faith. IMOO, what contributed the most to that trend is the growing pressure put on scientist to obtain results. Positive ones. Not in 1 or 10 years, but now, otherwise you might be looking for another job pretty quickly.
What is the more puzzling to me being that the said pressure has been put on the scientists mainly by the public itself, so well, I guess you reap what you sow.

Positive bias is the wrong term.

[Karmashock]

They're overstating precision. Which rather then a forgivable error is an elementary mistake no trained scientist should ever make.

A VERY basic concept they teach at the lowest level of science education is the distinction between accuracy and precision. This is science 101.

Accuracy is whether or not a given conclusion is correct.
Precision is to the degree of specificity.

Typically you run into problems on complex subjects because they overstate the precision of their data or their ability analyze the data.

This can boil down to simple thinks like significant digits.

For example, I'm measuring volume to two significant digits in a giant data set with thousands of measurements. When and if I average those numbers the final average can't have more then two significant digits. That sounds elementary but you see this error made on some big studies. You'll have a situation where something is being measured in a crude sense by many sources and then in the analysis a much higher degree of specificity is implied.

Often that degree of specificity is required to make certain conclusions which is why they break the rule. This is lazy and a breach of scientific ethics. What they need to do is collect the data all over again this time to the level of specificity they need.

Simply saying its too hard to collect the data properly so they're going to make assumptions is not reasonable or ethical. I suppose you could do it so long as you kept an asterisk next to the data and the findings to make it very clear throughout that the conclusion is a guess and not in any way empirical science since at some point people were guesstimating results.

Re:Positive bias is the wrong term.

[Anonymous Coward]

No, they're blatantly lying so that they can get publishable results. Sorry, seen it here. I really wish we had a peer reviewed journal of uninteresting results to prevent repetition of negative results and so that folks who do good science and find something uninteresting can, say, get thier PhD. We've been lying for decades now, and that's why industry is doing most of the interesting applied science now.

Re:Positive bias is the wrong term.

[medcalf]

That's one of the problems I have with global warming. The precision of the best made, best sites thermometers is about a degree. One common class is +/- 5 degrees. Yet these are being used to derive results in tenths of a degree. Satellite proxy measures help, but that's a very short record in climate-significant time frames. Is the Earth warming? Almost certainly. Is that outside the norm? Hell, we don't even know if it's outside the margin of error.

Re:Positive bias is the wrong term.

[ceoyoyo]

"When and if I average those numbers the final average can't have more then two significant digits."

Yes, the average can have more precision than the individual measurements. That's actually kind of the point of an average. It can't improve accuracy though, for the most common definitions of accuracy.

Your definitions of accuracy and precision are sort of right, but also sort of misleading. And the way you use specificity is incorrect. But as you correctly point out, a lot of working scientists are a bit fuzzy on all these concepts too.

Re:Positive bias is the wrong term.

[Karmashock]

Well, it's funny when they do that with proxy data. They use samples of biomass in sediment from 10,000 years ago and pretend to derive a .01 temperature difference. You can't get that from pine cones, ice cores, and fossilized vegetation.

The real problem with all this stuff is that the data was never collected for this purpose. The standards and precision were for crop forecasts and whether or not people should go to the beach. It was never collected to be compared to a tenth of a degree across continents and throughout centuries.

Even the satellite data has problems because it's calibrated. The sats don't know what temperature it is on earth without calibration. Basically, you ask the sat what temp it is in a place where you know the temp and from that you infer what all the other temps are in it's readings.

Where things get squirrelly is that you can have two way calibration. That is data is used to calibrate sat data and then that sat data is used to filter the ground stations. So rather then everything being based on a wide data set everything suddenly becomes based on the calibration data.

And what's troubling is that the calibration data for the sats is not fixed. It changes. If you use the old calibration data for new readings most of the sat data reports cooler temperatures. Its only after calibration that it reports higher figures.

Possibly this is all legitimate. But it would be very easy to bias the calibration data slightly to show a warming trend.

Ultimately, the solution is to collect a lot more data, make that data publicly available with no filtration, and then hand that off to another group of scientists to analyze it.

There is too much theory crafting and too little data collection.

Re:Positive bias is the wrong term.

[Karmashock]

I'd rather not get drawn into a stupid semantics debate.

Can we agree to this:

"Junk in; Junk out."

I think what we're seeing in many fields is that the data is poor or doesn't exist. So scientists have convinced themselves that they can use magical computers and mathematical models to turn fuzzy imprecise data into something crisp.

It's like taking a picture of the Mona Lisa with a 3 MP camera from 50 feet away and believing that using a mathematical model you can see every crack in the paint. You just can't.

If you want that level of detail you need to have better data. Or purge any greater specificity then that from your conclusions. During analysis, I believe it's allowed to keep raw averages. But when you're stating final conclusions you are supposed to limit your results to significant digits. Which means the conclusions can't claim to be more precise then the input data.

Re:Positive bias is the wrong term.

[ceoyoyo]

I'm sorry, from your first post I thought you were an interested layman who had a few common misconceptions. From your second post it seems you're more of the anti-intellectual and doesn't want to hear anything contrary bent.

To be clear, your first post was flat out erroneous. Not semantically challenged, or debatable. Wrong. If you're interested I can explain to you why it is wrong, or you can continue thinking your smarter than those idiot scientists who average two numbers together and end up with a more precise one.

And please stop using the word specificity. You have no idea what it means.

Re:Positive bias is the wrong term.

[geekoid]

I could explain what there isn't actually a problem with the temperature reading, but I"m not sure you would understand it, so I'll say this:

It doesn't matter because it's + or - and we see a constant rising, If the error bars were a problem and we didn't have global warming we would see the normal rise and fall + or - a degree.

" Is that outside the norm? Hell, we don't even know if it's outside the margin of error."
That's just wrong. Like every denier, you are 40 yeas behind in the science.

Re:Positive bias is the wrong term.

[Karmashock]

I think a good way to deal with would be similar to the separation of powers in the US federal government.

hear me out.

What if one group of scientists only collected data. Nothing else. No theories and they didn't store the data or interact with the scientists that want to support theories.

The next group are librarians and information specialists. They collect all known data and sort it. They also have all studies on file. They are the keepers of the knowledge. But they do not collect the data and they do not theorize on it.

The last group neither collects data nor stores it. These are the theorists. They collect data from the librarians and use that data to form their theories. The data held by the librarians is open to all. So rather then having to state your data in a study, you could simply offer a reference number to the data stored in the system.

If the librarians don't have the data then theorists request the data from the librarians and they then pass that request publicly to the data collectors.

This all sounds complicated but what it does is make everything very transparent. The people collecting data have no incentive to bias their data. They have no vested interest in any conclusion. Rather, their bias is in getting the most precise and accurate results possible. They might over state their precision but they are less likely to bias their results.

The librarians are very key to the whole system because they record all requests for information from the theorists, all the data from the data collectors, all studies published by the theorists, and all opinions written about those studies. Again, everything categorized and filtered. So lets say a given paper has 10,000 different opinions written about it. Well, some of those might not be written by scientists. Some of them might be written by scientists with no experience in the field. And then you need to break it down by year. Maybe a given opinion was written right after the paper was published and another one was written ten years later. Then break it down by location... etc. It's not for the librarians to say what is an isn't relevant. They should collect all information they're given, categorize it, and make it available.

The above concept would make scientific disclosure redundant. The theorist wouldn't need to disclose anything. All the data would be open to everyone from the exact same place the theorist got it.

Re:Positive bias is the wrong term.

[Karmashock]

I'm not anti intellectual.

I'm just not a fan of intellectual masturbation. Keep it useful and keep it relevant. That's all I'm asking.

As to not wanting to hear anything contrary, I love being proven wrong. I simply require it be proven. I don't like people saying I'm wrong and not proving it. How would you like it if I simply contradicted you and provided no basis for that contradiction? You don't like it either? Don't do it.

As to "specifically," define it immediately. This comment is especially annoying. If I've said something that is wrong and you're claiming to in anyway represent some higher order of learning then you have to justify that pretense.

Re:Positive bias is the wrong term.

[Anonymous Coward]

... and you wonder why Fox News has a heyday with the ivory tower types? He had an argument that makes sense for anyone who was sober at statistics class. Your response was "you're a stupid, vilified minority".

Re:Positive bias is the wrong term.

[renoX]

> They're overstating precision.

Replace "they" by "nearly everybody": who is using interval computation instead of floating point numbers?

In first approximation, nobody!

Re:Positive bias is the wrong term.

[argStyopa]

Not just science, I see this in business all the time.

People with "serious" business school degrees do something like this:

Wonk1: ok that's...ah...we'll estimate 100,000/year.
me: so we'll say about 9000/mo or about 2000/week in the report.
Wonk1: no, that's 8333.3 per month or only 1923.1 per week. We'll use those figures because they're more accurate than your rounded number.
me: but you pulled the annual number out of your ass as a rounded number?
Wonk1: right, but that's annual. We need the precise data for the monthly/weekly totals. /facepalm.

Re:Positive bias is the wrong term.

[Karmashock]

Much of the market crisis came about because they had boiled down "risk" which is an extremely complicated subject using mathematical models down to one number.

Basically, the higher the number was the more risky the investment was according to the models.

The problem is that you can't boil risk down to one number. There are different types of risk and it's impossible to combine them.

The model made all sorts of hilarious assumptions like "the market will never crash"... Never mind that it has in the past and therefore can and therefore you can't make that assumption.

So yeah, you see it in finance.

Re:Positive bias is the wrong term.

[Uberbah]

That's one of the problems I have with global warming.

We know. It's obvious that you are very, very, concerned.

Re:Positive bias is the wrong term.

[Anonymous Coward]

A/C. I write scientific websites. Don' ask.

People talk about positive bias, trust. Two scientists talk about public opinion and fuzzyness -- and they can't even agree on how to do basic addition and division to compute an average. Or whether it should be done.

I understand this is part of tailoring your method and presentation to the test.

But how do you expect the public to trust you when you can't even agree to average 1.5 to 1.5 and whether you get 2 or 1.5 ?

And don't even get me started on how in anything interesting these days, research has lots of data processing...

And believe me, I'm a programmer, not a scientist. And your scientists...they're mostly awful programmers. And as a result--most of their science is awful science that isn't reproducible save a few that publish it everything. Which they don't. Ever. Not in journals. Maybe on a personal website. Maybe.

In the journals, they always hold something back because they're always working on the next grant or the next variation. By the time they're drafting the paper people are gathering data for the next experiment.

    I've seen libraries, methods, functions etc... identified by the wrong name. I've seen mode labeled average and mean by people who called the wrong numeric library routine. I've seen "social scientists" do population simulations with random number generators that they reseed on every iteration... but the result is distilled into a chart or graph and published, and the source sits on their hard drive until their next desktop upgrade when they lose it. I've seen one asshole generate climate data (not for AGW, but for a local micro study) using cheap thermistors where they stream continuous data to a logger -- but never accounted for self-heating, power fluctuations (cheap hardware), sensor stabilization... or the fact that they were reading voltages -- not temperature.

And their 'averaging' routine had a serious bug in it that would have resulted in a time-bias. But they 'rectified' all of that to get an abstract temperature series at a particular time interval and called *that* the data. Not the original material they processed wrongly. That they probably failed to properly compute the sigfigs in to start with and I don't even want to tell them that.

No. Scientists and their conclusions should not be trusted. Not until every single data source, private or not, every single point of processing, every correction and adjustment is up on the web with revision history and changelogs.

None of the people I mentioned set out to commit scientific fraud. But they managed to do it by accident anyway.

Re:Positive bias is the wrong term.

[Anonymous Coward]

I'm measuring volume to two significant digits in a giant data set with thousands of measurements. When and if I average those numbers the final average can't have more then two significant digits.

That's not even a little bit true. Imagine you have 10,000 identical pencils fresh from the mill and you want to know what a single pencil weighs down to tenths of an ounce, but your scale is only precise to +- 1 ounce. Put those 10,000 pencils on the scale. Divide the result by 10,000. Bam! Now you know the weight of one pencil to several significant digits.

Re:Positive bias is the wrong term.

[MightyMartian]

You didn't mention anybody. You asserted some special knowledge because you write a "science website", and then went on to make unevidenced accusations. You're precisely the problem, the layman who thinks he's the smartest boy on the block who makes grandiose claims about a huge level of incompetence-cum-fraud.

The worst thing to ever happen to science was the science journalist.

Re:Positive bias is the wrong term.

[MightyMartian]

So you want to make science even slower and less likely to produce results by turning it into a bureaucratic nightmare? It's a ludicrous idea that indicates you don't have the foggiest idea how science works. I mean, a first tier of data gatherers... What are you going to do, give them lobotomies to protect them from making any kind of intuitive leap that a series of samples indicate some new property? How would you fit this to a whole host of sciences, like archaeology, where expertise in the field in general is absolutely essential to extraction, preservation and research?

It's an idiotic idea. Science can be a messy business, but at the end of the day the theory stands or falls on the evidence. That's the final objective, however you manage to get there. Evolution wasn't formulated by one guy drawing sparrows, another guying cataloging the drawings and a third guy browsing through the library. It was all done by one guy.

I have no idea what problem you think you would solve. It's not as if the vast majority of people could interpret raw data anyways, and any researcher who won't release methods and/or raw data is pretty much immediately suspect from the get-go.

I guess if your purpose it make science stop dead in the water, it's a great idea. That way any theory of an inconvenient nature might take a hundred years to come to fruition, if at all.

Re:Positive bias is the wrong term.

[Anonymous Coward]

Well, technically you know the *average* weight of one pencil to several significant digits. (If you really want a good result, you'll cross-check that result by using a different of scale as well. Preferably one which is not of the same type.)

Re:Positive bias is the wrong term.

[Karmashock]

As to turning science into a beurocratic nightmare, that is not my intent. If that is the result then my idea is poor. However, I would ask for constructive criticism on the point rather then flat rejection. That way I know what the problem is and I know whether your complaint is valid.

As to the datagathers, they don't even need to be scientists. The level of training would be enough to collect the data with a high degree of accuracy and precision. The theories it might well relate to would be irrelevant to this profession. As to forbidding them from making theories, I wouldn't forbid it. I would instead require that they disclose the data as they collect it and if they want to download that data later and theorize on their own time they are welcome to do it.

But they can't collect data, not release it, study it in secret, and then propose theories based on data no one has ever seen before.

What is important is that the data all gets pushed to the librarians and that all theory is based on data in the librarians database. So if you want to collect data and then theorize on it, the right way under this system is to collect it, submit the data raw to the system immediately, and then reference that data in your theories. Withholding data for any reason what so ever would be considered an ethical violation under this system.

We could radically increase the amount of data we collect. Might the quality go down? Possibly. The data would also note who collected, who that person is, when they collected it, any comments other people have made upon the data EVER, who those people are, and anything else you can possibly think of remembering for any reason. The librarians would be remembering everything under this system and the context and the comments.

As to archaeology and other fields where a certain amount of specialized education is required to gather data, what is really important is that they submit all the data as they collect it in the rawest possible form. If they want to theorize upon it at a later, it doesn't really matter so long as the raw data was recorded first. And of course, it would be expected that any theories offered would be based upon something in the database even if they submitted it to the database themselves. I'm trying to avoid situations where people propose theories based upon data that isn't publicly available or even available for peer review in a raw form. I am also trying to remove the incentive to withhold data until it can be analyzed. Analysis is optional. Submit it immediately.

That said, just as aceologists use students to do most of the work, I think it is possible in most fields to use less educated and trained personnel to collect the data. By all means, if you want to do it yourself, have fun. But this would hopefully encourage scientific organizations to use laymen with enough training to collect good data but not much else. There are a lot of things that aren't practical to study because there aren't enough people to do it.

I believe the British natural history musem had a policy where they encouraged people throughout the empire to send plant and animal samples back to England for study. An enormous number of new species were discovered in this way. People would pull some leaves off an unusual tree, press them between dry paper, and mail them to the museum. They have thousands of books of such leaves to this day categorized by location, time, etc.

Don't discount the value of laymen in collecting data. I wouldn't forbid theorists from collecting the data. I'd prefer if they didn't. But the real worry on my part is hording information and then throwing out theories based largely upon private data sets.

As to science standing or falling on evidence, that's if the evidence is disclosed. Often only filtered evidence or data is disclosed and and the raw unfiltered data and the exact methodology used to filter it is both not released and more importantly can't be verified because you can't compare the raw and filtered data.

As to

Re:Positive bias is the wrong term.

[Karmashock]

Is that increase before or after the calibration. Is it before or after the filters?

I guess what is troubling here is that there is so much data and the quality of that data varies radically. Then all this data of variable quality is dumped into a single data set and a level of specificity that goes down to a tenth of a degree is inferred. The instruments with historical data were not the accurate even if you assumed the equipment were the application would introduce a large margin or error. And this becomes geometrically more problematic when you introduce proxy data. You can't infer a tenth of a degree temperature change 10,000 years ago from lake sediment. I'm sure you can get within ten degrees but within a tenth of a degree? I find that extremely hard to swallow.

And once you lose that precision, the fuzzy zone above and below your numbers widens out significantly and it becomes very hard to claim an increase when most of your data didn't move outside the margin of error.

We're talking about 1 degree MAX change. So any move from then to now will naturally be smaller then that. we're talking about tracking a fraction of a degree change over time using data that really doesn't have that many significant digits.

Please don't call me a denier or some other derogatory comment. It isn't helpful to an intellectual discussion on the matter.

I am skeptical. I find many aspects of the situation suspecious. And this is not helped by repeated instances of sloppy behavior. The Himalaya claim for example in the IPCC was based upon a misquoted phone interview of a scientist in a climbing magazine. Literally. That was the whole basis. And who found that out and got the IPCC to edit it? Was it peer review? Nope. It was skeptics that found it.

Now, you can say whatever you want about the education or intelligence of skeptics or deniers or whatever. But if they occasionally spot something that their intellectual betters don't and its a patiently absurd as basing a huge claim on nothing... then what does it say of the those intellectual betters? To a certain extent, arguing that skeptics are stupid doesn't work in your favor because they often spot things the warmists or alarmists don't spot. You'd do better to claim a smaller gap in the superiority if any at all if only because it makes mistakes less embarrassing.

Re:Positive bias is the wrong term.

[Karmashock]

You're assuming you can say the pencils are identical as a given. That there are givens makes your example difficult for me to relate to the discussion.

Could you rephrase your argument without assumptions like something being identical. The reality is that you'd have to weight the pencils before that to know if they were identical.

What are the odds that if I took two collections of 10,000 pencils each and weighed them on the same scale that there wouldn't be a some kind of difference?

We're talking real world here. Little things like that crop up all the time and it either doesn't matter or people find ways around it. But starting with a given of perfect precision identical copies is not realistic and whether or not that is relevant to your point it kills the argument. Please rephrase without those sorts of givens.

Re:Positive bias is the wrong term.

[Karmashock]

Is it really such a great sin for the public to say "show me your work"...

If you take a math exam and simply write out the conclusions you'll get an F even if the answers are all right. They want to see how you got there.

I think it's very reasonable to expect full disclosure. I mean everything. All the data being used in it's raw unfiltered form with full disclosure as to equipment and context. Then methodically walk us through the process of how that was turned into your final conclusions.

Short of that, you're asking me to trust you. I want to trust you.

I really do.

But... I can't. It's nothing personal. I have to know.

Re:Positive bias is the wrong term.

[ceoyoyo]

Specificity, in science, is the number of true negative results over the total number of negative results. More generally, it's how specific something is, i.e. how well it excludes false alternatives or how well something rules out an effect. You seem to be using the word to mean precision, or accuracy, or some combination of the two. Your statements actually make more sense switching "sensitivity" for "specificity." Sensitivity is the ability to detect something that is present.

Your claim: averaging two or more values cannot provide you with an improvement in precision, therefore you should never report average values with more significant digits than your individual measurements. This is fairly easy to prove incorrect mathematically. Basic statistics classes cover the topic, and generally cover the derivation of the square root law: the mean of N independent measurements has a precision that is sqrt(N) times greater than the precision of the individual measurements. Sometimes it's stated as the noise is decreased by a factor of sqrt(N).

You seem to have taken someone's (your high school chem teacher maybe?) insistence on using significant digits in calculations and extrapolated it WAY too far. Significant digits are a quick and dirty type of error propagation calculation, and point out the fact that you can't increase information by calculation. You can reveal existing information, or display it so it is easier to see, but you can't create it. Significant digits is a poor method though, because it does not take into account magnification of errors when you do things like multiply, which proper error propagation analysis does, and it assumes that the precision of the measurement is equal to the number of quoted significant digits, which it rarely, if ever, is. Almost any non-trivial scientific measurement is made multiple times and a proper standard error is calculated. And the standard error has a denominator that is... sqrt(N).

Averaging is NOT a simple calculation - each measurement you're averaging brings new information to the table, and so averaging them together increases precision. That's the point of averaging.

As for your example, it's kind of unfortunate as well. You CAN increase resolution by taking multiple images that are related in specific ways and performing calculations to combine them into a single, higher resolution image. It's called super resolution, and is possibly most commonly used in light microscopy. You can also increase the resolution and other characteristics of specially sampled images by adding information about their sparsity. This is called compressed sensing.

You started this thread by criticizing scientists for doing something correctly, when you apparently don't even know the basics of the field. Then you brushed off the gentle suggestion that you might be wrong as "a semantic argument." May I suggest that the actual issues are a little more involved than the application of a rule of thumb taught to sixteen year olds?

Re:Positive bias is the wrong term.

[Karmashock]

On a great many points you seem to be right and I'm not so vain or petty that I won't acknowledge errors on my part or spite someone for their errors especially if they've shown an interest in a real discussion. To do so would be childish and I'm not.

As to my language, I would ask that you be somewhat forgiving. I'm mostly speaking english here. If I use a term, it should be assumed unless otherwise stated that it is english. "Specifically" for example was the english definition and not whatever specialized reinterpretation you were using. It would be unfair for example for me to use legal definitions of your term which while spelled the same way have distinct meaning from standard english and your scientific offshoot language.

As to increasing resolution, I do remember an astronomer talking about increasing the resolution of telescope images by combining many different images into one composite.

The issue is interesting. Exactly how would you do this for example with largely low quality and inconclusively captured climate data? I mean, a weather station in Arizona that had been operating since 1920 for the purposes of local weather prediction is unlikely to be of the same make as another no farther away then new mexico that was built in the 1970s. The differences in equipment, method, maintenance, diligence of staff, location of the station, whether or not the station was moved, there are problems with just assuming that data is any good. There are thousands of such stations throughout the US and most of the records were kept in pen and paper log books.

And while there are "reasonable" (That is the legal definition of reasonable and not the english definition) ways to try and tease out good data from bad, I'm not sure if you can actually do that scientifically and claim the end result is empirical evidence of anything with any great precision (I'm using my understanding of the scientific definition).

What really hurts this situation as well is that it's very hard to get the raw unfiltered data to compare against the filtered data. They keep loosely describing their methodology (my understanding of the scientific term) but you can't falsify (my understanding of the scientific term) their claims because you can't compare the raw with the filtered data.

It's not only laymen that have had a problem with this. Mathematicians and statisticians that are expert in their fields of mathematics and statistics have found major problems with the methods and calculations made by these people.

To then brow beat any laymen that has noticed this controversy is beneath anyone presuming to speak for science or reason.

Please have a civil discussion with me. There is no need to be rude. I am arguing in good faith and I am not stupid. If a person arguing in good faith that is not mentally impaired finds the issue controversial then regardless of your opinion the issue is controversial if only because the situation has become confused.

Re:Positive bias is the wrong term.

[ceoyoyo]

What you're describing in terms of climate data is data that is of potentially low precision, poor accuracy and may be biased. Using data from a large number of independent such stations improves precision through averaging and can also improve accuracy and decrease all but systematic bias. You can also try to identify sources of bias specifically. For example, urban weather stations report systematically higher temperatures. We know this because we can compare urban data with nearby rural data, and as a result we can account for the bias.

If you know anything about science, or climatology in particular, you would know that data is never "assumed to be good." By suggesting using significant digits you were actually doing this - you were assuming the data was as precise as the stated number of digits. In real science you almost always MEASURE the precision of the data by using multiple measurements, or multiple datasets, and calculating the standard deviation or standard error. That way you know how precise your final values are, and can use statistical tests to quantify the probabilities of various conclusions. Problems with accuracy and bias are more difficult to deal with, but they can also be quantified and improved, usually by comparing with data from an independent source, preferably using a different measurement method. In climate research you can use different weather stations using different instruments, or compare data from tree rings with ice cores, with weather stations. In particle physics, the LHC has two detectors, each designed, built and operated by different, independent teams, using different methods. There are a number of metrics that quantify accuracy and bias. Inter-operator agreement and the ratio of intra- to inter-class variance among them. Provided you have multiple measurements you certainly can take data of low precision and produce measurements of higher precision, and make reliable conclusions based on those measurements.

You seem to have particular issues with climate science and are attacking science in general on that basis. Even specifically in climatology, your criticisms seem to be overly general. There have been some quite reasonable suggestions that climatology is now too important to allow raw data to be proprietary, and the field has moved to address that, although most of the data seems to have been available anyway. As for criticism of methods, you will always find someone who will criticize your methods. The proper thing to do then is for someone (you, them, or someone else) to replicate the experiment or analysis with other methods and confirm or refute your conclusion. That's also been done in climate, and resulted in confirmation.

Yes, brow beating is wrong. In climate, the people who do that are usually not scientists but are often people with some political agenda (for OR against). Sometimes a scientist might brush off criticism like yours that claims the scientist (or all scientists) are obviously wrong for doing something, when it's clear that you have very little understanding of the issue. If you then dismiss the civil counter argument as "semantics" and stick to your erroneous assertion, you're sure to be either ignored or ridiculed.

I'm happy to have a civil conversation with you now that you're being civil. You don't seem to be stupid, but you do appear to have very little knowledge of some of the things you're talking about. There's nothing wrong with that, but beware of making bold assertions, particularly critical ones, about things you don't know much about.

Re:Positive bias is the wrong term.

[Karmashock]

So you're telling me that you believe or it is proven or effective practice to get a tenth of a degree of precision in temperature by analyzing proxy data like pine cones, ice cores, or sediment?

Perhaps I just don't know what I'm talking about but that seems a stretch given that the margin of error on any one data source is huge.

It's not that I don't think they can get a general idea of temperature but I don't think they can get it that precise simply by taking hundreds or thousands of samples and averaging. At some level you're assuming how accurate every result is or you're assuming that if all the results are fairly close together that the top of the bell curve is correct.

Lets say you have 10,000 color blind eyes... there's an inherent lack of accuracy that isn't going to be fixed through averaging.

I also am familiar with what happened to the Earth Simulator in Japan. I'm not sure if you've heard this story but the Japanese donated a super computer to crunch climate data.

When they fed the models into the machine, it kept creating extreme feedback loops. The oceans boiled in some of the iterations. They were only able to stabilize the system by using plug variables. That gives me ZERO confidence in the models. Because lets face it, a child could come up with models that looks reasonable if they used plug variables.

I worry about not just climate science but any models based science that has sketchy data. I worry that everyone is playing fantasy games in the computer and making mystical dragons and unicorns to fit their pet theories. Maybe I'm just out of touch. But science didn't get where it is now by saying "trust me"... It got there by saying "let me show you"... Cartoons are not going to cut it. Forget trying to explain it to the majority of people. The majority don't care. If you can make this understood to the top 10 percent then we might be getting somewhere. As it stands, especially with AGW that is not the case. The whole issue broke on ideological grounds and there are as many smart people on both sides of that divide.

Short of that, both sides are going to leverage their considerable political power to pull the parking break and then nothing goes anywhere.

Re:Positive bias is the wrong term.

[ceoyoyo]

I gave you the formula, which is not only empirically demonstrated, but mathematically proven as well. Suppose each of your measurements has a standard deviation of ten degrees, which is a precision you could probably achieve just by going outside and feeling how cold it is. To get the 1/10th of a degree precision you mention, you need N = (10 / 0.1)^2 = 10,000 such measurements. If your measurements have a standard deviation of one degree, which a grade schooler should be able to achieve with a household thermometer, you need only 100 measurements.

Yes, you don't know what you're talking about. You're drawing conclusions based on uninformed instinct and a pretty big bias. You're also conflating climate prediction through modelling, which is by its nature quite speculative, with historical climate records, which are much less so.

Re:Positive bias is the wrong term.

[Karmashock]

Again, there seems to be an implication that an average of bad information can be used to determine very accurate and precise information if there is enough of it. That seems specious.

And as to the modeling versus historical records, since scientists mix the two seamlessly when they present their argument to laymen exactly who's fault is it when laymen question both together.

I'm sure I'm making mistakes here but you need to hold the scientists accountable as well because the whole presentation has been badly botched.

Re:Positive bias is the wrong term.

[ceoyoyo]

"Again, there seems to be an implication that an average of bad information can be used to determine very accurate and precise information if there is enough of it. That seems specious."

To me it seems specious that your feelings are a better guide to reality than empirical evidence and mathematical proof.

Re:Positive bias is the wrong term.

[Karmashock]

Well, since you're just going to be rude, I suppose this discussion is over.

Good day, sir.

No secret in academia

[crazyjj]

I used to be in academia, and it was no secret that researchers almost always found the results that they had planned to find from the beginning in their studies. I don't think I ever once saw a case where a researcher started out with a hypothesis, found it was completely wrong through the research, and then let it go. There are a million ways to cook the numbers to reach the conclusion that you want to (and the one that gets you the grant money and publication). If a certain position is popular (and well funded), expect everyone to jump on it and start producing paper after paper confirming it.

Re:No secret in academia

[hey_popey]

This seems quite complicated! The right way to do stuff is to first do it, and write your hypothesis afterwards according to your results. The same goes for engineering: you make the product, and then write the specification. Otherwise you would be wrong / request for deviation too often, and that would be depressing!

Re:No secret in academia

[Anonymous Coward]

I agree. We have known about this for as long as I can remember. If something is published in a journal, that doesn't mean it's real, or neutral, or whatever. It's just another thing for the scientific community to look at.
Yet the papers take every paper published and run with it. "Apples cure cancer!" "Worm hormone will give us eternal youth!" "Meat eaters are jerks!" "Your subconscious is prescient!" "Huka Puka gletscher grew; global warming fake!"
We've been telling people for ages not to believe everything they read, but to little effect. We've been shouting off the rooftops that if you're a journalist wanting to write an article on something, you can call us to inform yourself, but generally they don't bother. In the end, people will believe whatever is thrown at them in the biggest quantity, and in re science that means the science section of the Saturday paper, which to those in the field looks more like some kind of bitter parody of science.

Apophenia

[concealment]

When you have a hammer, everything looks like a nail.

Anything can become a religion, as a result. We're less critical of our data when that happens, and we "nudge" it into place.

The problem is not "science" per se but our social approach to it.

Re:Apophenia

[geekoid]

The large percentage of bias isn't nudged. It isn't even conscious.
It's a long time know property of research, and it's why several studies should be done.

You're hammed analogy is nonsensical in this context.

Wrong Angle

[wisnoskij]

I would say that the bias is not so much positive as it is "get noticed and get more grant money" biased.
Unfortunately, you get noticed more and get more grant money if you find what you were looking for instead of disproving your initial hypothesis.

Also this article seems to imply that the problem that needs to be fixed is that of public trust, while I would argue that the public should distrust a community that gets it wrong so often (it is the skeptical, scientific thing to do). Unfortunately, far too many people have the opinion that science is absolute truth and always right; but science is not an "exact science" so to speak, scientists overlook variables all the time and prove what they want to prove just as readily as everyone else.

I left academia and science because of this.

[Anonymous Coward]

Academia is a Ponzi scheme built on fudged statistics, unreported failures and outright forged results.

Unfortunately, Publish or Perish and the overadmission of PhD candidates has resulted in a system where getting a paper out is more important than what goes into the paper.

Science is about money now

[hshana]

Like just about everything else in this world, science is about money. And how do you get money in science? By finding and/or hyping the next leap forward. Being successful in science is all about getting grants. You don't get tenure without bringing in grant money, you don't get grant money without publishing in the best journals, you don't publish in the best journals without finding the next leap. Your typical PhD finishes school in their late 20's, probably with significant school loan debt. He or she then gets a postdoc where they can barely afford to live in the city with the prestigious school that they think they need to further their career. At the same, it's probably time to think about starting a family (especially if you are a woman). And as a postdoc, the pressure to publish is even greater in a more compressed time frame. There is so much pressure there financially, emotionally and mentally, that is it is no wonder that some people cave and take shortcuts and fudge results. And then, if you do make it to a tenure track position, you don't do much science any more. Instead you spend all your time writing grants and churning through postdocs, who may or may not be fudging their results to get a recommendation to get a better position...

Positive bias in engineering research

[l00sr]

In engineering research, there is definitely a positive bias; in fact, negative results are rarely published at all. This is both because negative results have less sex appeal than positive results and because peer reviewers are trained to outright reject publications without positive results. Although there is huge pressure to publish positive results, I'm not aware of systemic fraud in the literature. What does happen, however, is roughly this: 1) researcher gets great idea. 2) researcher tries idea. 3) idea fails to produce state-of-the-art results. 4) researcher adds hacks and kludges to marginally improve performance. 5) repeat steps 2-5. So, what you get in the end are journals filled with "positive results" that mean nothing and a bunch of "scientists" who make a living doing things that do not really resemble science at all.

Re:Positive bias in engineering research

[GJSchaller]

In engineering research, there is definitely a positive bias; in fact, negative results are rarely published at all. This is both because negative results have less sex appeal than positive results and because peer reviewers are trained to outright reject publications without positive results

This. "People" don't want to hear things perceived as "negative," "failures," or "I don't know," even when it's accurate. The common person thinks science has all of the answers, and that if a scientist doesn't know the answer, it's the fault of the scientist, not science as a whole. Obviously, if problem X can't be solved, it's because the scientist working on it is lazy / stupid / biased, and not that society as a whole doesn't have the answers yet.

I can link this to two common phenomenon in current society: 1) TV / Movie science, in which the fictional scientist does something risky and cool, and solves the problem of the week using "SCIENCE!". See also, the CSI effect in court rooms. 2) Corporate culture, where people don't want to hear "I don't know" for an answer. Most corporate people are conditioned to have an answer, ANY answer, rather than say "I don't know yet." (Pet peeve of mine - being asked to solve a problem I haven't been informed of yet, and / or then being asked why I don't know the answer when I don't know the parameters of the issue / question.)

Until the concept of "It's OK to admit you don't know everything, and learning stems from this" is common place, there will be positive bias in published results, and other facets of society, even when the results are faulty.

Re:Positive bias in engineering research

[ceoyoyo]

I bet what you think are negative results are actually inconclusive results. LOTS of people, including many of the ones writing papers about positive publication bias, make that mistake. An insignificant p-value is NOT a negative result. It's an inconclusive one. In order to actually show a negative result you have to do more work, and delve into the (usually very simple) stats that few people know. Most studies don't have the power to show actual negative results, but in my experience if you actually do the other work you get published. It's almost a given because the reviewers never see that sort of thoroughness.

Re:Positive bias in engineering research

[Anonymous Coward]

...or papers presenting yet-more-endless iterations of ideas. There are N ways to do something, there are M ways to do something else... hey, I got a *novel* idea of publishing N*M papers on each combination and labeling it a novel hybrid approach to solving some problem.

Some ideas are just too obvious to deserve publication.

Re:Positive bias in engineering research

[Anonymous Coward]

In engineering research, there is definitely a positive bias; in fact, negative results are rarely published at all. This is both because negative results have less sex appeal than positive results ...

If anyone is reading scientific journals for sex appeal, they are doing it wrong !

Science for Smart People

[Anonymous Coward]

What passes for science is not always science. You can't blame people for distrusting the conflicting and wrong headlines. Yes, the media deserves a lot of blame, but it's not all their fault. Watch this video from Tom Naughton and you'll learn how bad some areas of "science" can be: http://www.youtube.com/watch?v=y1RXvBveht0

R&D

[Lord Lode]

I'm not sure if it's just me, but I've got the feeling that these days, most useful and ground breaking new research and technology comes from companies, not universities.

What do you think?

Re:R&D

[ceoyoyo]

It's just you. Corporate R&D produces products. Sometimes incremental engineering improvements. Not science.

Corporations USED to do some decent science, even more or less basic science. But not anymore.

Re:R&D

[geekoid]

I think that's a classic keyhole problem.

Private companies have a shit ton of marketing and PR.
90% of all projects in large companies fail. 10% succeed. The marketing never mentions the failure.

In the government, 90% of all project are successfully, 10% fail. Because the PR(news media) only focuses on the negative, the success don't get talked about.

This is the same thing.

.

Publication bias

[overshoot]

Publish or perish. And journals prioritize first results on a research question, just like newspapers want to scoop the competition. On top of that, even first news on a hypothesis is less likely to be published if it's negative.

So what do you expect academics to do? Reinvestigating a previously reported research result is unlikely to get funded, it's unlikely to be published even if it is researched. They live and die by publications.

Unless and until the publication system makes it possible for academics to check each others' work without killing their own careers, we won't see it happen.

Re:Publication bias

[drwho]

So, this is behaviorism making itself felt in the realm of science. Good, so perhaps we can take advantage of this by making the penalty for badly conducted science much higher...wait, then you have Fleishman-Pons...and wait again, there's the problem of irrationality in loss avoidance behavior. Well, basically we're screwed.

Re:

[Reibisch]

And ignores the fact that once published, there's no reliable corrective mechanism to propagate those results down beyond a standard literature search. I'm posting as AC because quite a few years ago I published results that I believed at the time to be correct, but were shown to be wrong in a subsequent paper. Despite this, I'm *still* being cited in new papers while the paper that refuted mine is seldom cited. Science isn't some infallible field. We make mistakes; Sometimes those mistakes are accidental, sometimes they're sloppy, and yes, sometimes they're even intentional. That doesn't reduce the validity of science, but it requires us to be more vigilant.

Obligatory XKCD

[codegen]

The case of the green jelly beans.. http://xkcd.com/882/

Re:

[Reibisch]

Crap. I only thought I was posting as AC. Doh! :)

Who cares?

[DogDude]

This article is about trying to convince religious nutjobs that science is valid? WTF is the point of that? People, in 2012, who don't understand the scientific method are hopeless. Ignore them and move on.

Re:The "science is settled"

[benjfowler]

I was gonna say, "Give it time before all the flat-Earthers showed up".

Slow news day on Drudge Report, huh?

Re:Not in Climate Science!

[benjfowler]

LOL.

"First they ignore you, then they ridicule you, then they fight you, then you win." -- Mahatma Gandhi

Same Old, Same Old

[scruffy]

This has always been the case. Science is not a uniform march to the Truth. There is a difference between well-verified and understood results (think engineering) and working at the margins with not much data and the usual human failings (the vast majority of publications). Scientists are humans, not gods. It takes a lot of effort and error to get to the well-verified and understood part.

Scientists and Catholic Priests

[wcrowe]

This is pretty serious. I know that my own confidence in scientific research has been eroded quite a bit in recent years. I can only imagine how it vindicates those who despise science. As far as harming public perception, this is about the equivalent of Catholic priests diddling little boys.

Re:Scientists and Catholic Priests

[geekoid]

You are comparing a know byproduct of science with raping children?

You're an idiot,.

Re:Scientists and Catholic Priests

[wcrowe]

I said "as far as harming public perception". I did not say they are equivalent acts, but the end result is the same. Especially if you try to cover it up by saying that fudging results in order to get more grant money, or to forward a personal agenda, is "a known byproduct of science." If science had a clergy you could be a bishop.

 

Lack of funding has already killed it

[sl4shd0rk]

When you consider what the US gives to it's science budget, the only thing eroding public trust is the paid-for blathering by bought-off scientists. Don't try and hide the real problem with "Positive Bias". That's what you get when you hire someone at a pharmaceutical company to write up a paper for Nature. You don't get independent viewpoints you get advertisement with a scientific undertone.

"A half a penny... The most powerful agency of the dreams of a nation is currently underfunded to do what it needs to be doing."
  - Neil deGrasse Tyson

http://www.tor.com/blogs/2012/04/neil-degrasse-tyson-on-the-nasa-budget

...and inaccuracies

[Roger W Moore]

You forgot to mention that it is yet another piece of published work that suffers from positive bias...

Nevermind that what about the inaccuracies? Economics is not a science and 'biomedical research' (where the article claims is where the biggest problems lie) and epidemiology are medicine. Perhaps the solution is to educate people as to what science is because it is not defined as any subject which publishes research papers containing numbers and/or long sounding names.

Re:...and inaccuracies

[atlasdropperofworlds]

Yes. But it was, I dunno, too circular to mention.

Re:Not in Climate Science!

[tp1024]

You are currently at step 2.

Papers are advertising

[gr8_phk]

Most papers I've seen make magnificent speculative claims.

This novel material could usher in a new era in...
This novel method could become...
OMG this material has an IOR of 1.00000001 for gama rays - this will enable.... (yes this was a few days ago).
OMG jelly spread on glass can produce a voltage when exposed to sunlight - organic solar cells could change the world but more research will need to be done to get the efficiency above 0.3 percent.

Anything that isn't in a textbook is "novel" because that's an important word in patents and it's important to seem new. Then the grand claims are made either to make the research seem relevant or to line up funding for future work. Papers have become promotional material.

It happens in Slashdot too...

[elsurexiste]

I remember a few days ago someone submitted a story about piracy for "The Avengers" being low compared to potential profits from them. A few high-ranked comments were like "This is yet another proof that [insert common /. parlance here]". I saw very few comments that stated the most plausible reason: a camcorded action film, with crappy audio and a shaking image, can't compete against the real thing. I thought the same thing: confirmation bias.

People do it all the time. If something can somehow support their views (specially if they don't RTFA) they'll use it as yet more confirmation. "I still don't get why this piece of evidence is discarded by everyone else! They must be delusional or have bad intentions". For example, I imagine this article will be used as evidence for: lack of funding, falling standards in the US, the demise of education, lack of scientific reasoning (maybe they'll even extend it to scientists themselves), and other common /. utterances. I wonder how many of them will actually say what I found out after RTFA...

So, everyone is playing the same game, and scientists are no exception. But hey, that study has numbers on it. At least you can try to replicate the findings, if only the entry barrier wasn't so high: these tests are *hugely* expensive. More collaboration may be a good idea. Shared laurels are better than none, right?

P.S., a nice article on confirmation bias (and other goodies) here.

Re:It happens in Slashdot too...

[Anonymous Coward]

The thing is, the folks at the RIAA actively claim that piracy is costing them *billions* each year, and is responsible for movies failing to make a profit.

If 'The Avengers', a hugely expensive film which was widely available from piracy sites *before* the movie was released, still turns a massive profit on it's opening weekend, it *is* evidence that the RIAA folks are overstating things. Besides, most of the piracy that happens these days isn't bootleg recordings, it's leaked digital copies transcoded down to downloadable sizes *before* the actual movie is released. If *that* "can't compete against the real thing", then what's the problem?

Captcha: ballyhoo

Science comes when results are confirmed

[Geoffrey.landis]

Actually, science is stll working; the real trouble comes with the publicity of the science.

You should never believe the results of any single study. Every scientist knows this; or should know this. Science comes when results are confirmed, not when somebody publishes the first paper. The real work of science just starts when somebody publishes a study saying "we show that x has the effect y." That initial paper really is no more than "here's a place to start looking." However, newspapers want to publish news, and they need to publish whatever's hot and interesting and being done today, not "well, scientist z had his team take a look at the xy phenomenon to see if there was anything interesting there, and they couldn't really find anything there, although maybe some other research lab might have different results."

And, I suppose that somebody should post a link to the obligatory xkcd: http://xkcd.com/882/

Re:Science comes when results are confirmed

[silentcoder]

There's a brilliant line on this in "The science of discworld" - I won't pretend I can quote it 100% accurately off the top of my head but it goes something like this:
"In the media you will often read that a certain scientist is trying to prove a theory. Maybe it's because journalists are trained in journalism and don't know how science works or maybe it's because journalists are trained in journalism and don't care how science works - but a good scientist never tries to prove her theory, a good scientist tries her best to disprove her theory before somebody else does it for her, failing to disprove it is what makes a theory trustworthy."

Re:Science comes when results are confirmed

[Anonymous Coward]

The FTL neutrino findings are a perfect example.

Re:Science comes when results are confirmed

[Anonymous Coward]

Duplicating results has always been more an ideal than a practice. No one has time for it and it's sort of unnecessary because of two things everyone knows. First, scientific knowledge proceeds like a shotgun: most published articles, right or wrong, lead nowhere and are pretty much ignored with no need of replication. Second, validation can happen not by direct, targeted repeat of an experiment, but by having that experimental result work its way into other experiment designs as an assumption. People assume a result is true, proceed as if it were true, and fInd that things turn out as if it were true. Or false, in which case a targeted reexamination might be performed.

Re:Science comes when results are confirmed

[slew]

I think the real problem today, is that jounalists think they can be stars out of the gate. In the old days, you started out as a fact-checker, before you could get even get a word that you wrote published (usually anonymously at first). By the time you got a "by-line" you have come up through the trenches and seen all of the behind the scenes mistakes that the "star" writers made. Today you have a blog and no editor. Not only quality journalism goes out the window, but these new so-called journalists don't learn the consequences for some of their habits, because they haven't seen others make them (and haven't been motivated like a fact checker to find the problems).

Of course this "star-at-birth" issue isn't just problem with journalists, but many professions (e.g., scientists, chefs, programmers, etc)...

Re:Science comes when results are confirmed

[rgbatduke]

Actually, you need to read "The Black Swan" by N. N. Taleb. Science that tries to confirm a theory is already infected with confirmation bias. There are a pile of examples that demonstrate the fallacy of confirmatory inference. Taleb uses a variant of Bertrand Russell's -- a turkey might reasonably infer, based on his daily experience, that humans exist for the sole purpose of feeding him, caring for him, providing for his every need. This might go on for day after day, increasing the turkey's degree of belief in his hypothesis of a good and beneficent humanity filled with love of turkeys, right up to the day that -- ulp -- something unexpected happens.

I'm not a Popperite, rather a Jaynes-Cox-Bayesian, but nevertheless it is important to avoid confounding the relative strength of positive and negative evidence. Absence of evidence is not the same as evidence of absence, yet we almost invariably confound the two.

Taleb damn skippy agrees with you about publicity, however, and the near-criminality of publicity and reporting of science. A newspaper necessarily takes a scientific result or observation and transforms it two ways: First of all, it creates a narrative. It isn't just "a tornado hit Houston", but "a tornado hit Houston, possibly caused by Anthropogenic Global Warming" with the subtext "this isn't an act of nature, random an unpredictable, but is instead our fault". Aztec priests couldn't have come up with a better excuse for ripping the still beating hearts out of a stream of slaves and war captives. Second, it necessarily reduces the complexity of the result to no more than three variables, ideally one. It "Platonifies" it (according to Taleb) -- wraps it up in a pretty, easy to understand package that makes it more predictable, less random than it really was. Global warming is a much simpler "cause" than "A cold front overrunning a warm wet surface layer of air near the ground, creating turbulent rolls that break off and terminate on the ground, sustained and driven by the thermal difference, and it is a better story too.

Sadly, as you point out, real science is all too often (and should be) scientist z looked at something and didn't find much. But what they failed to find and how they looked is actually often as or more important than a study that claims to find something, especially when the latter uses questionable methodology to try to prove something, cherrypicks data (for the same purpose), ignores silent evidence (ditto) etc. Medical science is permeated with this. Nobody gets famous, or rich, or even a job, for looking for a cure for cancer and not finding one. This too is addressed by Taleb. Great book.

rgb

Re:Science comes when results are confirmed

[rgbatduke]

Well said and yes.

Re:Science comes when results are confirmed

[Belial6]

Bad "journalism" started WAY before blogs.

Re:Science comes when results are confirmed

[Anonymous Coward]

There's a brilliant line on this in "The science of discworld" - I won't pretend I can quote it 100% accurately off the top of my head but it goes something like this:
"In the media you will often read that a certain scientist is trying to prove a theory. Maybe it's because journalists are trained in journalism and don't know how science works or maybe it's because journalists are trained in journalism and don't care how science works - but a good scientist never tries to prove her theory, a good scientist tries her best to disprove her theory before somebody else does it for her, failing to disprove it is what makes a theory trustworthy."

Unless, of course, that theory is someone's sacred cow.

Then if you try to disprove it, you'll be labelled a heretic^H^H^H^H^H^H^Hdenier.

Can you spell A-G-W?

Re:Science comes when results are confirmed

[Geoffrey.landis]

Duplicating results has always been more an ideal than a practice. No one has time for it and it's sort of unnecessary because of two things everyone knows. First, scientific knowledge proceeds like a shotgun: most published articles, right or wrong, lead nowhere and are pretty much ignored with no need of replication. Second, validation can happen not by direct, targeted repeat of an experiment, but by having that experimental result work its way into other experiment designs as an assumption. People assume a result is true, proceed as if it were true, and fInd that things turn out as if it were true. Or false, in which case a targeted reexamination might be performed.

Yes, exactly. When a result is right, people build on it, and in the process of building on it, they repeat it and confirm it.

Scientists rarely say "here's a published result, let's make an exact copy of the apparatus and do the exact same experiment with the exact same conditions and see if we get the exact same result." It's much more "X published an interesting finding in p-doped GaAlSb crystals produced by LPE; we have a MBE system with an antimony source; let's give it to a grad student who needs a project and see if we can grow those crystals by MBE; if we can do that, then we can go on and do Y."

Re:Science comes when results are confirmed

[Anonymous Coward]

a tornado hit Houston, possibly caused by Anthropogenic Global Warming

AGW will cause more frequent occurrences of extreme weather and climate events. Each event by itself cannot be said to have been caused as a result of AGW because we're talking about averages. How do we know last week's storm wasn't the one that wouldn't have happened vs. this week's? On the other hand, you cannot either say that it ISN'T due to AGW. There's no way to link to two, ever. That does not mean that AGW is a falsehood, or that the reporting is even incorrect.

Re:Science comes when results are confirmed

[Anonymous Coward]

Unfortunately if you try to disprove a theory in the scientific orthodoxy you will have problems getting published and funded. Worse you may be branded a skeptic and then a denier and then the pseudo-science crowd (those who think they are experts in a field because they read a book about it of follow a couple blogs) also pile in.

Re:Science comes when results are confirmed

[Curupira]

On this theme, I prefer the obligatory PHDcomics: http://www.phdcomics.com/comics.php?f=1174
:)

Re:Science comes when results are confirmed

[slew]

Bad "journalism" started WAY before blogs.

True, yellow journalism has been around for a long time. Just like most things it wasn't invented by the internet, but there were options in the past. Now it's pretty much all sources of news are generally un-researched journalism...

I'll submit that scientific publishing is getting to be this way as well. The bad science isn't new (n-rays?), but as the cost of entry goes down for scientific publishing, and the older journals become "too-big-to-fail", the desire for more "impact" the scientific publishing industry seem to be getting more "yellow"...

Re:Science comes when results are confirmed

[Anonymous Coward]

No, but climate science is. Even the average Slashdot reader does not know how many valid scientific studies there are that goes against mainstream CAGW, they just keep repeating a popular mantra about "consensus", "97%" etc.

CAGW is a textbook example, actually. Did you know that all hockeysticks have been falsified? Haven't their authors told you?

Re:Science comes when results are confirmed

[Anonymous Coward]

Sweet lord thank you. I can't believe I had to scroll as far down as I did to see a post about NNT. His work is how to avoid EXACTLY this sort of problem.

Re:Science comes when results are confirmed

[Anonymous Coward]

http://en.wikipedia.org/wiki/Hockey_stick_graph

More than twelve subsequent scientific papers, using various statistical methods and combinations of proxy records, produced reconstructions broadly similar to the original MBH hockey-stick graph, with variations in how flat the pre-20th century "shaft" appears. Almost all of them supported the IPCC conclusion that the warmest decade in 1000 years was probably that at the end of the 20th century.

Sorry, your assertion is not credible. Put up with evidence, or shut-up.

Re:Science comes when results are confirmed

[Anonymous Coward]

AGW will cause more frequent occurrences of extreme weather and climate events.

Not according to available data. The above is a hypothesis, with currently very little support.

Your post is a good example of the article subject.

Re:Science comes when results are confirmed

[Anonymous Coward]

Bristlecones: falsified (Mann #1)
Tiljander: falsified (Mann #2)
Yamal: falsified (Briffa)

There are no others. Note the weasel words in your citation.

Your post is a good example of the article subject.

Re:Science comes when results are confirmed

[Anonymous Coward]

Sadly, as you point out, real science is all too often (and should be) scientist z looked at something and didn't find much. But what they failed to find and how they looked is actually often as or more important than a study that claims to find something

Sadly, such real science is usually unpublishable.

Re:Science comes when results are confirmed

[dewrox]

You say you should not believe the results of an individual published paper. Is it not more common sense to not publish a paper until results are reproducible and verified? Every other person in the world has this common sense. Why is it that sientists, who should by default have this common sense since they are supposed to be the more elite thinkers, do not?

Re:Science comes when results are confirmed

[Anonymous Coward]

This is what people would like to believe scientists do. Actually, most scientists don't try that hard to disprove their theories. They send them out into the world, and some other scientist tries to knock them down. From this rigorous debate emerges - eventually - the battle-scarred victor.

Re:Science comes when results are confirmed

[squiggleslash]

That's complete bollocks. Virtually all science is about trying to disprove theories. The less successful scientists are at disproving theories, the more a theory holds up.

What you meant to say is "Unfortunately if you claim to have disproved an established theory in the scientific orthodoxy, but fail to do so in a convincing way, ignore the counter arguments, and then pretend there's a giant conspiracy against you, you will have problems getting published and funded." That's pretty much the only time what you're trying to describe, badly, happens.

Re:Science comes when results are confirmed

[WOOFYGOOFY]

I am only going to take the first of this denier meme because I am short on time. perhaps someone else has heard of Google and can contribute rebuttals to the other denialist talking points

"Bristlecones falsified" WRONG WRONG WRONG.

This is completely normal science where papers are properly subjected to post publishing criticism, and then that criticism is answered and back and forth like this. There was NO FALSIFICATION and if you want the details they're here

http://www.realclimate.org/index.php/archives/2005/01/peer-review-ii/

Number 5 at the bottom.

Re:Science comes when results are confirmed

[WOOFYGOOFY]

Sorry that's number 3 at the bottom... number 3

Thanks for that.

[Anonymous Coward]

In my early middle age, sometimes when looking back I wish I went into the natural sciences. My current business/tech career seams really meaningless most of the time and I think if I did something more "worthwhile" like research, then things would be better - you know that whole the grass is greener over the septic tank mind tricks.

When I see things like what you posted, I realize that things turned out OK. The World didn't lose a brilliant scientific mind, either - I honestly think I'd be a mediocre scientist at best. I had this dream briefly in college of being an experiemental physcist, limnologist or a botanist - I'm sooo glad I dodged that bullet.

It's just not your post either. I hear things from rank and file researchers about their own misgivings about their work, the profession, and the BS they have to deal with.

What ever happened to trying it yourself?

[Anonymous Coward]

Get an independent lab to confirm your results if you want to be published. Ftfy

Re:What ever happened to trying it yourself?

[Anonymous Coward]

The point of publishing is to document your processes so that an independent lab *can* confirm your results.

Hypothesis: Personification improves receptivity.

[VortexCortex]

Materials:
You will need an inanimate entity or concept, an emotional situation, a forum of discourse, and a mind with approx 80 to 120 billion neurons.
Procedure:
0. Select commonly known, generally accepted concept or phrase.
1. Personify the inanimate subject.
2. Rephrase the concept or phrase including the situation designed to invoke an emotion, such as sympathy.
3. Await until an opportunity with sufficient relevance presents itself.
4. Present the phrase in the forum of discourse.
5. Cite a nebulous entity, so as to lend artificial credence.
6. Observe the Science!
7. ...
10. Prophet!

------

"If you torture the data long enough, it will confess to anything."
-- Unknown Econometrician.

Scientists are individuals

[Burz]

...not just "scientists" who collaborate and challenge as a whole discipline.

They interact with the mass media at the level of individuals and very small groups. As such, most journalists won't pass up an opportunity to quote a researcher who says something like, "This changes everything!" and the spirit of such statements (if not the actual statements) are usually found throughout the articles written by "science" reporters. They play up the significance of individual studies and competition and downplay (or attempt to discredit) the consensus-built and collaborative aspects of science so that the stories fit in with the individualist and consumerist narratives favored by the corporate press.

That dynamic contributes greatly to the public's distorted notions of scientific pursuit. Thus, the popular American sentiment that scientists are mostly bozos "lacking in common sense", and that is when they feel charitible. You'll need to change the overall media culture before you can improve the situation.

Also, I keep thinking about the TV show "The Big Bang Theory". Is it me, or is the cast segregated into the engineers who manage a real social life, and the scientists who are not just eccentric but extreme (and extremely pitiful)?

Re:Obligatory you're an idiot.

[SirFatty]

Really.

Software peer review

[jackjumper]

With my five seconds of thought,

It seems to me that if scientific papers were published with a series of standard meta data they could be submitted to software based peer review that could go through the published literature and look for inconsistencies, missing references, misleading statistics and other things that need addressing.

Sort of a semantic web for scientific publishing.

Re:Software peer review

[oh_my_080980980]

They are or shall we say to publish any research your data and methods must be published in the paper. This way people can review your work.

The key here is REVIEW THE WORK! This jackass has you believe the scientific method is in questions when it's the publishing methodology of scientific journals that needs to be called into question.

The sky is not falling. Corporations push biased research results. No surprise there they've been doing it for decades. What matters is that peer reviewed journals do their jobs and actually review work. Make sure what they publish is verified.

This is not new

[geekoid]

It's know effect and why repeated testing is an important part of science.

The problem is the media keeps reporting single studies as if they're an answer, or written in stone.

Re:This is not new

[the eric conspiracy]

It's a problem alright. But not with science. If you get a positive result that appears useful, publish it so others can try to repeat it to see if it's valid.

The problem is that every old exploratory result is breathlessly slammed into the mass media by journalism majors who think if it's published in a peer reviewed journal it must be Graven in Imperishable Stone and one of the Fundamental Laws of the Universe.

Science is a much slower process than all that. But journalists don't care, their need is to sell ad impressions.

Erode trust? That's what they are concerned about?

[Arker]

That's dumb. It's eroding *science* - turning science into religion. If the only problem you see with that is the erosion of public trust then there is something wrong on your end.

That said, the observations in the article are spot-on, and it's good to see others finally noticing this. And as the article notes, it's not limited to medical research. Not by any means. Climateology in recent years has been another area where this problem is quite severe.

The myth of positive bias

[ceoyoyo]

Okay, it may not be exactly a myth. We can't tell. I strongly suspect there is actually a negative publication bias.

What most people think are "negative" results are actually inconclusive. A non-significant p-value is NOT a negative result. That misunderstanding is very widespread, and leads to lots of high level mistakes. Half of the neuroscience papers published in top journals including Nature the last two years that could make a mistake based on that fallacy, did. And neuroscience didn't seem to be particularly worse than most other fields.

A non-significant p-value is just that - not significant. Inconclusive. Getting an actual negative result is considerably more work than getting a positive one. You need to figure out what the minimum effect size you're interested in is (you should do that for positive results too, but almost everyone just uses zero for that) and show that your confidence intervals do not include it. As Ionnadis points out, you really should consider the power of your study as well (also for positive results), and take a stab at estimating the priors too.

If you go and do all that, and also do a quality experiment, in my experience you actually have a pretty good chance of getting published, because a) it's clear to the reviewers you've done a really thorough job (any idiot can run some data through a t-test and get a p-value, negative results are harder) and to show a negative result your study is probably much higher powered than a positive result one, meaning an impressively big p-value.

The problem is not that there's a positive publication bias, it's that most scientists don't know how to show negative results so there are very few negative papers around.

Its also about non-scientists expectations

[siberian]

Increasingly people have been taught to trust only certainty. Science is anything but a certain process and people take that uncertainty as false or, worse, a wasted investment.

Until people are more scientifically literate with the process and the value of failure scientists will be driven towards only success and, ultimately, the positivity bias.

Re:Its also about non-scientists expectations

[cdrguru]

Sorry, but you are asking for something that can be called the "everyone should be at the top" problem. Yes, it would be nice if everyone could understand science, scientific methods and what constitutes the difference between one result and a confirmed, peer-reviewed body of research.

It isn't going to happen that way.

What we need is to stop publishing unconfirmed, preliminary results in popular media. That by itself solves 90% of the problem. So who gets to decide when something is ready for 60 Minutes, Fox and Friends in the Morning or the Today Show? I don't know, but I know it isn't the news media and it should not be the original team reporting some unconfirmed results. Just a rule that says nothing gets copied out of scientific journals until it is marked as having been confirmed by at least one independent team would go a long, long way.

Re:Its also about non-scientists expectations

[siberian]

To do this we need to stop the 24/7 news info-tainment cycle, totally agreed. But I don't agree that asking people to vaguely comprehend the scientific method is a case of 'everyone should be at the top'. People can drive cars and do any other number of complex things with minimal education. They can do this.

Re:Not in Climate Science!

[geekoid]

Also
"First they ignore you, then they ridicule you, then they fight you, then you die." -- Hitler

Mahatma Gandhi was an ass.

Re:

[jbcksfrt]

If you have a personal web site or something similar, it may be a good idea on your part to make that clear and link to the refuting paper. It would take some guts on your part, of course, because it could negatively affect your career, but it would probably be the right thing to do.

Perception IS the problem

[macraig]

Do Slashdot readers perceive positive bias to be a problem?

It doesn't matter what I perceive, it's a problem regardless. I can perceive any damned delusional thing I want, and that changes the laws of the universe not one whit. It might make me happier in the short term, until one of those laws I chose not to perceive happens to bite me in the ass.

Perception IS the problem here. Scientists engaging in this behavior are almost certainly doing it knowingly; they're aware they're not practicing the Method. They desire to be perceived as successful, inventive, and ground-breaking, and they're willing to cheat, in essence, and forego inconvenient parts of the Method to achieve that perception. Their own personal success is more important to them than the science.

A few thoughts

[Cobble]

1) This is all part of the process, and shouldn't be too much cause for concern. Interesting results will be re-tested for confirmation. If these results can't be replicated, it casts doubt on the original study and the reputation of the authors. Negative results are rarely published except when they challenge a prior paper with a positive result. 2) The citation index is another way that bad data gets ignored. The average citation for a paper is less than 1. This means that most papers are never cited and a few good ones are cited heavily. It is true that if wrong ideas become generally accepted, they may remain in the literature unchallenged for some time and can be hard to root out. 3) I suspect that research in biomedical and other highly profitable fields bias these results, since whenever there is huge money to be made, there is pressure for a positive finding. Many of these studies are even bankrolled by the drug companies who want to show that they have the next promising cure or, better yet, treatment--since if you cure someone, you lose your market ;) It's always good to look at who's funding the work.

Science doesn't require 'Trust'

[Anonymous Coward]

Skepticism actually strengthens science.

Sometimes when people hear about a study in the news about some vague statistical connection, and treat it like it's a newly discovered fact. If that 'trust' is starting to erode, then it's time for rational people to throw a party!

Public trust?

[Anonymous Coward]

You must be american.

Correction

[Arker]

Global warming: Fraud with a leftist political agenda
Creationism: Fraud with a rightest political agenda

FTFY

Re:Correction

[Anonymous Coward]

You are a moron. Evolution is full of so many fucking holes you could sail a boat through it.

Re:Correction

[Anonymous Coward]

Yes, but they're holes in the knowledge of particular 'steps' in specific evolutionary branches. And they get closed all the time when new specimens are discovered.

The overall evolutionary model has stood up to *hundreds of thousands* of attempts to disprove it, and has withstood all of them.

The problem isn't the science

[Dcnjoe60]

The problem isn't the science. It is the publish or perish mentality. The science is just as valid if result "a" happens or not. However, it's a lot harder to get funding for research if your research keeps coming up negative. 75 years ago, research occurred for the sake of research -- to expand knowledge. Today, research is for the purpose of developing something that can be monetized. 75 years ago, a research paper that showed that x didn't happen under y circumstance was still considered valid.

Today, it is not. Look at the wording on many grants to day: To develop a ...... whereas yesterday it was: To determine the effects of ...... As more and more research got funding by the private sector, colleges and universities shifted their focus to apply research in areas that could be profitable as that is what the buyer (the one paying the grant) wanted. Put differently, as the private sector shifted more and more of its R&D to colleges and universities, the positive bias was introduced because you get what you pay for.

It's everywhere

[Anonymous Coward]

In many fields the problem gets hidden by statistical tests. Frequentist statistics _assumes_ that the hypothesis being tested is true. Even a totally bogus conclusion, provided it is sufficiently different than a non conclusion, will end up being statistically significant. Many people take statistical significance as confirmation of their hypothesis and neglect to consider alternative hypothesis.

The solution to this problem is to demand verification of predictions, and not just rely on a p value. If your research involves building a model then you should test the model against data that wasn't used to fit the model parameters. The measure of utility should be how much better the new model performs versus other models.

It really doesn't matter how bizarre a model looks as long as it works better than the alternatives. It's not as though quantum physics is exactly intuitive, but we accept it because it's currently the best predictor.

Positive Bias?

[tpstigers]

What the hell does 'Positive Bias' have to do with the latest AMD processor, Linux distro, or Apple product? Do you even know which site this is?

Re:Positive Bias?

[game kid]

You mean the reviews those things get don't have positive bias? ;)

How do we know?

[mcmonkey]

Evidence is mounting that research is riddled with positive bias.

Maybe that's just what the researchers wanted to find.

The difference is...

[Dcnjoe60]

The difference is positive bias on Slashdot, as in the Avengers movie doesn't really have much impact on the world. Positive bias on the cancer trials mentioned in the article has serious ramifications. So while it may happen everywhere, some places are more critical and some occupations should be held to a higher standard.

it's a problem, here's how to fix it

[Goldsmith]

I'm a scientist. It's a big problem.

Here's how you fix it:
The metric for success for both researchers and their government funding sources is published papers. It's hard to change the cultural view of the scientific community, but it's easy to change government metrics. Paper publishing as a metric is easy to track between programs, but has had a terrible impact on scientific culture. It's also led to a large bias in how the government decides what areas to fund. If your metric is paper publishing and you're looking at energy issues, do you fund a sub-field with a historic high paper publication rate, a moderate paper publication rate or a low publication rate? It's fine for us here to say we'd fund the best research, but a government program manager may lose his job for picking a field with the lower publishing rate.

Other metrics such as how many other researchers use some results or whether a practical implementation of some new technique is developed will be harder to judge and take a longer time to evaluate, but would at least give us an honest assessment of the quality of government funded research. Tie future funding to what our broader society is looking for out of science, and eventually the scientific culture will follow.

Re:it's a problem, here's how to fix it

[davidannis]

Having the government stop funding based on publication rates won't fix the problem. As long as Universities retain and promote based on publications scientists will do what is good career wise. What happens is that a scientist spends years and millions of dollars gathering data to answer question X. Lets say for example, can I increase the IQ of people by having them play WoW while doing multiplication problems. He finds that the answer is not what he had hoped. Nobody will publish his paper, but he has a family to support so he tries to "salvage" the data by looking at subsets. After testing the data against 100 different hypotheses he finds that he it supports the idea that teenage girls can increase their IQ that way, which is shocking and publishable. Of course, it is just an artifact of testing enough hypotheses against the data. He goes ahead and publishes. Unlikely that anyone will bother to replicate the study but now he can pay the mortgage and send Jr. to college. To really make things better you need to require publication of negative results and score science based on methodology rather than just on whether results are surprising. If every researcher who was funded had a guarantee of result publication included in their grant whether the was positive or negative and the research was peer reviewed and was publicly scored not for how unexpected the result was but for how well the science was conducted that might solve the problem.

You're all misunderstanding this

[PHCOSci]

Discussing the public perspective on science is fine. It's waning, why? That's a good question. But that's not the point here.

Talking about how scientists get "good" data is also a great topic, statistics, error bars, how we come by those numbers. Great points. But also not what's important here.

What is being highlighted, and has been since Ioannidis' publication, is that the overall CONCLUSIONS of these works are just wrong. The data is good. We collect great data, we put it out there, it's within reasonable error and is a true and observed phenomenon. The major problem is that we tend to do the WRONG experiments, or we direct our research toward answering a specific question, instead of gathering data comprehensively and looking to that data exclusively to find appropriate questions.

This bad rigor is prevalent, at least in biological science publications. I can't speak with any authority to any other discipline. Data taken out of context, and worse still, ignoring "inconvenient" observations that debunk proposed models is where a lot of this "bad" science comes from. Scientists will proclaim until they are blue in the face that they'd "never do this" - but I see it happen every single day, from major researchers. I fight with myself every day to not act this way, and it's still very hard, because I'd like to move forward in my career - and some crappy piece of data that shoots holes in my hypothesis is staring me in the face. The easy route is to just ignore that. "Oh, yes, this is a caveat we don't want to get into as it muddies the publication. We'll approach this problem in the next article". That's crap. You can't observe something and then ignore it and publish to the contrary of what that data says just to have a neat little story.

What I've outlined above is the underlying problem. Most of what is published, the conclusions being made, are flat out WRONG. The data is right. But what researchers are saying that data means based on their myopic and "I need to publish this" drive causes tunnel vision and bad conclusions.

I have no idea what the answer is, but I think it's important we recognize what the problem is and what the question is, if we have any hope of getting to an answer.

I just knew it!

[Anonymous Coward]

Evidence is mounting that research is riddled with positive bias.

I just knew it! This was exactly the result I was expecting!

But then comes heritage.

[Isaac Remuant]

Even if you had a group of responsible an fair enlightened people, there's no guarantee that those people would have friends and families that lived up to those standards. Why am I saying this? Because people want to take care of their friends and family and will eventually exercise their privileges in order to do so.

Eventually, their efforts to ensure the well-being of their circles will advance inadequate people into the ruling groups.

It happens regardless of the political system.

Well, duh.

[rickb928]

One of my cousins has worked for a pharmaceutical company for a few decades as a statistician. Her job is to evaluate research and report on the validity of the analysis. She has told me often that she rejects a lot more than she accepts, and her bosses have always implored her to challenge the findings she's been given, to prevent what we are discussing as 'positive bias' (or as they put it, 'avoiding recalls, liability, and financial ruin'). She calls it 'lying' or 'wishful thinking', if she's feeling charitable. And her opinion of some researchers is that they are gifted, but blinded by their fervent wishes to be successful. And yes, delivering products to save lives and ease suffering is secondary to 'success' for many of them. They don't, as she puts it, 'get the concepts of liability or deceit'. All they want is a bigger lab, more assistants, and an unassailable position. That's all. And a nice hot cup of tea, I'm sure.

My favorite quote from her: "That's a statistical test they use to make data out of noise".

Positive bias is inevitable. Probably started before Galileo, and of course he struggled with that a lot. No less a problem today, though we punish heretics somewhat more humanely today. We either premote them to government jobs when we agree, or we fire them when we don't. Not many go to the stocks any more.

troubles doing proper science

[bzipitidoo]

There's nothing wrong with science itself. But there are many barriers against doing science correctly. I can't say for sure if the barriers have gotten worse in recent decades. But I think yes, it has become harder to do good science. Why?

The Space Race and Bobby Fischer's day in the sun are long gone. They were good contests, and well-liked, with very positive outcomes, especially when compared to the alternative of nuclear war. In a way, it makes up for science having discovered The Bomb. It was possible that no one would make it to the moon. We were not going to suffer big losses if that had been the outcome. As for Fischer, he looked more and more unstable as the years passed.

But one thing about The Bomb: it demonstrated the power of science, even if very darkly. After that, the Space Race demonstrated the power of science again.

And now? It's really hard to top The Bomb and The Moon. But, by some measures we have. We have the Internet, which has been, among other things, a virtual nuke to the traditional business model of Big Media. Perhaps that's too subtle for the more cretinous among us.

That's what I think is a big factor in the current state of science. We're still in the letdown following the big high of the moon landing, and the awesomeness that is the Internet has not been appreciated enough.

Some of the stretching and outright fraud is a symptom of that letdown. We're straining to meet impossible expectations. It's as if we have to top the Moon to impress the public. One of the more crudely obvious ways to do that is to visit Mars. Send a human to Mars. Except it can't be a mere visit, it would have to be a prelude to colonization. Can we do that? Not presently.

Is there any other reason to suppose that fraud in science is any worse than in the past? Maybe. I think also that science has become more accessible, which on the whole is good. But a bad consequence is that it's easier for quacks, cranks, and cons to fake it.

Finally, there is motivation for the increased fraud: bias against bad news. Right now, one of the loudest message scientists are giving is AGW. The frauds have seized on public dislike of that negative message. They've been riding high ever since Big Tobacco showed them the way with "Doubt is our product", and they've corrupted and captured the social conservatives. The Republican Party has fallen a long way from the sober fiscal pragmatism of the 1950s. In those days, they were the sane, careful counterbalance to the misty and foolish idealism of the Democrats. The book "The Republican War on Science" would have been laughable in the 1960s. Most "squares" were Republicans, while many Democrats were dirty, drug addled, brain rotted, uneducated hippies. No one would have asked if a Republican president was our worst ever, as Rolling Stone did. Now, the Republicans are the crazies in denial.

Absolutely

[Will Steinhelm]

I work in optics and computer vision, and investigate research papers on a regular basis. In the majority of the papers I read the methods proposed either only work for the specific example in the paper (some don't even do that), or only work under such "ideal" circumstances that they barely work under lab conditions, much less real world cases. There are a few gems, but you can tell that much of what is published is just written so someone gets that degree or that quota to reach tenure. Prior to working at this job, my trust in published research was much higher. Now my tendency is to treat it all as garbage until it proves itself otherwise.

Yeah, Right

[Anonymous Coward]

I would believe it, but then I would be skeptical of this findings so know I think the meta-meta-data is wrong...

The publication process is probably in part to blame for the perception people have of science. The notion of peer review may make one more inclined to believe an issue is settled when in fact there may be problems raised by reviewers that, although addressed to suit publication, are still open. The public rarely sees the questions others in the field may have with respect to a paper so are willing to accept the published paper as conclusive fact rather than further evidence.

It's all about evolution and climate change

[Anonymous Coward]

People who question evolution hear up and down, "Evolution and climate change are proven true, you're nuts to deny them". Then they hear "Scientific theory X has been changed". They see this as a disconnect in the logic -- science declares things as absolutely true when science has shown its own previous beliefs to be false. And thus, they no longer trust science.

Re:Not in Climate Science!

[Anonymous Coward]

No, we're currently at the transition between Steps 2 & 3. Fortunately, because this is a 'war' of ideas, the fighting is simply stepped up ridicule, and trumpeting of falsities to combat the truth.

Scientific Method is an obsolete tool of "Deniers"

[Anonymous Coward]

... psst. Follow the money...

Either/Or

[ThomasLB]

I think the real problem is the "all or nothing" approach the public tends to take.

"I found a study on climate change that is mistaken; therefor, every study on climate change is mistaken!" "I found a minor tenant of Darwinian Theory that has been modified; therefor, every theory of evolution must be a lie!"

Science has always been contentious. It's really hard to admit we're wrong.

A perfect example of what?

[Anonymous Coward]

To me it seems a perfect example of how NON SCIENTISTS have made the expectations that scientists are not able to live up to.

The report was basically "Hey, guys, this looks odd, but we've corrected for everything we could think of. You may want to see if you can replicate this".

Nobody else managed to replicate.

MEANWHILE the original posters were being told how they had it wrong. They changed their process to accord for this and retained some extra difference, despite this. Another attempt by another team showed no effect, so the original scientists CONTINUED to see what could be the cause if not FTL neutrinos.

They then found that the 50ns difference can be accounted for if the connectors were not tight.

NOT, at that time, that this was the cause of the discrepancy, but that

a) the couplings were loose NOW, but they didn't check THEN, could have loosened since then
b) an effect of this is a slight delay, enough to remove the difference they'd seen

All absolutely fine and no "positive bias" AT ALL.

But what is the public saying about it? What were the press saying about it?

THAT debacle (and how it's ignored by the parties involved in it) is a perfect example of the problems science has with people who aren't scientists.

Re:Not in Climate Science!

[tp1024]

I was referring to his post.

You need to work on your OWN statistics.

[Anonymous Coward]

3C+/- 5% and 4C +/- 7% has a difference of 0.97C.

20C+/- 15% and 40+/- 35% have a bigger difference (3C).

And the old global models had a climate sensitivity range of 1.5-6C per doubling. Now we get 2-4.5C per doubling.

I fail to see where your claim "every year a new estimation would come out that was completely incompatible with the previous one".

I'm surprised at how biased it is

[swb]

It's amazing to me how common bias is in the healthcare field.

After reading Gary Taubes' "Why We Get Fat" and another piece he wrote for Science on the politics of salt research (http://garytaubes.com/wp-content/uploads/2011/08/science-political-science-of-salt1.pdf) it seems clear to me that there are a lot of well-placed individuals, who in spite of a solid science background aren't willing to give up long-held beliefs, even when the scientific studies they championed don't prove their theories.

In fact, the more ambiguous the results the more they tend to push the results as if they were definitive in favor of the researcher's viewpoints.

While this is bad for science generally, in healthcare its particularly disturbing because you have people whose reputations and careers are tied to opinions that can't be scientifically validated or worse -- are invalidated -- and yet they continue to be voices for health policies which are at best ineffective and at worst dangerous.

Wow...

[Will Steinhelm]

Thanks for posting. This should have a slash dot story of its own. Would love to see more opinions on this.

Don't conflate medical research with all science

[Benfea]

Let's all try to be careful, here.

Medical research was a little late the science party, and they still have serious issues to work out. Some of these problems can't be helped (low sample sizes lead to higher p values, but higher sample sizes place more human beings at risk), others can (not being skeptical enough of research conducted by organizations with a financial stake in the outcomes).

Most of these problems are peculiar to medical research and should not be conflated with all of science. While we should hold the medical research community's feet to the fire over this, the reason we are hearing so much about this is that they themselves are talking about it and they themselves are conducting the meta-research that is producing all of these articles. Other branches of science have looked down on medical research for a long time, but at least they are now getting serious about addressing the problems.

Positive bias...

[Anonymous Coward]

My first epiphany on the subject of positive bias was dark mater and dark energy. Quite frankly, the properties of dark mater and dark energy are nothing short of fantastic, least of which, the the universe is almost entirely made of the stuff. It's far easier for me to believe some math wizard made a mistake than to believe the universe is made from pixie dust. Given the current state of science, I made it a personal mission to go all the way back to science of the Victorian period to present and review firsthand the important experiments forming my own conclusions. I my three years of review to date, I discovered a number of unexpected surprises. First, history was very kind to Mr. Einstein. Many of the ideas attributed to him in history books were discovered and discussed long before him. Let's take one example, the famous equation -- E=MC^2. The matter/energy equivalence was published by Gustav LeBon long before Einstein published his papers. In fact, let's let Mr. LeBon speak for himself on the energy locked up in matter,

“Certainly it would be quite different if radium or any other substance were dissociated rapidly instead of requiring centuries for the purpose. The scholar who discovers the way to dissociate instantaneously one gramme of any metal — radium, lead, or silver — will not witness the results of his experiment. The explosion produced would be so formidable that his laboratory and all the neighboring houses, with their inhabitants, would be instantaneously pulverized.”

Mr. LeBon's description is simply chilling and has great impact upon us and our politics today. He is, of course, describing nuclear reactions. LeBon is almost entirely forgotten today. Science is subject to all our human frailties. Science is supposed to be empirical but data must be collected, interpreted, and conclusions presented -- all of which can be influenced by those in positions of prominence. In some cases, many dollars and jobs are at stake when new science leading to disruptive change occurs. This is not a new phenomena, science has never embraced disruptive change. One only needs to look back to sciences greatest heros for evidence. Great men of science like Ohm (Ohm's Law) and Robert Meyer (Conservation of Energy) were almost completely disenfranchised from science for their radical (and correct) views. milton.smith.rr@gmail.com

The problem with "landmark" studies

[tgibbs]

Something that most scientists know, but that is not widely appreciated by the public, is that "landmark" studies are particularly subject to positive bias. For a study to be acclaimed as a "landmark," it is the first report of a novel phenomenon, which by definition means that it has not yet been confirmed by other investigators, and moreover that it is in some sense unexpected--which means that there is not even much "indirect" evidence from other sources that it is correct. It is also likely to be submitted to one of the high-profile "newsy" journals, like Science or Nature, where submissions are not evaluated solely on the basis of whether the science is high quality, but are first screened (and usually by an editor, before it even gets specialist peer reviewers) on the basis of whether it is of "broad interest." Because these journals are widely read and cited , they have high impact factors, so even getting published in one of these journals is a feather in a researcher's cap. In contrast, a publication in a middle-rank journal does not give a major immediate boost to a scientist's career, but may have a long-term benefit--but only if it turns out to be correct. Being "scooped" (by having somebody else publish research leading to the same conclusion) will not prevent you from being published in a middle-rank journal, but it will knock you out of the newsy journals. So the alluring prospect of publication in a newsy journal may lead a scientist to be a bit less cautious than normal, and the risk of being "scooped" means that he/she may not take usual precautions (like seeing if somebody else in the lab can reproduce the result independently). High profile journals are stringently reviewed, but peer reviewers even in these highly-regarded journals necessarily take the authors at their word, unless there are blatant errors--nobody visits the authors' labs to look over the shoulders of the researchers or to verify that negative results have not been discarded. On the other hand, if you are planning to publish a result in a middle-rank journal, you are more likely to take the time to make sure that your conclusions will hold up over the long haul.

Working scientists know this, so they take these "landmark" studies with a grain of salt until they are independently confirmed. But the public can easily confuse "high profile" with "most reliable."

Ob' comics

[DrYak]

And, I suppose that somebody should post a link to the obligatory xkcd: http://xkcd.com/882/

And the obligatory PHDcomics: http://www.phdcomics.com/comics/archive.php?comicid=1174

In addition of the media needing "the latest hottest news" and thus over blowing the latest new subject (which merely has 1 study using a sample of 10) instead of making a deep review of a well studied subject, there are also problems comming from within the scientific community too :

Lots of research groups are under "publish or die" pressure: they a under pressure to publish as many papers as possible, and thus will tend to publish whenever they got a small "bump" in the signal - "which may under some circumstance be interpreted as something unexpected" - specially when it was never seen before (= unknown, more chance to attract attention, than being study number #23 that confirms that the well studied substance "XyZ-wateverocetone" and disease "Somethingitis" have no interraction, no matter what a paper published 13 years ago). There's a preference for original research in the scientific media, conferences, and so on.

Also, most research groups need funding, and thus whenever something has a slightly above average correlation factor, the paper will automatically jump to the conclusion "could one day be used to cure cancer" trying to whore some investments (this is the life sciences equivalent of putting "might also have some possible military applications" at the end of an engineering paper).

Also, pharmaceutical industries have the need to patent the crap out of anything remotely interesting, and think about actual uses later on, just to be sure to be able to secure potential revenue if the substance turns out to be successful. (This also leads to a patent minefield, where some substance aren't investigated much, because they are patent covered by company AbCorp, which doesn't study at all the diseases which might benefit from it).

All this has a rather negative result both on the scientific community :
- too much hype which ends up making a bad signal-to-noise ratio in the litterature, and difficult to mine the litterature for leads, inspiration or good data upon which to build.

and on the general population:
- because a lot of hype in the media turns out to be fluke or overrated or simply not further researched, they tend to think that most science is based on wrongs.
In practice, one might be right to be sceptic about a single paper making wild predictions out of 10 samples, but on the other hand, when there has been absolutely massive amount of research all bringing the same conclusions, there's no need in being sceptical (like evolution...)

Scientist communication skills often lacking.

[jweller13]

The media (tv news AND journalists) have a large degree of fault in the public's distrust in science. False balance, equating correlation with causation (they love using the phrase "scientists have found a link between x and y), and basic lack of fact checking propagates incorrect information and sometimes outright lies. Also most scientist seem to horrible communicating skills when communicating to the media and the public.

Only deniers say that.

[Anonymous Coward]

"When pro-AGW folks scream that we are all going to be underwater in 20 years" HAS NEVER HAPPENED.

The only time is there when you proclaimed it to then make your hate filled ignorance valid.

Don't read that article

[ThatsNotPudding]

I heard about a study finding it causes cancer.

That is how proper science is done

[Sycraft-fu]

And Feynman said as much in his famous "Cargo Cult Science" speech, which I encourage everyone to go read.

The problem is a lot of scientists DON'T do science that way these days. Bad science, "positive bias" as this article calls it, was rampant in the behavioural sciences when I was studying them. I basically never read a paper where they falsified their theory, or where they said things were inconclusive. They always found a way to hammer the results in to support of their theory, and none of them were ever willing to give me raw data (admittedly I only asked a few, but still).

So while journalists are part of the problem, some scientists are another. They can't ever have their theory be wrong, so they'll do whatever they can to "prove" it. Rather than trying to be the ultimate skeptic and work to find evidence to falsify it, they discard any of that and try to show how it might be true.

Re:That is how proper science is done

[buybuydandavis]

and none of them were ever willing to give me raw data (admittedly I only asked a few, but still).

I'd say that's the key. You should be expected to publish all data and lab books online. No excuse not to do this.

This is not a flaw in the system

[wen1454]

This is not a flaw in the system. This is a virtue of the system. Nobody wants to read 100s of papers about how bananas do not cause cancer. No scientist (at least no scientist that I know) believes that a paper proves anything. The paper is a finding worth sharing which must then be confirmed repeatedly by other scientists to be accepted as a fact.

Social studies

[mkdx]

Sadly, results shaping is an old problem in social fields : survey questions, selecting the sample, the size and the diversity of the sample, the preset study direction and ignoring factors not in line with that direction.

NEWS FLASH

[spongman]

Newton was wrong!

where would be to day if we'd believed him !?

interesting yourself...

[Chirs]

I interpreted it as eating more than you grow...as in growing plants/animals and harvesting them.

more fair towards Fox than I would be

[Chirs]

My problem with Fox is not that it reports from the right, but that it often reports a mindblowingly slanted and outright false version of the news. They've also been known to go out of their way to *create* news, and then cover it as though they had nothing to do with it.

For instance, they've had an "entertainment" show on the channel say something outrageous and inflammatory, and then what is supposedly a straight news show reported that "people are saying...." without mentioning that the only people saying it were other people on Fox.

Re:more fair towards Fox than I would be

[Anonymous Coward]

The thing that amuses me about Fox is some of the "mistakes" they make. How many times when some Republican politician has done something bad do they put a (D) after their name when they report on it then correct it later. I've heard of at least 3 occasions that has happened. Last year when they were reporting on the protests against Scott Walker in Wisconsin they showed some footage of a union protest that wasn't very complimentary. Only problem was there were palm trees in the background. Really! Palm trees in Wisconsin in February?

Publication bias

[Anonymous Coward]

I'm a transplant surgeon, and (sometimes) researcher. I can tell you that there is a strong positive bias in most clinical articles in medicine. Basically, it is much more noteworthy to read of a clinically significant advance of some sort than to read that a certain treatment doesn't work, or doesn't work better than previous treatments.

Essentially, academic clinicians tend to report good results. A clinical series with outstanding results lends prestige the researcher and his/her institution, and is also more likely to be accepted for presentation or publication. So, if there is a particular surgical technique being developed, it is much more likely to be published if the clinical series has good outcomes, even if the good outcomes were partly due to chance. There doesn't have to be any dishonesty or misrepresentation involved. Of course, there are also rewards of prestige and promotion that can come with "important advances", so plenty of basically decent researchers fall into the trap of trying to prove that something works, as opposed to impartially testing whether something works. Also, leaving self-promotion out of it, in medicine it is completely understandable to hope for a positive result, because a genuine advance may save lives or cure diseases.

Computer security research has this problem.

[drwho]

I do computer security research for a living Because of the general lack of scientific rigor that accompanies this field, no one is interested in results which do not produce some newsworthy result. A well-designed study which shows the lack of a security problem, where there is a good reason to suspect there may be one, is not considered valuable by anyone other than the manufacturer of the product.

There is also a lack of interest in problems which are not 'interesting'. If an attack methodology is only successful in one in a thousand cases, it is not sufficiently interesting to get attention because it is not easy to show. If you can't craft an easy plugin for MetaSploit, it is as though the problem doesn't exist.

error estimation is complex

[Chirs]

I'd quibble with your thermometer accuracy statement. The wikipedia thermometer page says, "According to British Standards, correctly calibrated, used and maintained liquid-in-glass thermometers can achieve a measurement uncertainty of...±0.05 C up to 200 or down to 40 C".

Also, you need to make a distinction between significant figures and decimal places. If I measure a temperature of 11 degrees accurate to 1 degree, that is two significant figures. If I need to divide the temperature by 10, I get a result of 1.1, which is still two significant figures. In any case, "significant figures out" == "significant figures in" is only an estimation. To be rigorous, proper calculus error estimation should be used to determine the error bounds on the output of a calculation

Statistics also comes into play. If I take a thousand samples and 998 of them are 11, one is 10, and one is 12, I can state a margin of error of much less than 1 degree at a 95% confidence level.

Re:error estimation is complex

[Karmashock]

Given that the purpose of the termmometers is to determine local weather conditions as relevant to humans and agriculture... why would anyone calibrate their equipment to that extent?

I'm not arguing that the equipment can't technically do it. After all, the Egyptians built the Pyramids remarkably well given their primitive tools. However, why would I care if I put a given sensor next to an air conditioner out vent? It's not going to change the reading enough to matter to anyone in the area. No one was collecting this data with this sort of study in mind.

This is why in their own data set they throw out up upwards of 70 percent of the data. The data is filtered to remove outliers. However, there is no way to know why a given reading was or was not an outlier. Possibly it was a bad reading. But possibly it was a good reading and the other readings are bad. And once you've determined what is and is not acceptable as n answer in the data, you're now predetermining the output.

This is why the raw data is so important and why it's so important to put the raw data side by side with the filtered data and then demostrate methodically how A is converted to B.

Possibly I'm full of crap. I'm not stupid. And to the best of my knowledge, I am not lying to you. So if I am confused, know that you're talking to an otherwise intelligent and well meaning person that is honestly trying to evaluate a complex subject that has regrettably been biased by partisan politics.

I hate the GW subject because it's both scientific and political. And it's also much more complex then creationism for example. So it's impossible to be totally sure unless you're an expert or you take a blind leap of faith somewhere.

I don't have faith for either side. I don't know if either side is right.

But the skeptics at least invite discussion and the warmists insult me when I ask questions... and that at the very least annoys me. Emotionally, that inclines me to be sympathetic to the side that doesn't seem to open every discussion with an insult. But science isn't a popularity contest. So you can be an a-hole and right. I know that. It simply becomes difficult to take people seriously when they resort to that sort of rhetoric so freely. In my experience it tends to be an intentional mask for something else. As if they offending people to stop a discussion because if it continues there will be problems.

I don't know if that's why such people are so frequently rude. That has just been my experience throughout my life in many different issues.

I'm being upfront about my bias here... I also don't want GW to be true because it is exceptionally inconvenient. But if it is, then that is reality. And if that is reality, then we should come up with real world solutions for it. The ideologues on both sides can go f' themselves. When it comes to real problems the focus has to be fixing it first and arguing about who's pet philosophy is right later.

Re:error estimation is complex

[Karmashock]

thermometers
demonstrate

sorry for the typos...

Belief in religion = positive bias

[Anonymous Coward]

Belief in religion - specifically with respect to creationism, is a very good example of positive bias that exists in the scientific community. I am in a high ranking biological science graduate program and am astonished at the sheer number of my peers that do not believe in evolution. And this lack in belief is due to the fact that they are biased towards believing in creationism which has absolutely no quality scientific data to support it. When I discovered their bias, it made me wonder how objective they can truly be about the data they collect.

music vs. movies

[KingAlanI]

I figure movie piracy is different from music piracy, since big theater screens provide a different experience (whereas an official CD isn't that much different from a downloaded copy)

You're wrong

[ifwm]

Economics is not a science

Wrong. http://en.wikipedia.org/wiki/Economics

"Economics is the social science..."

Don't like wiki? http://dictionary.reference.com/browse/economics?s=t

economics [ek-uh-nom-iks, ee-kuh-] Show IPA noun 1. ( used with a singular verb ) the science

So you're completely, irrefutably wrong on that one. You can dislike it all you want, it changes nothing. And medicine ISN'T science? What? http://en.wikipedia.org/wiki/Medicine

Medicine is the field of applied science

You're wrong. That you think your attempt at pedantry had merit doesn't make you any less obviously and irrefutably wrong.

Perhaps the solution is to educate people as to what science is

As long as it's not you doing the educating, since you're completely wrong on what is and is not science. Educator, educate thymotherfuckingself.

Update your definition

[Roger W Moore]

Wrong. http://en.wikipedia.org/wiki/Economics [wikipedia.org]

Only if you take the original latin meaning of the word meaning "knowledge". Since you trust Wikipedia I'm surprised that you did not look up science itself, and I quote from the article:

In modern use, "science" more often refers to a way of pursuing knowledge, not only the knowledge itself. It is "often treated as synonymous with ‘natural and physical science’...

Clearly this is not economics or indeed any of the social sciences where social critique and symbolic interpretation are acceptable tools to use - something that is an anathema to what the vast majority of people would undersand when you say "science". Medicine is closer to but, in case you are unaware of what natural science is I quote again from the relevant Wikipedia article:

The natural sciences are branches of science that seek to elucidate the rules that govern the natural world by using scientific methods.

Medicine applies science, so yes technically using the ancient definition of the word it is a "science" but using the definition of the word that most people (though clearly not you) use it is not science. As an example: if a doctor studies the effects of Magic Pill A on patients with a disease and finds that it cures them then he is a huge success medically because the aim of medicine is to cure people. Scientifically he would be far less of a success because he does not know why Magic Pill A cures people.

The scientist is interested in understanding how and why things behave the way they do. Some medics are indeed scientists because in order to develop the best way to cure people they want to understand how something works but in many cases the focus is on the result rather than the understanding which is not science in the modern, natural science, meaning of the word.

That you think your attempt at pedantry

Who is the pedant? The person using the modern, generally accepted interpretation of the word or someone insisting on the ancient dictionary definition? My apologies if I inadvertently caused the modern world to intrude on you but I had not realized that Slashdot posts were available in ancient Greece....still perhaps you have managed to learn something!

Re:

[Anonymous Coward]

Well... was it accidental, sloppy, or intentional? :)

Science is not as trustworthy as most people think

[Paul Dubuc]

Science journalists William Broad and Nicholas Wade were writing about this back in 1982. See their book, Betrayers of the Truth: Fraud and Deceit in the Halls of Science . Science is a human enterprise. We have always trusted it more than we should. Better that our romantic notions about the objective standards of science be exposed for what they are than to continue believing them.

Positive bias

[ToddInSF]

is just code for we know most published studies are fraudulent, and when the general public realizes just how low we've stooped, there WILL be hell to pay.

Belief in things you do not understand IS superstition, and "science" is the new religion for most people who like to think of themselves as being so fucking brilliant without actually doing any work.

America defines science

[Anonymous Coward]

America science is religion and you berrer not question that or we'll fire you from your teaching position

Reproducibility is not easy

[Anonymous Coward]

Just because an experiment cannot be reproduced by one group does not mean the original experiment was wrong.

Many experimental techniques require years to get right and the subtleties are not going to make it into a scientific paper , even if the experimenters are aware that there is a subtlety. The experimental techniques are often derived from trial and error, so a subtlety may not even be known.

In many cases if you can't reproduce something in your own lab you will have to spend some time at another lab to learn from the horses mouth.

Unless the original article has addressed these issues it is not saying anything useful.

I experienced this

[WOOFYGOOFY]

OK I experienced this as an undergrad at UCSD. Our prof (who shall remain nameless) "let it be known" that our undergrad experiments would be graded better if the results contradicted the null hypothesis (no effect). It's very subtle and informal, it's not like they stand in front of the class and say it, but someone gets told and by the time the papers are turned in, everyone knows and has complied.

Research universities are notoriously political and cutthroat; even not complying could send the inadvertent signal that you think you're morally better than your TA, who did the same thing as an undergrad and everyone knows you never want to offend your TA in any way shape form or size; they'll end your career, at least at UCSD.

There's a lot that's wrong with the university and the more competitive the university the truer that becomes. With a surfeit of very over-qualified candidates, careers are decided by who likes whom or more often who doesn't like whom. Everyone know this and the whole scene is despotic with the power to of life and death over careers being handed over to temperamental 23 year olds.

So now you're telling me this system is also corrupt? Oh my!

Well, don't despair, we can clean this sewer right up.

1) anonymous grading where the the grader doesn't know whose paper / test / etc they are grading.

2) Enough experiments randomly selected for rigorous anonymous, independent confirmation so as to make any cheater properly nervous with a three strikes and your out policy (or such like) by a body composed for just this task.

Rolling back of the excessive metricization of performance. They essentially "grade" you as a prof by how many times you're cited and how many grants you pull down for the university. In many cases, the university acts as your pimp, taking their cut or what you get. How do you pull down grants? By being a golden researcher- one who consistently makes new discoveries, and that has come to mean positive results even though a negative result can bear just as much new information as a positive one if the experiment is aimed in the right place.

Tenure and grant money are decided by who gets positive results, even though a negative result can bear just as much new information as a positive one if the experiment is aimed in the right place.

An institution's character can largely come to be dominated by how many sociopaths it let through its doors and into positions of power. The dishonesty of these personalities can have enormous influence over every aspect of everyone's else's life and survival strategy. Then that attitude becomes institutionalized, meaning people who came up through that system and won by playing it reanimate during their own careers. Right now, the university culture, at least where I went, is an open running sewer.

It' s almost not worth fixing since the university as we know it will be dead within 15 years thanks to super-inflationary tuition models, layers and layers of overpaid bureaucrats thrashing around looking for something to get busy with in order to seem indispensable, no bankruptcy stipulations on quarter million dollar student loans and the creation of low cost / no cost alternatives for the vast majority of majors.

It's really too bad since the collective efforts of research university system has essentially saved civilization a few times now and been the only source of truly independent knowledge free from mundane political pressures and as such effectively kept progress technological and civic progress going for a least a century now. Such a thing must continue to exist in some capacity and it must be funded through public measures (or it's not independent.) Either that or society is going to extinguish itself.