Many Surveys, About One In Five, May Contain Fraudulent Data (sciencemag.org)

sciencehabit writes: How often do people conducting surveys simply fabricate some or all of the data? Several high-profile cases of fraud over the past few years have shone a spotlight on that question, but the full scope of the problem has remained unknown. [Tuesday], at a meeting in Washington, D.C., a pair of well-known researchers presented a statistical test for detecting fabricated data in survey answers. When they applied it to more than 1000 public data sets from international surveys, a worrying picture emerged: About one in five of the surveys failed, indicating a high likelihood of fabricated data.

Comments

Self referential?

[goombah99]

23.7 % of statistical analyses make up their statistics.

One in 5 surveys incompetently fraudulent

[Diss Champ]

It's not that only 1 in 5 surveys may contain fraudulent data, it is that the fraud is only incompetent enough to be caught by this method in 1 in 5 surveys.

I'm not actually surprised ...

[gstoddart]

Like it or not, a lot of public opinion polls are paid for by people who want to support a specific point.

Public opinion polls these days are as much PR and marketing as anything else.

Honestly, Pew makes money doing this stuff; honest player or not, they have a vested interest in keeping up the belief that their stuff is honest, unbiased, and accurate.

But I'm entirely willing to believe opinion polls are carefully crafted, or sneakily tweaked, to arrive at the conclusions they've been commissioned to a arrive at.

Re:I'm not actually surprised ...

[gstoddart]

Right, like the Gartner Magic Quadrant shit that most C-suite people drool over.

Or bond ratings agencies.

I suspect most people, except the people who cite those things, have long since assumed they're full of shit and the conclusions are paid for.

Why would you assume it's honest and objective information? Someone has to make money off it.

And if you don't like that, start your own foundation or think tank, and have them publish stuff to your liking.

Sorry, it's all PR and marketing. It sure as hell aint facts or accurate predictons.

Re:Gee, you don't suppose respondents lie?

[EndlessNameless]

Your response will most likely be washed out by a sea of honest responses.

Most participants respond to the best of their ability---although it cannot be assumed they are always correct. Respondent error, even about themselves, is more common than outright deception.

Researchers are aware that participants lie due to self-deception, social desirability, deliberate sabotage, and other reasons. Surveys often incorporate measures to detect deception.

If you answer in a nonsensical fashion, the worst you'll do is reduce the likelihood of the results being significant---assuming your results are contrary to the hypothesis and are not eliminated for being invalid or an outlier. It's not like the entire survey method is going away; its weaknesses are well-known, and it is generally used as a last resort.

Note that this behavior can only hurt honest scientists. If someone is manipulating data, they will purge your result or fabricate enough desirable results that your behavior is irrelevant. You are literally helping no one.