Slashdot Mirror
Gambling Could Reveal Which Scientific Studies Are Worth Their Salt (sciencemag.org)
sciencehabit writes with this interesting story about using prediction markets to weed out false positives in scientific experiments. From the article: "Gambling is frowned upon in many circles. But what if the gamblers are researchers betting on how each other's experiments will turn out, and the results are used to improve science itself? A group of psychologists has found that their collective gambling—with real money—predicted the outcome of attempts at replicating experimental results better than their own expert guesses. They propose that this type of gambling setup, known as a prediction market, could become part of how science gets done."
The interesting part seems to be that at least scientists use different weights between asking their opinion (such as a poll) and gambling on outcomes with real money.
WTF are all these MBA's doing in my lab?
Las Vegas casino overs will have something to say about this. But better than mere gambling, let's put these guys in the ring. Make it into a Gladiator contest. Then you'll have a market.
“He’s not deformed, he’s just drunk!”
No wonder gambling is more accurate than their predictions!
It could be worse, they could be "environmental scientists."
At least psychologists haven't built an entire community based on altering all data to support one hypothesis.
Who's money will they gamble with? Why care when your skin isn't it the game? Taxpayers obviously do not care, otherwise dozens of agencies would have been defunded already.
Let the largess and corporatocry be!
... then why not also for science?
The scientists are probably as real as the money is.
Nae king! Nae laird! Nae yurrupiean pressedent! We willna be fooled again!
if it's actually science, then you can repeat the experiment. we solved this problem, it's called the scientific method.
Anons need not reply. Questions end with a question mark.
OK! $50,000 on Haruko Obokata of Riken to get successful peers review on her stem cell research.
Cmon .... cmon ..... oh shit ... DAMN ..... sorry kids, there goes the college fund.
It's just enforcing consensus - It doesn't mean the science will be right or wrong anymore than which horse wins the race.
Yeah, the odds increase the probability of a success but that's not a guarantee.
Geez, is this what's become of "science" in the world today?!
Model it after fantasy football, you pick your team of scientists and win or lose based on how they do.
Draft kings is on the job as we speak.
The shepherds did so well protecting the flock that the sheep no longer believed that wolves existed.
We already have institutionalized team based gambling to help decide which science to pursue. It's called "grant review committees."
I see they have been having difficulty getting participants.
Good.
The shepherds did so well protecting the flock that the sheep no longer believed that wolves existed.
God/nature does not play dice, but apparently scientists do or should?
Estimating confidence in the completion date for an IT project can be hard. When I've been asked to review a troubled IT project, I've found that a very useful technique to determine how far along the project actually is -- and when it's actually likely to be done -- is to ask those involved at different levels what their "$1,000 confidence level" is -- that is, what date they would be willing to bet $1,000 out of their own pocket that the project will actually be done by. I find that even the most optimistic engineer suddenly turns cautious and starts thinking of all the things that could go wrong.
I then ask what their $10,000 confidence level is. Those answers tend to be the most accurate. ..bruce..
Bruce F. Webster (brucefwebster.com)
Yes, it'll have a high level of correctness. You bet on things which seem odd being wrong, and mostly you're right.
But you won't bet on things which seem right and reaffirm what everyone thinks. So FRAUDULENT results made to shore up initial work which everyone thought made sense won't be questioned under this system.
It's going to be systematically wrong in favor of conservatism. The fact that that pattern is usually right and wins money doesn't further the cause of science. it just reduces noise from absurd results.
Proposals are gambling. When you write a white paper at a company it goes through layers of managerial review if it would be worth the companies time to have you complete a full proposal. Then the Government review panel looks at and decides if they want to invest in that. Peer review informs on the odds foliding in many predictors of success, such as past performance.
Finally the reviews are parimutual gambling. Your odds of winning are not strictly the value of your proposal but the relative odds amongst a group of competitors.
Some drink at the fountain of knowledge. Others just gargle.
In the Palormares incident in 1966, a B-52 collided with a refueling tanker and both planes were lost. The plane was carrying 4 nuclear weapons, 3 of which were found on land (and two had their conventional explosives go off), but one fell into the Med. Dr. John P. Craven used a Bayesian Search theory to locate the lost bomb. The first step in the Bayesian Search is to formulate hypotheses for where the bomb might have gone down. Craven put a bottle of Scotch up as a prize for the person who came closest to where the bomb would actually be found, and got several hypotheses for where the bomb went down. While this isn't exactly the type of gambling discussed above, putting a real reward up clearly motivates people to give their very best guesses! Oh yeah, they found the bomb, and the actual recovery of it was fictionalized in the movie Men of Honor with Deniro and Cuba Gooding Jr.
Brawndo: It's what plants crave!
I'd like to put a grant request together that will use "research" to show only my research (purely objective) is valid and these gambling sites are charlatans!!
Yes, only scientists can be trusted and that's why I should have all the public funds ever.
Everything that does not give me money will soon be discredited !!
"Would I bet money on it and with what odds?"
A group of psychologists has found that their collective gambling—with real money—predicted the outcome of attempts at replicating experimental results better than their own expert guesses.
That might work for something like psych, where few results are reproducible - or even where few "experiments" are re-tried. However, I doubt that the real world reflects the wishes of some profs with a penchant for a flutter and too much free time.
Now if they could apply quantum mechanics to the problem, instead of gambling, there might be some interesting results.
politicians are like babies' nappies: they should both be changed regularly and for the same reasons
Perhaps you'd like to place a wager on that?
One is gambling ones reputation when making a public prediction. But the estimating a value of one public statement (correct or incorrect) is unenumerated. Money is a token for enumerating exchange. Currency doesn't have to exist in the form issued by a government entity. Stackexchange, for example, is pretty good at enumerating reputation gains and losses (and has methods for staking ones reputation gains on answers to certain questions). It's hardly surprising that enumerated exchange of knowledge is more precise than unenumerated one.
Any guest worker system is indistinguishable from indentured servitude.
The Biggest most important part of gambling is The Bluff. Someone is going to figure out how to make more money off being wrong than right and then the whole entrenched system will fubar science even worse than it is now.
Aren't the most important results, the ones that suprise us.
Supposedly everyone in the middle ages thought that heavy objects fell faster than light objects, because gravity pulled harder on them. Aristotle had said so, and because of friction, there is an obvious anecdote where you compare dropping a rock to dropping a feather -- the rock falls faster. So everyone assumed it was true, without ever testing it.
the prediction market would give us near-100% certainty of the result being true, since everyone believed it.
however, it wasn't true. a rock doesn't fall half as fast as bigger rock that weights twice as much. unless friction becomes a major factor, things all fall at the same speed.
Someone had to actually try it, and notice things were not what we expected, and then we learned something.
unless they are specifically using the prediction market to find the things that "everyone says not to bother test, so we definately need to test it in case we get a suprise". but it doesn't sound like they are proposing anything counter-intuitive in the summary.
meh, go watch mythbusters - at least someone knows how to be empirical.
Wouldn't that devalue the big breakthroughs in science? You know.. the discoveries made by thinking outside the box.
I have a feeling they're doing better science because if they bet against one of their peers, they know if they have skin in the game that they're going to be a lot more critical and call bullshit easier. When it's a dolt/dumbass that's handing out public money, he doesn't care. Let's see, followed outline, has some good looking graphs, isn't disproving Man Made Global Warming - don't care, give 'em money.