Slashdot Mirror
A Crowdsourcing Project To Make Predictions More Precise
databuff writes "Predictions are critical to modern life. Police predict where and when crimes are most likely to take place, banks predict which loan applicants are most likely to default, and hotels forecast seasonal demand to set room rates. A new project called Kaggle facilitates better predictions by providing a platform for forecasting competitions. The platform allows organizations to post their data and have it scrutinized by the world's best statisticians. It will offer a robust rating system, so it's easy to identify those with a proven track record. Organizations can choose either to follow the experts, or to follow the consensus of the crowd — which, according to New Yorker columnist James Surowiecki, is likely to be more accurate than the vast majority of individual predictions. The power of a pool of predictions was demonstrated by the Netflix Prize, a $1m data-prediction competition, which was won by a team of teams that combined 700 models. Kaggle's first competition is underway, and it is accessing the 'wisdom of crowds' to predict the winner of this May's Eurovision Song Contest." Understandably, participation requires registration.
I predict a first post!!
"Past prediction is not an indicator of future performance."
While we're at it, why don't we let everyone pool together their lottery number predictions?
Behold my amazing precognitive abilities, as I look into the future of crime and predict:
Most crime will take place in that part of town with the highest concentration of check-cashing and liquor stores, between 5 pm and 3 am. Most of the alleged defendents will not be college educated and will have prior criminal records. Very few actual crime arrests will involve white collar fraud or the elaborate, diabolically-planned crimes that make up the bulk of criminal activity shown in popular TV shows, comic books, and movies. The vast majority of accused criminals will be, in fact, guilty of the crime they are accused of. Very few criminals will be represented by a crusading public defender with the resources to conduct a thorough analysis of their case and order elaborate DNA tests to prove their innocence in a last-minute dramatic countroom reveal.
SJW: Someone who has run out of real oppression, and has to fake it.
Where the bad guys get to the guy who has the best predicted performance, kidnap his loved one and make him commit a bank robbery or something.
Mr. Marks, by mandate of the District of Columbia Precrime Division, I'm placing you under arrest for the future murder of Sarah Marks and Donald Dubin that was to take place today, April 22 at 0800 hours and four minutes.
Police predict where and when crimes are most likely to take place,
There's going to be, ... er, a crime. A big one. Yeah, that's it. Clear over on the other side of town. Send all the cops. Right away.
Have gnu, will travel.
Crowdsourcing Project To Make Predictions More Precise
I think they used to call them "polls".
How can I believe you when you tell me what I don't want to hear?
Because if most people think something is true (or in this case, think something is going to happen a certain way), then it simply must be so.
I judt got a nre Kinesis keybiartf so please excusr ant egregiou typos.
it's just a rehash of PREDICTION MARKETS which is OLD news.
I lovde my crowdsourcing project, but will it BLEND?
Yours In Vladivostok,
Kilgore T.
If you take enough samples, with approximately the same error rate, you will get an accurate result if you average them together.
Therefore I conclude that any answer can be calculated by running: answer = (answer+rand())/2; enough times
Can they accurately predict the global tempature 1000 years in the future, but have to estimate past values, the the Global Warming people?
Who would win this election: Andrew Weiner vs Andrew Weiner's weiner.
from 1 or 1000 sources. conscience & spirit can also play roles. good luck with that.
without barely a thought, it might be worthwhile to predict that it would benefit many to move inland.
For example, what if we crowdsourced a prediction for which stocks will do well tomorrow?
There's a sort of unavoidable feedback loop when the entity responsible for prediction is also responsible for execution, even if that entity is "everyone".
The wisdom of crowds works when everyone is looking in the same area for the answer to a question with a somewhat fuzzy answer. The group average can often be better than any single expect that attempts to calculate it. However this is a poor approach when the crowd isn't even looking in the right place. Simple majority decision making would be disastrous for many of the big decisions organizations make. The pubic is massively ignorant on scientific issues and continues to be plagued by religious, corporate, and state imposed falsehoods. Freeing people from these shackles and providing full education for all could allow us to crowd source more important decisions and lead to a more efficient and just society.
------ Take away the right to say fuck and you take away the right to say fuck the government.
I think it was a short story in Analog or Asimov science fiction magazines. Someone got tired of the weather forecast being right only about half of the time and created a nation-wide betting pool for people to bet on the weather for the next few days for their area. The theory was that most people would bet on what they thought would actually occur instead of trying for long odds. In the story the forecasters eventually started subscribing to the pool because its predictions were accurate more often than theirs.
I always wanted to try this to see if it really would work. I guess someone has finally done so, just not yet with weather.
Edward Burr
Having a smoking section in a restaurant is like having a peeing section in a swimming pool.
Averaging only reduces random error giving you a more precise result. It doesn't help with systematic errors and therefore not necessarily a more accurate result.
Accuracy being closeness to the bullseye and precision being the grouping of the shots.
Deleted
First off, the idea of crowd-sourcing predictions has already been done, and it's called "Intrade". The neat thing about Intrade is that, as a stock-market system, it ads a monetary incentive to your prediction. You have to "put your money where your mouth is" so to speak and, the more money you put up, the more your prediction weighs. So, the experts in that field (or, even better, those with inside information) will buy/sell a lot more "stock" than the other people with mere hunches.
The other problem with the "wisdom of crowds" is that it has been shown to only work for things where "common sense" is generally right. It works fine for stuff like guessing the number of jelly beans in a jar, but consider, for example, the question of whether government taxation and spending go up and down together. Almost everyone would assume that they do, but it turns out that taxation and spending move almost exactly opposite to each other(*). So, this is one case where the crowd would be just as wrong as a single individual.
(*) Don't believe me? Go check the numbers. I did when I first heard this. It sounds preposterous when you first hear it. But it turns out that the numbers back it up. As far as *why* it happens; one theory is that, at any moment in time, the government either places a higher priority on reducing the deficit or on making the voters happy. If the gov't wants to reduce the deficit, it takes a two-pronged approach of raising taxes and reducing spending (probably also to help sell the tax increase to the public). On the other hand, if the gov't just wants to make the voters happy, then writes off any hopes of keeping the deficit reigned in, reduces taxes and increases spending to programs that the public likes (like public projects or defense). If anything, I will never again believe this nonsense some politicians spout about "We're going to lower taxes because it will force us to reduce spending". Bzzzzt. Wrong!
They're just legally prevented from interceding before the when. In fact, in my old home town, the police knew a certain criminal had been murdered because of the reduction in the crime rate at certain times and areas.
Only a small proportion of real crimes and criminals are not predictable.
Deleted
Do polls work so well because the people voting in the earlier polls influence the later polls?
If the predictions were shared in real-time with the people they were to predict upon, would they still have the same accuracy?
It seems to me that predicting is only useful when its use is unknown to those it's used on.
...because none of us is as dumb as all of us.
When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
prediction markets have been around for a while. nothing really new here...
This model seems to follow 'genetic programming' principles, but is flawed in many ways: (a) It assumes that most people know everything relevant to the problem under consideration - they often don't.
(b) What the model is looking for is an expert among the crowd. On average, you can find an expert among 1024 people, to predict 10 coin tosses - this is with random data having no relation to specialized wisdom.
(c) Eurovision (mentioned above) is in the rare category of scenarios that can make use of 'crowdsourcing prediction', but only because the simulation correlates to the probable reality: it's effectively a poll, where the opinions of lots of people are used to model the opinions of lots of people.
(d) Can you really assume that if someone gets it right 10 times, the same person will get it right a further 10 times? It needs to be the same specialism.
(e) You'd need to iron out the randomness by running lots of trials that will be of no use to anyone. Can this operate commercially?
Fact: No, they didn't. Supreme Court decided in favour of Bush by just one vote. What special favours do you think were going around at that time?
Personally, I regard Bush as a criminal that stole the precidency and started a war on false grounds. Afghanistan was barely OK, alas just as poor decisionmaking as the russians did.Not Iraq though, that was just lies upon lies. See Green Zone, for a recent movie about it. Basically the idea in the movie hints at that someone (CIA and Administration) HAD to know the truth, but it seems justice doesn't mean squat for the rich and powerful in USA. Someone WANTED a war in Iraq, regardless of facts, and did everything they could to fake "intelligence information". That the consequences of this murdered thousands of people and destabilized an entire country seems not to faze these people. Actually, I personally believe that was the entire idea behind the circus, including increasing the debt of USA to new scales. It's easy to manipulate people in "war" and "crisis", so why not create some more of it? Osama Bin Laden was never caught, because he was the biggest assets to such politicians.
Problem is to get proof of such criminals exploits. They're very good at hiding their tracks.
They can get away with murder, while upstanding people like Clinton gets tored down by a little fun on the side. A marital problem more than anything else, and most presidents before him have done similar injustices to their wives. But why did he get knocked out of office for this? For all we know, maybe Hillary was doing similar things on "her side". Their personal relationship is no business to us.
Good thing is that now it seems many Americans is tired of this, and want change. It's not without a price though, and it won't be easy, as many reptiles^H^H^H^H^H^H^H^Hpoliticians just do everything they can to throw dirt and be destructive rather than play along constructively.
I don't know I find these kind of opening phrases amusing/annoying, but saying "Predictions are critical to modern life" seems to imply that somehow they are more important than ever. Aren't most major religions based on "predictions" of some kind and didn't they begin a wee bit before "modern life"?
Crowd-sourcing predictions will undermine all sorts of religions, and we all know what happens when you threaten the monopoly on truth help by religion...
Thanks everyone for your comments! Sounds like many of you are skeptical that 'wisdom of crowds' can work in this setting. It'll be an interesting experiment, but I'm encouraged by the Netflix Prize case study. Out of interest, does anybody have any interesting ideas for prediction competitions? I'd love to hear from you either in the comments area or at statsbuff@gmail.com.
This is REALLY old news. 66 years ago, it was known as the DELPHI method, and it's been studied to death in the interim.
As someone who knows a little about the Netflix Prize, metalearning is not crowdsourcing. It's not a prediction market. It's none of these things. It's essentially taking a weighted average to match some prior data and then using that for new predictions. It's machine learning. It's not magic. If you wanted to draw an analogy in the real world, it would be like asking, who do you believe more when predicting changes to the climate? Some know-nothing wingnut who never went to college, but listens to conspiracy AM talk radio and creationist programing? Or a climetologist with a PhD and years of experience? Oh sure, the wingnut *might* be right, but probably not.
The real question is whether prediction markets even work? The answer is, they don't. They're at best lagging indicators of existing knowledge, and at worst completely useless.
I think eventually businessman just want to predict stock price, so they can be rich.