Slashdot Mirror
SXSW: Nate Silver Discusses Data Bias, the Strangeness of Fame
Nerval's Lobster writes "Nate Silver feels a little odd about his fame. That's not to say that he hasn't worked to get to his enviable position. Thanks to his savvy with predictive models, and the huge readership platform provided by The New York Times hosting his FiveThirtyEight blog, he managed to forecast the most recent presidential election results in all 50 states. His accuracy transformed him into a rare breed: a statistician with a household name. But onstage at this year's SXSW conference, Silver termed his fame 'strange' and 'out of proportion,' and described his model as little more than averaging the state and national polls, spiced a bit with his algorithms. "It bothered me that this was such a big deal," he told the audience. In politics, he added, most of the statistical analysis being conducted simply isn't good, which lets someone like him stand out; same as in baseball, where he made his start in predictive modeling. In fields with better analytics, the competition for someone like him would be much fiercer. He also talked about, despite a flood of data (and the tools to analyze it) in the modern world, we still face huge problems when it comes to actually understanding and using that data. 'You have a gap between what we think we know and what we really know,' he said. 'We tend to be oversensitive to random fluctuations in the data and mistake the fluctuations for real relationships.'"
I think Silver stands out because unlike too many modern American politicians, he is interested in the facts, and not what bullshit he can use the data to support.
So it's not so much that he's done a fantastic job figuring all this out, it's just that he's fucking honest about the results unlike a certain perpetually-deluded political party I'm sick of naming.
One day I feel I'm ahead of the wheel / the next it's rolling over me / I can get back on / I can get back on
Since when did South by Southwest become a tech conference? I find it interesting that all of these technology gurus are talking at a music festival. What's next, Ballmer speaking at Bonnaro?
I keep seeing these /. articles about SXSW, but where the fuck are the videos?
If you don't have a link then stop cockteasing us ffs.
Very, very little in this world actually happens because the data suggests its a good idea. People make decisions based on their comfort level, tradition, who their friends are, etc. Suggesting that we should listen to the data disempowers the powerful. It's 2013, and the principles of evidence based medicine were only developed 20 years ago, and are still not widely used in practice. We're going to have to wait centuries before evidence based public policy becomes the norm.
Give me Classic Slashdot or give me death!
Come back with your praise after four or five more Presidential elections.
If he maintains his accuracy across multiple elections with much different dynamics, then he's on to something.
Since when did South by Southwest become a tech conference?
Apparently sometime around 1995 though that part of the festival doesn't appear to have come into prominence until about 2005 or so.
We're going to have to wait centuries before evidence based public policy becomes the norm.
I think you are being optimistic. Very optimistic.
Deadlocking them produces the best feasible outcome.
Except on the periodic occasion when we need them to do actually something. You know, like not endlessly raise the national debt because they want to promise everything but don't want to have to tell the voters they have to actually pay for it someday.
Silver's rise to prominence most closely mirrors Google's. Find a good model and apply it to an area that has been underserved.
When they write the book on the history of the Information Age, it will be about how we learned to leverage analytics for the common good.
Yeah it had nothing to do with the Republican party's platform...*eye roll*
Both sides are not equally wrong, but that doesn't mean both sides are right. The Republicans are wrong 99% of the time. The Democrats are wrong 95% of the time. Why can't we field a candidate who's right even half the time?
Because "right" is for better or worse often a matter of opinion. There is no single objectively right answer for many questions. Think abortion or gun control. Lots of opinions on both sides but there is never going to be a single "right" answer. At best there might be a consensus but probably never a unanimous one. Even for questions where a single objectively right answer may theoretically exist, there often is insufficient data to figure out what that answer is. (for example what is the optimal tax rate)
Some guy uses an AR-variant to shoot some kids, and people are screaming for a ban on such rifles to make the country supposedly much safer.
Two problems: One, such mass murders are an extremely small percentage of total murders, so even if you could stop them with a law, the statistical effect on murder would be negligible. Two, murders by rifle are a small percentage of total gun injuries/murders, and "assault weapons" make up a percentage of those, so even if you could remove every one from civilian hands, the statistical effect on gun injuries/murders would be negligible.
Even with magazine size limits, the vast majority of gun injuries/murders involve the firing of fewer than ten rounds. Completely eliminating magazines, not just banning future sales, of over ten rounds would stop an insignificant number of gun injuries/murders, if any.
Legislating on emotion rather than fact.
Some people think a tiger who kills a human is evil; it is in it's nature to kill it's food. The human animal is still just an animal and does natural things lacking any conscious rational but unlike the tiger, it has the brain power to rationalize its instincts into an illusion of free thinking and therefore believe it is unlike other animals. This is extremely hard for the human ego to accept.
Politicians succeed by tribalism; not reason and not logic. To some degree they must reflect the populace; even a dictator has limits and must bend to expectations.
What is evil is how to make generally good people collectively manifest "evil" deeds. A mental hack which makes somebody do antisocial things is evil - I use the word make because of the above statement. Free will is not as strong as people BELIEVE it is. Now if you don't have your parent's religion, politics, tastes... maybe then you are in a position to argue otherwise.
Democracy Now! - uncensored, anti-establishment news
I'm amazed. There are still Republicans stupid enough to believe that kind of nonsense. I pity you.
The world's burning. Moped Jesus spotted on I50. Details at 11.
The Ds think they don't have a spending problem. The Rs think only the Ds have a spending problem.
It's not solely a problem with either the Ds or the Rs really. It's a problem with the voters who elect them. Their disagreements usually are just a symptom of the problem. WE are the ones who demand all these services (medicare, defense, etc) but WE are the ones who vote people out of office who dare to suggest it will cost something and that we might have to pay taxes for them. WE are the the ones who refuse to acknowledge that we might not actually need 11 aircraft carrier battle groups or perhaps we might be ok with a bit less Medicare. Our leaders are to an alarming degree a reflection of our own dysfunction. It's easy to blame them but collectively if we want to point fingers the mirror is a good place to start.
Cathy O'Neil (Mathbabe) offers a well-argued criticism of Nate Silver when he stepped beyond his area of expertise in his recent, popular book, '"The Signal and the Noise: Why so many predictions fail – but some don’t"
http://mathbabe.org/2012/12/20/nate-silver-confuses-cause-and-effect-ends-up-defending-corruption/
According to Ms. O'Neil, Mr. Silver fails to recognize situations where bad models are deliberately used to game that system.
When will his 15 minutes run out?
I am becoming gerund, destroyer of verbs.
The party that is behind will never say "we've lost, don't bother voting" lest they lose their base as well.
The party that is ahead will never say "we'll cream them, don't bother voting", lest they lose.
Most of the media is not going to say "most of our ads and news coverage are irrelevant, so tune out till the election is over".
The appearance of uncertainty is in all the player's interests. Not sure he'll ever have much competition
people want to hear, so that you can better pander to them. LM324N
It became broken when FDR fucked the constitution and started transfer payments.
Ummm, sure... whatever. I'll just nod my head and pretend that makes any sense at all even though it doesn't.
Now that a majority of voters don't pay significant taxes we are permanently fucked.
Where did you get a ridiculous notion like that? Your name isn't Mitt Romney by an chance is it? Most people pay quite a lot of taxes. Not enough to cover Social Security, Defense and Medicare but they pay about 2-3 Trillion per year to the federal government alone. Most pay some combination of Federal Income tax, FICA (Social security + Medicare), State income tax, property tax, vehicle registration, gas tax, and quite a few more.
Does anyone know where we can see this talk online?