Slashdot Mirror
All of Nate Silver's State-Level Polling Predictions Proved True
kkleiner writes "For the last few months, the political pundit class has been at war with NYT/FiveThirtyEight blogger Nate Silver. Joe Scarborough of MSNBC called him a "joke," while an op-ed in the LA Times accused him of running a "numbers racket." But last night, Silver triumphed: every one of his state-level presidential predictions proved true. "
http://www.xkcd.org/1131/
As Nate Silver pointed out, many times, over the course of the campaign, predicting what will happen one day before the election is easy. Very easy. Most everyone gets that right.
Predicting what will happen in June is hard. And much more interesting.
The only ones who believed the race was a 'virtual tie' were those who had gains to be had by it being so, namely the media.
You can't really prove a probability wrong (unless it's 0% or 100%). While his most likely outcomes played out, it doesn't mean that he would have been wrong if a few of them hadn't.
Another possibility is that the Founding Fathers understood just what kind of dumbshits most of "the people" are, and built in some safeguards to allow civilization to survive the inevitable popular vote to shut it down.
Why does Nate get so much attention, when other sites like electionprojection.com and electoral-vote.com do a similar service, are open on their methods and have had almost perfect results for the past two elections. This past election, those two sites only missed on Florida and that one was truly too close to be 90% confident on one way or another.
See my journal for slashdot ID's by year. Mine created in 2005. http://slashdot.org/journal/289875/slashdot-ids-by-year
...that's not how statistics work either. A result does not alter the underlying probabilities.
Too bad the electoral system in the USA is a joke and doesn't represent the vote of the people.
So you wanted Al Gore to be president in 2000?
Sig. Sig. Sputnik
While the possibility of that is certainly true, it is false for this recent presidential election. Not all districts have reported in, but the most recent numbers show that Obama is ahead in the popular vote by a hair under 2.9 million votes.
http://www.google.com/elections/ed/us/results
I wish people would stop blaming the electoral college. The system is fine. It is the method in which the individual states assign the votes that is the problem. Florida in 2000 wouldn't have been such a big deal if they had distributed the electoral votes by district, rather than winner take all.
Win California by one vote? You get all 55 electoral votes! How stupid is that?
Pundits are there to draw people to a news organization, not provide accurate information. How many people would tune into the election if they said "Obama's got this one in the bag" (which we've known ever since Romney was nominated). Of course they will say you don't know who's going to win! Otherwise no one will watch their show.
As opposed to the alternative? Jesus Fucking Christ, yes.
There is no memory shortage. yes I have heard of XFCE. Go away.
It does tend to over-represent smaller states. This was somewhat intended, but as one analyst on Tuesday put it, "Should an electoral vote in Wyoming, that represents 130,000 people be equal to an electoral vote in California that represents over 600,000 people."
David Brooks gave an interesting response, he said that the electoral system forces the candidates to make an effort to play somewhat to the middle. Without the electoral system, Barack Obama would have campaigned heavily in California to get the liberal count up there. Mitt Romney would have campaigned heavily in Texas to instigate the conservative vote. The result was that they needed to go to places that weren't exactly on their side and try to convince them. They were forced to answer questions that both sides wanted to hear an answer to, rather than just their base. (I dare say most people haven't been pushing for real answers, but that's another issue altogether)
The result of the vote is that he won. The result of the electoral vote is that he won.
This is pretty much the textbook definition of whining about something that doesn't matter in the slightest.
-- "So they told me that using the download page to download something was not something they anticipated." - Bill Gates
This actually shows that Silver is poorly calibrated. if he were accurately calibrated, 80% of his 80%-confidence predictions would come true, 50% of his 50%-confidence predictions would come true, etc. But 100% of his >50%-confidence predictions came true. In the future, he should be more sure of his predictions.
Not necessarily. Most of the uncertainty in his predictions was due to the conditional probability of systematic bias in likely voter models. For example, Gallup was showing much better results for Romney and the Rac... err... Republicans across the board, which was probably due to how they screened people who responded.
Systematic error shows up as a conditional probability, so you are lumping together completely disjoint realities into your final result. In terms of discrete conditional probabilites, imagine that based on historical data you have three equally likley possible conditions: 5% Democratic bias, no bias, and 5% Republican bias. You run your simulations with each of these three biases, and you get a result that says senators D0, D1, D2, D3, D4, D5, D6, D7, D8 and D9 are all 80% likely to win. But 99% of that 20% chance they'll lose comes from the condition where there's a 5% Republican bias.
But remember: those three conditions describe completely disjoint realities. They are not sampling error, but statements of ignorance about the actual state of the world.
Now the world really is just one way (it may be ambiguous relative to some human categorization, but then that ambiguity is just part of the one unique way the world realy is.) So only one of the three conditions are true. If it happens that the no-bias case is the way the world really is, then 100% of those 80% chances will come true.
That said, in future elections Bayesian predictions of the kind Silver and everyone else in this space are making will lower the conditional probabilities of bias, because this election demonstrated good low-bias results, but so long as the ultimate uncertainty is dominated by the systematic error, Bayesian predictors will tend to appear either uncannily accurate or dismayingly inaccurate.
However, averaged over many, many election cycles (18 or more) you would expect to get statisics such that 80% of the 80% calls are correct, and so on. But within individual elections that use fixed likely-voter models that won't be the case.
Conditional probabilities are one of the most difficult things for humans to understand (the Monty Hall problem is a classic case where all the confusion comes from treating a conditional probability as if it was a total probability) so it's worth practicing the art of thinking carefully about these things, and the odds are still good I've said at least one confusing or incorrect thing in the above.
Blasphemy is a human right. Blasphemophobia kills.
The Electoral College system was never meant to represent "the will of the people". The House of Representatives is supposed to represent the will of the people. The presidency and the Senate had entirely different purposes and mandates.
The world's burning. Moped Jesus spotted on I50. Details at 11.
This could be framed another way; that is, "swing states" are the only ones the candidates care about. The rest of us are taken for granted.
That's only true for dusty old fuddy-duddy classical Newtonian statistics. These days quantum statistics is where it's at. Everyone knows that reading poll results can fundamentally alter data, even data that wasn't tabulated for that specific poll. (Something to do with bell curve entanglement and the spin/charm of pundits - gets kind of technical at that point.) Why just yesterday I glanced at a USA Today that someone had left behind on the subway, saw the usual 57-color pie chart about whether readers thought giraffes tasted great or were less filling, and next thing I knew the first president of the USA was George Washington instead of Herbert Whistlefjord.
A marriage is always made up of two people who are prepared to swear that only the other one snores.
It's as stupid as going with votes by district.
Win each district by one vote? You get all the votes, and hence all the states and the election! How stupid is that?
This needs to be stated as clearly as possible, because it is one side of the greatest flaw in first-to-the-finish voting.
The more districts where the votes are aggregated, the less of the popular vote you need to win.
The more (plausible) options there are on a vote, the less of the popular vote you need to win.
Take a second and read that again. Now I'll explain
First, a district example.
Let's say you have only 1 district, two candidates, and a million people in this imaginary nation. You get 500,001 votes, you're the new president. It's pretty straightforward, and you clearly have the popular vote, if only barely.
Now, let's say we still have 1 million people and two candidates, but we have 10 districts of 100,000 each. To win a district, you need 50,001 votes. To become president, you need 6 districts. Do the math and you only need 300,006 votes to win. While you may have the popular vote, it's not necessary to win.
Now, the candidate example.
Let's say you have only 1 district, two candidates, and a million people in this imaginary nation. You get 500,001 votes, you're the new president. It's pretty straightforward, and you clearly have the popular vote, if only barely. (Yes, it's exactly the same as above.)
Now, let's ay we still have 1 district and one million people, but we have 10 candidates. Given an unrealistically tight race, you could win with only 100,001 votes. Practically, you'd need more than that. How much more? Well, that depends on how close the candidates are in popularity.
So there you have it. If you think popular vote is the most reasonable way to choose the president (which is NOT the model the US uses), you want fewer districts and candidates, or you want to stop using first-to-the-finish voting.
Also, if you combine the two elements above, many districts and many candidates, the percentage of the popular vote required to become president becomes even lower. For a handy real-world example, see Canada.
For the conspiracy theory lovers out there, a nation in the state listed above with a lax immigration policy is a ripe target for a peaceful invasion. Immigrate enough people to become a third of the population in a third of the districts, wait until they have voting rights, have them vote for who they want to be in charge, and the government is yours. This may be easier if you make a new party for this purpose as it dilutes the voter power of those who aren't voting in concert to overthrow the current government. That's right, 4 million people, good planning, dedication, and 5 to 10 years, and a nation the size of Canada could have a peaceful revolution. Conspiracists, you may start to spin your tinfoil hats!
Sure I'm paranoid, but am I paranoid enough?
He was way too liberal. Kept talking about helping the poor and ran on a very anti-war platform.
There is no memory shortage. yes I have heard of XFCE. Go away.
David Brooks gave an interesting response, he said that the electoral system forces the candidates to make an effort to play somewhat to the middle.
David Brooks is completely wrong about this. For instance, I'm in Ohio, the swing state that everyone was focused on for months. Here are some of the appeals I got in my mailbox and on billboards:
- Obama is actually the son of a convicted drug dealer and a porn actress, not a Kenyan. And he's been lying about his name the whole time.
- Voter fraud is a felony (but only in neighborhoods that are mostly poor and black).
- Any kind of increase in taxes will cause the economy to collapse.
- Social Security should be abolished.
Tell me exactly how that forces candidates to appeal to the center.
I am officially gone from