I am in complete agreement with #23208332. Your summary is an unfair exaggeration of Knuth's position; he mentioned that unit testing was rarely useful for him, in an interview that was mostly about many other topics. To make "rips on unit tests" the main headline is a wild distortion of what he actually said. For shame.
If the community decides that a page isn't notable, just label it thus and move on. There's no reason to delete the page.
The same thing goes for page locking: although there are still some extreme cases where pages need to be locked, many of the reliability problems would be mitigated by labelling recently-changed parts or frequently-changed parts of pages. Readers can then take responsibility for their own level of trust.
Both cases are about matching expectations to reality: the situation can be improved by changing the content OR by making expectations more accurate.
Range or approval voting would be a better option. They truly eliminate the spoiler effect, they are easy to implement using unaltered existing equipment, and have simple, easily understandable behaviour.
The horse race in the media is driven by all this nonsense about "momentum" and "viability" -- people are trying to get a sense from the early states who is likely to win, so they can vote for someone who has a chance. But the only reason this is necessary is because the votes are being split 7 or 8 ways. That's madness.
Think about it: suppose a party has three major candidates, one in the middle with respect to the party's base of support, one a little to a left, and one a little to the right. If each voter registered in that party gets to vote for just one candidate in the primary, the two candidates on the left and right will split most of the votes, taking them away from the candidate in the middle. The candidate in the middle, who would be preferred by the most voters in the party, gets eliminated.
The whole purpose of running a primary is to choose a single representative behind which the whole party can unite. If parties didn't run primaries, and all the candidates just ran in the general election, they would split all the votes and probably none would win (imagine a general election with 9 Democrats running against 1 Republican). But running a primary by plurality voting (each voter votes for just one candidate) just re-creates the vote-splitting problem that the primary is supposed to solve.
I can't understand why neither of the parties has decided to run their primary by approval voting (each voter votes for as many candidates as they like). That would better reveal the total support for each candidate, thus giving a better reflection of how well they will do in the general election. The first party to do this, it seems to me, would gain a significant advantage, because it would actually choose the most broadly liked candidate, the strongest candidate for the general election. What they're doing right now seems to be just shooting themselves in the foot.
Elections are not simple, much as we might like them to be so.
Keep these points in mind:
Ballots in the U. S. typically have dozens of contests -- sometimes 60, 70, or more contests. Hand counting is significantly less practical in the U. S. than in say Canada, where your ballot is just a vote for a single candidate in a single contest.
Electronic voting has real security risks. Most folks here know that already. The risks can be big.
Electronic voting has real potential advantages. The number of undervotes, overvotes, and otherwise spoiled ballots is considerable and significant. We are talking about millions of votes here. Voters really do make more errors when they vote on punchcards or on centrally scanned paper ballots -- and these errors disproportionately affect poorer and less educated voters. Precinct-based scanning prevents overvoting (voters find out immediately if their ballot is improperly marked, and can try again). DREs prevent overvoting and also have the potential to significantly reduce error rates by warning the voter of skipped contests, giving better instructions, and supporting more languages. DREs also have unique advantages for disabled voters.
It is not inconceivable that switching to central-count optical scanning could actually leave Ohio worse off than DREs, depending on your assumptions about the frequency of voter errors and the magnitude of security risks. There are many factors involved.
At the moment, my favourite is precinct-based optical scanning (paper-based, simple, with immediate feedback to voters) or paper ballots printed by a computerized voting interface (all the advantages of computerized vote entry -- IF the UI is well designed, without losing the verifiability and auditability of paper).
The problem here isn't just that Facebook is collecting private information. Any company could say "look, if you use our service, here's what we're going to collect and what we're going to do with it," make a good-faith effort to inform everybody what's about to happen and how it works, and then proceed.
The problem is that Facebook is lying about it, and doing so repeatedly.
Zuckerberg led the press and advertisers to believe that Beacon would be opt-in (it would publish only with the user's consent) but launched Beacon as an opt-out feature (it published without the user's consent).
Both the original design and the current design of Beacon announce to the user that a story is being sent to their profile. They do not present themselves as a choice; they do not ask for consent; they present themselves as a notification that something is already occurring.
Even though the new design is "opt-in", the notification has only one clearly emphasized button: "Okay". A design that offered a true choice would offer two equally clear buttons (e.g. "Publish" and "Cancel"). Again, the design is crafted to give users the impression that they have no choice.
Facebook collects information about its users' activities on other sites through Beacon despite public statements to the opposite. According to Stefan Berteau, Facebook does this even when you are logged out and even when no notification is displayed.
Facebook did not give its users reasonable advance notification that it would start publishing information about their activities on other sites. It just went ahead and did it. And Facebook is still not being upfront about the fact that it is collecting this information.
Facebook continues to refuse to let users just turn off Beacon. Instead users have to individually refuse Beacon for each partner site, and they cannot do this in advance; they can only do it at the moment a partner site is about to publish a story on Facebook. Again, they are clearly trying to maintain as many obstacles as possible for users who simply don't want this information shared.
Facebook's official response is disingenuous and insulting. The problem is not that Beacon "can be kind of confusing"; it is obviously designed to mislead. Facebook's Paul Janzer wrote:
While we know "global opt-out" seems like the easiest solution, we believe that if we provide you with full control over your information, you and your friends can get the full benefit of sharing information and connecting on Facebook.
Of course, if they really wanted to provide users "full control over [their] information" they would let users turn Beacon off.
The one-minute estimate is too low, not too high. In Canada and other countries that do hand counts, there are usually just one or two contests on the ballot. In the United States, there are typically an order of magnitude more. In Chicago in 2004 the ballot had 90 contests on it. So we're talking about more like 5-10 minutes per ballot. Hand counts would only be feasible if the number of contests were reduced or only some of the contests were counted by hand and the rest were counted by machine.
I'm no fan of unverifiable electronic voting machines, but I do have to point out that the scales are different. In Canada and other countries that do hand counts, there are usually just one or two contests on the ballot. In the United States, there are typically an order of magnitude more. In Chicago in 2004 the ballot had 90 contests on it. Hand counts would only be feasible if the number of contests were reduced or only some of the contests were counted by hand and the rest were counted by machine.
How, then, do you ensure that the code that you vetted is actually running on the machines?
The short answer: chain of custody.
Like any other piece of election equipment, the voting machines have to be physically protected from the time they are configured to the time they are used. The same holds true for mechanical voting machines, paper ballots, electronic pollbooks, and so on.
I'm not saying that current procedures for this are adequate -- far from it. Obviously if you leave the machines for unattended sleepovers, they're likely to be vulnerable to tampering. On the other hand, protection of physical equipment is relatively well understood; the risks can be identified and reduced. There is certainly a much longer history of experience in moving things securely from place to place than in recounting voter-verified paper trails.
The problem in this case is not the same as the general attestation problem, which is extremely difficult (though many people think even that is not impossible). We aren't talking about mass-distributed consumer electronic devices that people can rewire in the comfort of their homes in an effort to make them lie about the software they are running. We're talking about moving a machine from one controlled environment to another controlled environment.
If your position is that physical security can be expensive, I have no dispute with you there. If your position is that you don't trust the computing hardware to function correctly, I offer no argument -- in that case all electronic voting is infeasible, VVPATs are moot, and Punchscan is the only solution. I consider that a position worthy of consideration, though I do not take it myself. I think it is possible to trust the hardware, but I'm willing to be convinced otherwise.
The only reasonable way to do electronic voting is to define a system such that there is no way the software could manipulate the vote without being detected, no matter how malicious the software.... A voter-verifiable paper trail accomplishes this rather easily.
I don't think it's all that easy.
Based on what you're written ("The mechanism has to be free of *undetectable* bias." and "Don't you think those behaviors would be noticed?"), I suspect you are making the assumption that detectability is what matters. But that's not enough; the bias has to be actually detected, and a detected bias is okay only if election administrators can recover, and do recover, from the problem.
It appears that where you and I differ is a matter of scope. In the theoretical world, all that you need is a conceivable possibility of an error being detected in order to rule out dependence on software. But in practice, it's more complicated than that.
For example, let's talk specifically about VVPATs. Forget crashing or bad ballot presentation for the moment; I'm just talking about a machine that, some fraction of the time, prints the wrong selections on the audit trail.
Take your scenario with 60000 voters, a machine that misprints 1% of the VVPATs, and a 1% probability that each voter will competently check their VVPAT. Each of those 600 misprinted VVPATs has a 99% chance of going undetected, so the chance that all of them will be undetected is 0.99^600 = 0.0024. So the chance that at least one will be detected is around 99.76%. So far so good. (I'm not sure how you got 95% -- I'd appreciate you walking me through your math.)
But what happens when a voter thinks there's something wrong with their VVPAT? Do you shut down the entire election and start over? Do you launch an investigation of the voting system vendor? Certainly not according to current practice. How many voters have to report a problem before you re-run the election? Keep in mind you can't just do a recount -- you have to repeat the polling process because it's the paper records that are in question.
Suppose it takes 10 voters who are confident that the machines are faulty to force a re-run of the election (an improbably tiny number by current standards).
The probability that 0 voters will detect a problem is 0.99^600 = 0.241%.
The probability that exactly 1 voter will detect a problem is (0.01)*(0.99^599)*C(600,1) = 1.458%.
The probability that exactly 2 voters will detect a problem is (0.01**2)*(0.99*598)*C(600,2) = 4.410%.
In general, the probability that k voters will detect a problem is (0.01**k)*(0.99*(600-k))*C(600,k). Here's what this looks like:
Now the probability that the problem will not be corrected is the probability that k is less than 10. That's the total of all the values P(k=0) through P(k=9), which is over 84%. The chance of recovery is less than 16%, which doesn't sound so good anymore.
But it's still worse than that.
What is the likelihood that a voter who detected a problem won't just think they made a mistake? Is such a voter likely to think that the machine made the error, or that they made an entry error themselves? Suppose half the voters who find mistakes on their VVPAT cancel and re-enter their votes. The other half (probably a generous fraction) are convinced that something is wrong and register a complaint.
That means 20 voters have to detect the problem in order to cause recovery. Now the chance that the problem will escape detection is the sum from P(k=0) to P(k=19),
I notice, though, that you completely ignored my point that source inspection is useless. Twice. Why is that?
Let's go back a bit to clear up what we're discussing. You made a number of points in your first comment. I chose to address just one of them because I didn't have the time to engage in multiple debates with you simultaneously. But I'll explain my thoughts more fully now so you can understand where I'm coming from.
I see three points you're made so far (please confirm):
Paper audit trails make source code irrelevant to voting system correctness.
Punchscan allows every voter to verify that their vote is recorded and counted correctly.
It is impossible to ensure that the code running in the voting machines matches what was reviewed.
Here are my positions:
I disagree. Even with paper trails, software is still relevant.
I agree. Punchscan's end-to-end verification mechanism goes far beyond what a VVPAT can do.
I think it depends on your level of confidence. I agree this is hard to do. If you want 100% confidence, sure, it's impossible. But the confidence level is also not 0%.
My more general point is that all these things -- the correctness of the code, the use of voter-verified records, measures against tampering with the installed software, and more -- contribute to increased confidence. You cannot point to one of them (VVPATs) and say that it completely eliminates all effects of another (the code).
I decided to respond to your point #1 because it was the one I had the strongest objection to. I'll separate out my more detailed response to that, since this is getting rather long.
Perhaps you're understanding something different than I mean when I say "voter-verifiable paper trail".
I'm pretty sure we're talking about the same thing.
Did you see this part of my comment?
If the software crashes more often in one district than another, or sometimes skips contests, or fails to display certain candidates, that's going to bias your result.
People keep saying how fast Canadian elections are. (I'm Canadian too.) But they're missing a huge difference.
In Canada you usually have one contest.
This is why hand-counting doesn't work in the United States. Chicago, November 2004: 10 pages, 15 elected offices, 74 judges, one referendum. That's 90 contests.
No -- a voter-verifiable paper trail, while useful, does not make source code irrelevant.
Voter-verifiable records only help ensure that votes are counted as recorded. They don't fully address problems that occur before the votes are recorded: votes can still be recorded incorrectly due to ballot presentation errors, or never recorded at all due to software failures.
Think of an election as a scientific measurement. In order to get an accurate result, the polling mechanism has to be free of bias. If the software crashes more often in one district than another, or sometimes skips contests, or fails to display certain candidates, that's going to bias your result. So it's essential to verify the whole software-controlled polling process.
And of course paper trails only matter when jurisdictions order a recount, and even then only for voters who carefully checked their printouts. So it's still quite possible that a vote could go misrecorded or miscounted due to bad software, despite the presence of a paper trail.
And finally -- what is responsible for printing the paper trail? Why... software. It's not hard to write software that produces misleading or confusing paper records. So again there is a vulnerability to software.
In short, if software is involved in ballot presentation, vote recording, or audit trail production, the software matters. I'm not saying paper trails are bad, just that they don't solve the whole problem. Source code inspections are essential.
Now for the subjective part of my comment. The concept of "software independence" is a laudable goal -- and achieving "software independence" as defined in the guidelines is certainly an improvement. Voting systems that fail to meet the guidelines' definition of "software independence" deserve little confidence, given what we know about bugs and complexity in software.
My problem with the term "software independence" is that it is misnamed. The guidelines give a definition of "software independence" that does not actually mean the election's correctness will be independent of software. Their definition is much narrower -- to achieve what they call "software independence," all that is necessary is a software-free way to audit the count of recorded votes. This has two big weaknesses:
Altering recorded votes is not the only way to tamper with an election. For example, this definition ignores the preparation and presentation of the ballots to voters. What about votes that are wrongly recorded, or never recorded at all? What if software failures are biased toward a particular group of voters?
It describes a vote count that is less than fully dependent on software. A voting system that is vulnerable to software bugs in 99.9% of realistic situations still counts as "software independent," as long as it's not 100% dependent. A system can technically be called "software independent" matter how vanishingly small the chances are of detecting a software error, and no matter how much work it would take to detect the error, as long as someone can conceive of a procedure that would detect it.
I think this is kind of sad, because it means we can no longer say "software independent" to describe voting systems that are actually independent of software, as in not dependent on software, i.e. what most people would think the term means.
It is interesting that the guidelines propose Open-Ended Vulnerability Testing, which is essentially described as a red-team exercise. This is a new and significant addition.
For those of you who have wanted voter-verifiable paper records, the new VVSG says:
Software independence means that an undetected error or fault in the voting system's software is not capable of causing an undetectable change in election results. All voting systems must be software independent in order to conform to the VVSG.
See section 2.4 for a discussion of "software independence." The draft guidelines present "independent voter-verifiable records" (IVVR) as one method of achieving "software independence," though it leaves the door open for other innovative ways of achieving the same goal (such as end-to-end cryptographic verification).
I definitely recommend reading the guidelines. There's a lot of stuff in there.
To encourage politicians to stand up for the things we believe in, we have to send a message, loud and clear.
(I do not work for the Dodd campaign. I just believe that if you want to have influence, you've really got to show some reaction when something goes right.)
If the Democrats who voted to condemn Moveon had two brains between them, they would have added language to the bill that condemned the attacks on Max Cleland, John Kerry, John Murtha, the generals who have questioned the war, and Rush Limbaugh's "phony soldiers". Then the Republicans in the Senate would have been forced to condemn the right wing's attacks on those who have served or are serving in the military, or be shown to be political hacks.
I believe they did propose just such an amendment (scroll down to "SA 2947"). The result: almost all the Democratic senators voted for it; almost all the Republican senators voted against it.
Here's one example: Chicago, Illinois, November 2004. 10 pages of choices, with 15 elected offices, confirmations of 74 judges, and one referendum. We're talking about 1 or 2 orders of magnitude longer than a Canadian ballot.
I do not support unauditable voting computers. I just wanted to explain why the voting problem is much different in the U. S., and give you some idea why the desire for automation is so strong. (I'm Canadian as well.)
Sure, many of these factors are related. But what is the book really about?
It isn't a book about making users more productive, or a book about reducing user error. This is a book about making websites fast. The other factors are only peripheral effects. Those 14 rules that Souders is pushing are all about speed, not these other factors.
Making a website fast may improve many things about the experience. But speed is not the only thing you need to make a website perform well.
The reviewer makes the same mistake of saying "performance" many times where he really means "speed". The more often that mistake is made, the easier it is to forget that speed is not the answer to every problem. See my point now?
The title of the book should be "High Speed Web Sites" or just "Fast Web Sites."
"Performance" is not a general-purpose synonym for "speed." "Performance" is a much more general term; it can refer to memory utilization, fault tolerance, uptime, accuracy, low error rate, user productivity, user satisfaction, throughput, and many other things. A lot of people like to say "performance" just because it's a longer word and it makes them sound smart. But this habit just makes them sound fake -- and more importantly, it encourages people to ignore all the other factors that make up the bigger picture. This book is all about speed, and the title should reflect that.
So, I beg you: resist the pull of unnecessary jargon. The next time you are about to call something "performance," stop and think; if there's a simpler or more precise word for what you really mean, use it.
Slashdot Mirror
I am in complete agreement with #23208332. Your summary is an unfair exaggeration of Knuth's position; he mentioned that unit testing was rarely useful for him, in an interview that was mostly about many other topics. To make "rips on unit tests" the main headline is a wild distortion of what he actually said. For shame.
If the community decides that a page isn't notable, just label it thus and move on. There's no reason to delete the page.
The same thing goes for page locking: although there are still some extreme cases where pages need to be locked, many of the reliability problems would be mitigated by labelling recently-changed parts or frequently-changed parts of pages. Readers can then take responsibility for their own level of trust.
Both cases are about matching expectations to reality: the situation can be improved by changing the content OR by making expectations more accurate.
No, IRV does not eliminate spoilers. The spoiler effect exists in IRV. With IRV there are still wasted votes, except it's much harder to tell which votes will be wasted and what effect your ballot will have; the behaviour of IRV is much more complicated, often pathological, and thus arguably even worse than the current system.
Range or approval voting would be a better option. They truly eliminate the spoiler effect, they are easy to implement using unaltered existing equipment, and have simple, easily understandable behaviour.
The horse race in the media is driven by all this nonsense about "momentum" and "viability" -- people are trying to get a sense from the early states who is likely to win, so they can vote for someone who has a chance. But the only reason this is necessary is because the votes are being split 7 or 8 ways. That's madness.
Think about it: suppose a party has three major candidates, one in the middle with respect to the party's base of support, one a little to a left, and one a little to the right. If each voter registered in that party gets to vote for just one candidate in the primary, the two candidates on the left and right will split most of the votes, taking them away from the candidate in the middle. The candidate in the middle, who would be preferred by the most voters in the party, gets eliminated.
The whole purpose of running a primary is to choose a single representative behind which the whole party can unite. If parties didn't run primaries, and all the candidates just ran in the general election, they would split all the votes and probably none would win (imagine a general election with 9 Democrats running against 1 Republican). But running a primary by plurality voting (each voter votes for just one candidate) just re-creates the vote-splitting problem that the primary is supposed to solve.
I can't understand why neither of the parties has decided to run their primary by approval voting (each voter votes for as many candidates as they like). That would better reveal the total support for each candidate, thus giving a better reflection of how well they will do in the general election. The first party to do this, it seems to me, would gain a significant advantage, because it would actually choose the most broadly liked candidate, the strongest candidate for the general election. What they're doing right now seems to be just shooting themselves in the foot.
This is why that doesn't work (in general) in the United States.
http://vote.nist.gov/ballots/il_chicago_20041102_01.pdf
One ballot = 90 contests.
Keep these points in mind:
- Ballots in the U. S. typically have dozens of contests -- sometimes 60, 70, or more contests. Hand counting is significantly less practical in the U. S. than in say Canada, where your ballot is just a vote for a single candidate in a single contest.
- Electronic voting has real security risks. Most folks here know that already. The risks can be big.
- Electronic voting has real potential advantages. The number of undervotes, overvotes, and otherwise spoiled ballots is considerable and significant. We are talking about millions of votes here. Voters really do make more errors when they vote on punchcards or on centrally scanned paper ballots -- and these errors disproportionately affect poorer and less educated voters. Precinct-based scanning prevents overvoting (voters find out immediately if their ballot is improperly marked, and can try again). DREs prevent overvoting and also have the potential to significantly reduce error rates by warning the voter of skipped contests, giving better instructions, and supporting more languages. DREs also have unique advantages for disabled voters.
It is not inconceivable that switching to central-count optical scanning could actually leave Ohio worse off than DREs, depending on your assumptions about the frequency of voter errors and the magnitude of security risks. There are many factors involved.At the moment, my favourite is precinct-based optical scanning (paper-based, simple, with immediate feedback to voters) or paper ballots printed by a computerized voting interface (all the advantages of computerized vote entry -- IF the UI is well designed, without losing the verifiability and auditability of paper).
The problem is that Facebook is lying about it, and doing so repeatedly.
The one-minute estimate is too low, not too high. In Canada and other countries that do hand counts, there are usually just one or two contests on the ballot. In the United States, there are typically an order of magnitude more. In Chicago in 2004 the ballot had 90 contests on it. So we're talking about more like 5-10 minutes per ballot. Hand counts would only be feasible if the number of contests were reduced or only some of the contests were counted by hand and the rest were counted by machine.
I'm no fan of unverifiable electronic voting machines, but I do have to point out that the scales are different. In Canada and other countries that do hand counts, there are usually just one or two contests on the ballot. In the United States, there are typically an order of magnitude more. In Chicago in 2004 the ballot had 90 contests on it. Hand counts would only be feasible if the number of contests were reduced or only some of the contests were counted by hand and the rest were counted by machine.
I think you're talking about a Vote of No Confidence.
The short answer: chain of custody.
Like any other piece of election equipment, the voting machines have to be physically protected from the time they are configured to the time they are used. The same holds true for mechanical voting machines, paper ballots, electronic pollbooks, and so on.
I'm not saying that current procedures for this are adequate -- far from it. Obviously if you leave the machines for unattended sleepovers, they're likely to be vulnerable to tampering. On the other hand, protection of physical equipment is relatively well understood; the risks can be identified and reduced. There is certainly a much longer history of experience in moving things securely from place to place than in recounting voter-verified paper trails.
The problem in this case is not the same as the general attestation problem, which is extremely difficult (though many people think even that is not impossible). We aren't talking about mass-distributed consumer electronic devices that people can rewire in the comfort of their homes in an effort to make them lie about the software they are running. We're talking about moving a machine from one controlled environment to another controlled environment.
If your position is that physical security can be expensive, I have no dispute with you there. If your position is that you don't trust the computing hardware to function correctly, I offer no argument -- in that case all electronic voting is infeasible, VVPATs are moot, and Punchscan is the only solution. I consider that a position worthy of consideration, though I do not take it myself. I think it is possible to trust the hardware, but I'm willing to be convinced otherwise.
I don't think it's all that easy.
Based on what you're written ("The mechanism has to be free of *undetectable* bias." and "Don't you think those behaviors would be noticed?"), I suspect you are making the assumption that detectability is what matters. But that's not enough; the bias has to be actually detected, and a detected bias is okay only if election administrators can recover, and do recover, from the problem.
It appears that where you and I differ is a matter of scope. In the theoretical world, all that you need is a conceivable possibility of an error being detected in order to rule out dependence on software. But in practice, it's more complicated than that.
For example, let's talk specifically about VVPATs. Forget crashing or bad ballot presentation for the moment; I'm just talking about a machine that, some fraction of the time, prints the wrong selections on the audit trail.
Take your scenario with 60000 voters, a machine that misprints 1% of the VVPATs, and a 1% probability that each voter will competently check their VVPAT. Each of those 600 misprinted VVPATs has a 99% chance of going undetected, so the chance that all of them will be undetected is 0.99^600 = 0.0024. So the chance that at least one will be detected is around 99.76%. So far so good. (I'm not sure how you got 95% -- I'd appreciate you walking me through your math.)
But what happens when a voter thinks there's something wrong with their VVPAT? Do you shut down the entire election and start over? Do you launch an investigation of the voting system vendor? Certainly not according to current practice. How many voters have to report a problem before you re-run the election? Keep in mind you can't just do a recount -- you have to repeat the polling process because it's the paper records that are in question.
Suppose it takes 10 voters who are confident that the machines are faulty to force a re-run of the election (an improbably tiny number by current standards).
The probability that 0 voters will detect a problem is 0.99^600 = 0.241%.
The probability that exactly 1 voter will detect a problem is (0.01)*(0.99^599)*C(600,1) = 1.458%.
The probability that exactly 2 voters will detect a problem is (0.01**2)*(0.99*598)*C(600,2) = 4.410%.
In general, the probability that k voters will detect a problem is (0.01**k)*(0.99*(600-k))*C(600,k). Here's what this looks like:
P(k=1) = 1.46%
P(k=2) = 4.41%
P(k=3) = 8.88%
P(k=4) = 13.39%
P(k=5) = 16.12%
P(k=6) = 16.14%
P(k=7) = 13.84%
P(k=8) = 10.36%
P(k=9) = 6.88%
P(k=10) = 4.11%
P(k=11) = 2.23%
P(k=12) = 1.10%
P(k=13) = 0.50%
P(k=14) = 0.21%
P(k=15) = 0.08%
P(k=16) = 0.03%
P(k=17) = 0.01%
P(k>=18) = small
Now the probability that the problem will not be corrected is the probability that k is less than 10. That's the total of all the values P(k=0) through P(k=9), which is over 84%. The chance of recovery is less than 16%, which doesn't sound so good anymore.
But it's still worse than that. What is the likelihood that a voter who detected a problem won't just think they made a mistake? Is such a voter likely to think that the machine made the error, or that they made an entry error themselves? Suppose half the voters who find mistakes on their VVPAT cancel and re-enter their votes. The other half (probably a generous fraction) are convinced that something is wrong and register a complaint.
That means 20 voters have to detect the problem in order to cause recovery. Now the chance that the problem will escape detection is the sum from P(k=0) to P(k=19),
Let's go back a bit to clear up what we're discussing. You made a number of points in your first comment. I chose to address just one of them because I didn't have the time to engage in multiple debates with you simultaneously. But I'll explain my thoughts more fully now so you can understand where I'm coming from.
I see three points you're made so far (please confirm):
Here are my positions:
My more general point is that all these things -- the correctness of the code, the use of voter-verified records, measures against tampering with the installed software, and more -- contribute to increased confidence. You cannot point to one of them (VVPATs) and say that it completely eliminates all effects of another (the code).
I decided to respond to your point #1 because it was the one I had the strongest objection to. I'll separate out my more detailed response to that, since this is getting rather long.
Don't you think that would influence an election?Did you see this part of my comment?
Ballots in the United States typically have 20 to 30 contests on them. The Chicago ballot in 2004 had 90 contests.
People keep saying how fast Canadian elections are. (I'm Canadian too.) But they're missing a huge difference.
In Canada you usually have one contest.
This is why hand-counting doesn't work in the United States. Chicago, November 2004: 10 pages, 15 elected offices, 74 judges, one referendum. That's 90 contests.
See more at NIST's ballot collection.
No -- a voter-verifiable paper trail, while useful, does not make source code irrelevant.
Voter-verifiable records only help ensure that votes are counted as recorded. They don't fully address problems that occur before the votes are recorded: votes can still be recorded incorrectly due to ballot presentation errors, or never recorded at all due to software failures.
Think of an election as a scientific measurement. In order to get an accurate result, the polling mechanism has to be free of bias. If the software crashes more often in one district than another, or sometimes skips contests, or fails to display certain candidates, that's going to bias your result. So it's essential to verify the whole software-controlled polling process.
And of course paper trails only matter when jurisdictions order a recount, and even then only for voters who carefully checked their printouts. So it's still quite possible that a vote could go misrecorded or miscounted due to bad software, despite the presence of a paper trail.
And finally -- what is responsible for printing the paper trail? Why... software. It's not hard to write software that produces misleading or confusing paper records. So again there is a vulnerability to software.
In short, if software is involved in ballot presentation, vote recording, or audit trail production, the software matters. I'm not saying paper trails are bad, just that they don't solve the whole problem. Source code inspections are essential.
My problem with the term "software independence" is that it is misnamed. The guidelines give a definition of "software independence" that does not actually mean the election's correctness will be independent of software. Their definition is much narrower -- to achieve what they call "software independence," all that is necessary is a software-free way to audit the count of recorded votes. This has two big weaknesses:
- Altering recorded votes is not the only way to tamper with an election. For example, this definition ignores the preparation and presentation of the ballots to voters. What about votes that are wrongly recorded, or never recorded at all? What if software failures are biased toward a particular group of voters?
- It describes a vote count that is less than fully dependent on software. A voting system that is vulnerable to software bugs in 99.9% of realistic situations still counts as "software independent," as long as it's not 100% dependent. A system can technically be called "software independent" matter how vanishingly small the chances are of detecting a software error, and no matter how much work it would take to detect the error, as long as someone can conceive of a procedure that would detect it.
I think this is kind of sad, because it means we can no longer say "software independent" to describe voting systems that are actually independent of software, as in not dependent on software, i.e. what most people would think the term means.It is interesting that the guidelines propose Open-Ended Vulnerability Testing, which is essentially described as a red-team exercise. This is a new and significant addition.
The second chapter of the introduction provides a good rundown of the new material in the guidelines.
I definitely recommend reading the guidelines. There's a lot of stuff in there.
If you care about this issue, show Chris Dodd your thanks RIGHT NOW.
Call him at (202) 224-2823, send him a note, contribute to his campaign, or comment on the blog post. Show him you mean it.
To encourage politicians to stand up for the things we believe in, we have to send a message, loud and clear.
(I do not work for the Dodd campaign. I just believe that if you want to have influence, you've really got to show some reaction when something goes right.)
Ballots in the United States are far longer than those in Canada. Have a look for yourself: NIST has a collection of ballots online.
Here's one example: Chicago, Illinois, November 2004. 10 pages of choices, with 15 elected offices, confirmations of 74 judges, and one referendum. We're talking about 1 or 2 orders of magnitude longer than a Canadian ballot.
I do not support unauditable voting computers. I just wanted to explain why the voting problem is much different in the U. S., and give you some idea why the desire for automation is so strong. (I'm Canadian as well.)
Sure, many of these factors are related. But what is the book really about?
It isn't a book about making users more productive, or a book about reducing user error. This is a book about making websites fast. The other factors are only peripheral effects. Those 14 rules that Souders is pushing are all about speed, not these other factors.
Making a website fast may improve many things about the experience. But speed is not the only thing you need to make a website perform well.
The reviewer makes the same mistake of saying "performance" many times where he really means "speed". The more often that mistake is made, the easier it is to forget that speed is not the answer to every problem. See my point now?
The title of the book should be "High Speed Web Sites" or just "Fast Web Sites."
"Performance" is not a general-purpose synonym for "speed." "Performance" is a much more general term; it can refer to memory utilization, fault tolerance, uptime, accuracy, low error rate, user productivity, user satisfaction, throughput, and many other things. A lot of people like to say "performance" just because it's a longer word and it makes them sound smart. But this habit just makes them sound fake -- and more importantly, it encourages people to ignore all the other factors that make up the bigger picture. This book is all about speed, and the title should reflect that.
So, I beg you: resist the pull of unnecessary jargon. The next time you are about to call something "performance," stop and think; if there's a simpler or more precise word for what you really mean, use it.
Thanks for listening!