Boulder's manual count audit is an important step forward. Thanks to City Clerk Alisa Lewis and the County Clerk's office for moving forward with this process. But the plan for the audit needs to be improved.
The plan should have been developed in concert with the groups that called for it before it was submitted to the Secretary of State. The audit is not independent, in the sense that the candidates, parties and citizens had very little input into who would do the counting, etc.
Only three batches of votes were selected, totalling about 600 ballots, by the same person who is running the election, in a manner that was not statistically random. This means there is little statistical confidence in the results of the audit.
But Boulder has a rare opportunity to be a national leader in doing more efficient, more sensitive audits, because our system has unique identifiers on the ballots.
There is active interest from the Colorado Secretary of State's office, County clerks, Colorado Senate Majority leader, and many others in efficient manual count audits. Legislation is pending in Congress, Colorado, and other states.
John Wack at NIST just presented a draft report to a technical committee of the US Election Assistance Commission saying that it would be very useful if DREs had serial numbers recorded both on VVPATs and on electronic ballots, to enable precise ballot-by-ballot auditing.
David Dill, Jonathan Wand, Paul Walmsley and other experts have suggested that auditing even a very small number (230) of individual ballots can provide a surprisingly robust statement of confidence in the accuracy of the counting and tallying process.
The voting system used in Boulder, Hart InterCivic's BallotNow, has serial numbers on each paper ballot, and a "cast vote record" database on the system (normally inaccessible to election officials) indicating how each ballot was interpreted, by serial number.
Boulder should demonstrate an audit that would highlight innovative ideas and systems, and have a positive effect on voter confidence and the speed and efficiency of elections. It would also reflect well on all the participants. We call on Hart to support this, perhaps initially by using internal development software to provide ballot-by-ballot results, given serial numbers from the auditors. We call on the Secretary of State, the Boulder City Clerk and the Boulder County Clerk to permit this manner of conducting a manual count audit as soon as possible.
There is widespread support for Voter-Verified Paper Audit Trails (VVPAT). But the paper trail needs to be looked at after the voter verifies it. Therefore an automatic Manual Count Audit of at least some part of the paper trail is also a necessary part of verifiable elections. This is also known as an automatic manual tally audit, random hand count, manual recount, independent hand audit, etc.
Manual count audits can catch a wide variety of problems, ranging from faulty software or hardware or manipulation of the machines or results by outsiders, to fraud by manufacturers, election officials or volunteers.
E.g. exactly the sort of audit described here detected problems with a vote in California's Napa Valley.
There is widespread citizen support for an independent audit in which a hand count of of voter-verified paper ballots is compared to the results of the current voting system. The hand count must be performed in public by representatives of the various candidates/parties/issue campaigns.
California, Kentucky and West Virginia currently require manual counts in between 1% and 5% of precincts:
KY: As part of the official canvass, a manual recount of randomly selected precincts representing three to five percent of the total ballots cast in each election shall be completed.
WV: West Virginia During the official canvass and any requested recount, at least five percent of the precincts are to be chosen at random and the ballot card cast therein counted manually.
CA: During the official canvass of every election in which a voting system is used, the official conducting the election shall conduct a public manual tally of the ballots tabulated by those devices cast in one percent of the precincts chosen at random by the elections official.
Colorado also has experience with audits via manual counting. In Boulder County a error in the original count was confirmed during a manual count of the St Vrain School District on Dec 7 2004 (see ManualCountAuditStVrain for details). Boulder will use a Manual Count Audit for the March 8 2005 municipal election also.
The general procedure is that the jursidiction produces an abstract of votes detailing election results for each race from each precinct or machine or other "batch" of ballots. It is best if the number of ballots per batch is small. Specific batches are then randomly selected. For each batch, a manual count of each race on the corresponding paper ballots is compared to the original results. They should of course be exactly the same. If not, further investigation is necessary, though some very small margin of error might make sense if it is well understood. Doing that for many batches results in a bunch of numbers which all should match.
As noted by elections expert Professor Douglas Jones, the power of a hand audit can be improved by picking some of the precincts (perhaps half of them) based on "measures of suspicion":
Permit the parties, for example, to name the precincts they're most suspicious of, or look at the vote distributions district wide and recount the precincts that represent extremes or that depart the farthest from what exit polling or other predictions suggested for those precincts. Allowing the parties to name their targets simply lets the parties compare the exit polls or statistical predictions. You must, of course, keep some percentage of the targets for the audit random.
If early and absentee ballots from all the precincts are mixed together, sorting thru them for the ballots for a particular precinct can be time-consuming. This is one reason for allowing batches which don't represent precincts, e.g. they could represent individual machines or stacks of paper ballots fed into an optical scanner. But there has to be a comprehensive report to compare the manual count totals with, so that observers can compare the subtotals by batch with the overall election total, and compare the subtotals for the batches with the manual counts.
Jonathan Wand at Stanford has a paper [Auditing an Election using sampling: the impact of bin size on the probability of detecting manipulation http://wand.stanford.edu/elections/probability.pdf] about the degree of confidence that manual counts can provide, which depends in large part on the number of batches examined. If the goal is e.g. detecting a bug or hack that causes _all_ machines to give incorrect results, not many batches need to be counted. E.g. if the machines boost candidate A by converting 2% of candidate B's votes to A, just 5000 total ballots need to be sampled (in precincts of any size), to get odds of 10,000 to 1 that the problem will be detected.
More savvy hackers might manipulate just 10% of machines. They would have to swing ten times as many votes in each hacked machine in order to have the same overall effect on the election. That helps evade detection, but still with a random sample of 20 batches, there is only a 12% chance that they will get away with it. 40 batches reduces that to 1.4%, and for 60 batches there is a 0.17% chance of getting away with it. The size of the batches again doesn't matter much. So in circumstances like this, counting more smaller batches/precincts is better than counting fewer big ones.
Manual count audits are supported in HR 550 by Rep Holt in NJ, and by Colorado's SB05-198 (http://www.leg.state.co.us/)