The Boulder County Elections Division broke new ground for Colorado and the nation by doing one of the first election audits that adhered to the "Principles and Best Practices for Post-Election Audits".
It was also perhaps the most extensive "risk-sensitive" audit to date. When the margin of victory in a race is wide, that means that there would have to be problems with a large number of votes to change the outcome. So we don't have to look at many batches of votes to detect that. But if the margin is small, we have to look more closely. Traditional audits look at the same percentage of votes in each contest. Risk-sensitive audits shift the effort according to the margin of victory and are thus more efficient and effective.
For example, the margin in the Lafayette 2A contest was much tighter than in the City of Boulder's issue 2A. So we audited 7 batches of ballots for the Lafayette measure and 2 for Boulder's, and got the same level of confidence for both. That is more effective than auditing 4 or 5 batches of votes for each race, and more efficient than auditing 7 for each.
Boulder will be championing the use of risk-sensitive auditing procedures before the Colorado Election Reform Commission, which is currently looking at improving audits in Colorado.
Boulder County augmented the "Random Audit" required by Colorado law. The improved methods, pre-approved by the Secretary of State,
compare hand counts of votes within batches against the actual counts produced during the live election, and
audit all types of ballots, including mail-in ballots which are mostly overlooked in current Colorado audits.
13 contests were audited, 7 state-wide and 6 local. All but the race for president were chosen at random, but with a preference for those with the closest margins. On three of the very closest local races we accepted a lower level of confidence, so that we would be able to audit other races. But we still got higher levels of confidence than the previous auditing method would produce. The detailed statistical analysis of this audit is continuing.
We audited more ballots in a single contest than ever before in Boulder, including 13 batches of votes for the tight Longmont Question 2A race, involving 8239 ballots.
We targeted two batches for audit based on high numbers of the "phantom votes" noted during the resolution phase of the original machine count. In each batch we audited two contests likely to be affected, and found no significant evidence of problems, so the special review procedures adopted during the original count appear to have been very effective.
The overall results give us more confidence than ever in the reported results for this election, based on the close agreement between the hand counts and machine counts. We did find one ballot in which an overvote was reported as an undervote, due to a confusing user interface in the resolution process. The hand count results most often matched the system count exactly, and generally differed by less than 2 votes, which is far less than what would trigger an escalation of the audit.
The audit also set new standards for transparency and verifiability. All the detail on results to be audited was published on the web before the random selection of batches to be audited was done. Peer-reviewed procedures for ensuring verifiably random dice throws and selections were used. A recently published "NEGEXP" method was used to select the batches in a way that is more efficient at detecting problems than previous methods.
We reviewed 40047 ballots looking for votes in particular contests, which is 0.5% of the total number of votes cast by voters in Boulder County. This is still about half of the total number of ballots that we would have audited using the rules in California, in which 1% of the ballots are audited for each contest.
The audit used a new open source program, ElectionAudits, which read the election reports by batch, independently tallied up the batches, and aided in the statistical analysis, random selection and publication of the results. Full details on the audit, the software, and the procedures used can be found at
http://bcn.boulder.co.us/~neal/elections/boulder-audit-08-11/
Note: Earlier versions of this report described the audits as "risk-limiting". But since we were mainly piloting in one county what such an audit would involve if the whole state were to agree to do it, that terminology does not really fit what we actually accomplished. More attention to the details of the escalation procedures and calculations of the levels of confidence are also necessary for a proper risk-limiting audit.