October 2008 Archive
ElectionAudits Software News
Help audit the election!
Proper audits are the gold standard of election integrity - they can tell whether the election system actually counted the real ballots properly. Unfortunately, they are rarely done properly. We need better audits of our elections in order to promote confidence in the results. Thankfully, there are resources to make it easier than it has been in the past, both in terms of procedures and statistical research, and in terms of open source software. If you are willing to talk to your local elections officials, you can make a difference. Read the Background section below for more information.
To do my part, I've worked with the Boulder County Clerk and started the ElectionAudits open source software project to help us audit elections with appropriate statistical significance, and I need help there too! If you're skilled with Python or Django (especially on Windows since it is developed on Ubuntu Linux), database query optimization, xml, xslt, css, setuptools for packaging cross-platform easy_install eggs, statistics, documentation or the like, see the ElectionAudits Software Home Page, download the latest code, and please lend a hand:
We all know there are many questions being raised about the systems used to count votes in our elections. The bottom line, as noted in a paper by MIT voting and security expert Ron Rivest and NIST voting expert John Wack, is the need for "Software Independence":
meaning that election software should not be a critical component which, if it fails, can threaten the results of an election.
That is why we've moving from using unauditable paperless DRE machines to systems based on paper ballots that can be audited. And we're slowly getting there. But even for optical scanner systems we also need to check the software that does the actual tallying and reporting of the results.
Those paper ballots are of little use if we never audit them. And we still have a long way to go on effective audits of elections: the voting systems don't support audits well, the laws and regulations are sparse and often inadequate, and the topic is confusing to many.
This prompted California Secretary of State Debra Bowen to establish Post Election Manual Tally Requirements and issue California proposed emergency regulations for them.
Along with that, recently, experts in elections, statistics, computer science, risk assessment and related fields have been working on better audits, coordinated by ElectionAudits.org, the nation's clearinghouse for election audit information. The latest results are the Principles and Best Practices for Post-Election Audits.
I've been very active in that effort, and convinced Boulder County to do a complete audit, far better than what is currently required in Colorado, starting with an audit of the 2008 Primary in Boulder:
Based on that experience, it was clear we needed software to help deal with the inadequacy of the reports available from our Hart InterCivic BallotNow system, and to make it clearer how to base the work on good statistics. So the software includes the "varsize.py" statistical software from Ron Rivest to guide efficient sampling of the audit units.
See the demo and more at http://neal.mcburnett.org/electionaudits/
Colorado needs to update its election auditing procedures. Rules that follow the "Principles and Best Practices for Post-Election Audits"
In Colorado the problems are worse than the existing California rules. Below is a list of places where Colorado doesn't follow the Best Practices. Colorado should follow the lead of Boulder County, which is addressing all of the following problems, as described at http://bcn.boulder.co.us/~neal/elections/2008-boulder-primary-audit/.
No real audit for mail-in ballots or central-count systems
Mail-in ballots will be heavily used in this election, and will typically be counted on "central count" systems. But the Post-Election Audit Rule, 126.96.36.199, from the Secretary of State is out of compliance with the Colorado Revised Statutes 1-7-514, for the case of central-count election machines, and does not specify an audit at all. It calls instead for a post-election test on live ballots. The purpose of an audit is to ensure that the device worked during the actual counting process, not that it can be made to work in a separate count afterward. The "Logic and Accuracy Test" rules already address testing. For precinct-count scanners and DREs, the law and rule are consistent in calling for audits of previously reported election results. The rule for central count machines should call for no less.
The root problem is that some central count systems used here don't report the original election day results that correspond to each batch of paper ballots fed in to the scanner.
The existing precinct-level reports can't be used, because in Colorado the physical mail-in ballots are typically all mixed together, not sorted into piles by precinct. When it comes time to audit a particular precinct from the report, the auditors would have to go through all of the mail-in ballots to find and audit the ballots for that precinct.
The alternative is for counties to sort the physical mail-in or early paper ballots into piles by precinct as they arrive. Many other states do this, although auditing the early votes on DRE machines remains a big problem, since the rolls of paper containing the VVPATs contain ballots from all precincts, mixed together.
The best solution is to either have the vendors do a simple report by batch, or to find another way to generate audit reports containing the results of each physical batch of paper ballots, as Boulder does.
Random selection prior to auditable public results
The choice of which machines have been selected to be audited is available for misuse before the results to be audited are typically produced and shared with the public, since the SoS selects machines within 24 hours after the close of polls. Thus anyone intent on corrupting the results would have an opportunity to be sure to only corrupt them on machines that won't be audited.
Instead, the machines or batches to be audited should not be selected until clerks first publicly release the unofficial results for each contest on each machine (or batch). And that audit report is what should be audited.
Random selection not transparent
The selection should also should be in public, using dice (Cordero, Wagner and Dill 2006) or perhaps a publicly verifiable random selection method based on public random numbers. But it seems that software is used to do the "random" selection, and it is not clear how the public can be sure it really is random. In the 2008 Primary, races that were uncontested or not even in the right county were selected, as documented in my Boulder 2008 Primary Audit writeup.
Audit of central count machines too small, doesn't comply with the law
Only a small fraction of central-count ballots are audited in many counties (a maximum of 500 per central count machine even if it counts e.g. 40000 ballots - Rule 188.8.131.52 - that would be 1.25%). This is not even the "specified percentage" which is specifically called for in Colorado Revised Statues 1-7-514: "where a central count voting device is in use in the county, the rules promulgated by the secretary pursuant to subsection (5) of this section shall require an audit of a specified percentage of ballots counted within the county."
Not a high enough percentage
Even in cases where there are 5% of ballots audited, the contests to be audited are spread out among all the machines to be audited, so very little auditing is done on each contest. E.g in Boulder County, 5% of the machines is perhaps 16 machines total, and the 71 contests to be audited are spread out over those machines. So only perhaps 5 out of the 71 contests in play will be audited on each machine selected, implying only 0.35% of the ballots per contest. Races that are uncontested are included in the selection, which is a waste of effort. California, by contrast, requires a 1% audit per contest.
Lack of paper audit trail in some counties
Some counties are not requiring that a paper record exist for all ballots, and those electronic DRE "ballots" can't be audited at all.
Wasteful focus on landslide contests, and little confidence for tight contests
The audit is wasteful for contests with a wide margin of victory and inadequate for races with a tight margin of victory, and is not designed to limit the risk of certifying an invalid result. State-wide contests that have a wide margin don't need much auditing at all - a Judge approval contest with a 32% margin might merit selecting only a half dozen machines state-wide, while a race with a one percent margin needs one or two hundred - more than one audit unit per county.