EAC - Voluntary Voting System Guidelines

Name : | Cem Kaner
Organization : | N/A
Post Date : | 9/30/2005
General Comments
Comment : | TESTING OF VOTING SYSTEMS The second class of issue that I want to raise with you involves the testing and testability of these systems. The Guidelines require an extensive and expensive independent verification and validation of electronic voting system software and equipment. This is good, but it is insufficient. I write to you as someone who has experience with independent verification and validation. I have trained testers who work for independent software test labs, and provided testing services through others as a subcontractor of independent labs. I have counseled lab executives sometimes as an attorney, sometimes as a technologically-oriented management consultant. I have run in-house test labs for prominent software publishers and served as the primary point of contact between independent and in-house test organizations, as the representative of the in-house group (the client of the independent lab) and I have counseled others in this task. I have served as an expert witness in cases that hinged on the details of testing of products whose defects led to litigation. I am the author of books on software testing. It is theoretically impossible to completely test a computer program of any significant size and it is impossible to prove through testing that a program is free of errors. An example developed by Glen Myers at IBM and widely taught in introductory courses involves a simple program (about 20 lines of code, depending on the programming language) that would require 100 trillion tests (at 5 minutes per test, that’s a billion years of testing). For more extensive discussion of this and other examples of the impossibility of complete testing, you might wish to watch a videotaped course segment at http://www.testingeducation.org/BBST/Intro2.html. (Thanks to NSF’s support, this course is available free to the public.)