CRT and STS Teleconference
Thursday, June 14, 2007
11:00 a.m. EDT
Draft Meeting Minutes
2) Issues received from TGDC members and the public (continued). See
the files under "CRT VVSG finalization working materials:"
3) Any other items
Alan Goldfine, Alicia Clay, Allan Eustis, Angela Orbaugh, Bill Burr, David
Flater, David Wagner, John Wack, Jon Crickenberger (NVLAP), Michael Koo,
Nelson Hastings, Paul Miller, Philip Pearce, Ron Rivest, Thelma Allen,
Wendy Havens, Rene Peralta
log being maintained by CRT contains over 273 issues, with 27 remaining
open as of today. Many of the open items are regarding terminology waiting
on feedback from Paul Miller. In order to make the deadline for final
draft, the open issues need to be closed by Friday, June 15.
# 6: Benchmarking Test Methods Look Wrong (Ron Rivest): There has been
extensive email regarding this issue and will continue to try and resolve
this via email. This has been re-written extensively in the language
of classical hypothesis testing. This has hopefully been clarified per
# 21: Threads and Memory Leaks (Ron Rivest): Should the following be
mentioned somewhere: "memory leaks" (memory allocations that
are never "freed") and "threads" (concurrent programming
techniques). Response: Memory leaks are addressed in III.188.8.131.52, though
not to the extent of proving their absence. Threads are not addressed.
Suggestion is to ban multi-threaded voting application logic, but this
could be controversial. It's common to have separate threads in gooey
applications. There are ways to deal with multithreads in a structured
way. The requirement should state that if a voting application's logic
is multi threaded, the vendor be required to provide demonstration that
it will not effect other programs or become deadlocked. David Flater
to add language.
# 20: Ballots for opscan volume testing (David Wagner): Is it OK to
substitute mechanically marked ballots for manually marked ballots during
test. The proposed requirement states that it is not valid to substitute
with mechanically marked ballots if the vendor specifies that the system
is designed to count manually marked ballots. The 2nd issue was in order
to reach the number of ballots needed to run the volume test, is it
ok to recycle ballots during the test. The proposed requirement states
that there must be at least 10,000 manually marked ballots to run through
the test and after that you can recycle that 10,000 to get the number
needed for volume testing. David W likes resolution proposed.
# 1: Terminology: Partisan contest to become party specific contest;
non-partisan contest to become non-party specific contest; after discussion
David F. will work to clarify the definition for primary election; the
term general election will be deleted.
# 2: Terminology: Jurisdiction. This means different things in different
states. David F. will find a new word to replace the word jurisdiction,
keeping the definition currently used.
# 3: Terminology: Scratch vote vs. crossover vote. It was decided to
replace with the term "straight party override".
# 15: Terminology: User serviceable: This term means for errors that
are correctable by trouble shooter or election officials. The requirement
will be changed to clarify that it is only consider user serviceable
if the directions for fixing the error are provided in vendor documentation.
# 19: Terminology: Ballot activation: This term will be defined to mean
what happens on the DRE itself. The epoll book issues ballot activation
credentials and John Wack has been using credential issuance in his
# 23: Terminology: Ballot choice term will be changed to contest choice.
# 4: Terminology: Should there be a different definition for ballot?
What should it be, there are so many options. Ron suggested that maybe
there should be a listing of all the definitions of ballots with footnotes
in the document to the one matching the text. David F to come up with
a list of possible definitions and forward to Paul Miller for feedback
from election community (with the understanding that comments may not
be able to get into the July draft).
Terminology: Is the term "during an election" ambiguous. This
is used in the access control section. Nelson Hastings and David F will
work on this issue offline to put in a requirement and will take the
term issue off the table.
# 14: Indicator for Whom (David Wagner): There is a requirement that
says that there will be a visual/audio indicator on the voting system
about the status of the voting device. It was decided to change the
requirement to narrow it to indicator for election judge.
# 27: Public Information Package (PIP): Proposed change to the requirement
by David Wagner is to write the standard to say the test report must
contain something labeled as the PIP and that the PIP must contain the
test plan and test results to the extent possible. No objections from
CRT meeting scheduled for Thursday, June 21. (Possibility it will be cancelled).
Cyrptogrpahy: The issue of what records have to be signed on the
voting system was discussed. There are multiple possibilities of what
a voting system does on Election Day and it was questioned whether different
keys were needed for different activities on the voting systems. It
was decided that the requirement would be written that the machine only
required one key, that everything must be signed, and there be able
to be a count of what was signed. [Bill Burr will write requirements
based on today's discussion.]
Security through obscurity: David Wagner thinks there should be
a requirement that makes it clear that the system must not rely upon
security through obscurity for security. His proposed text: The voting
system must not rely upon the secrecy of its design, implementation,
or documentation for the security or integrity of the election. The
question is where does it go and how do you test. It should be considered
a guiding principle the same way privacy is. It was decided that the
requirement should go in the documentation standards. [Angela Orbaugh
will update this section.]
Requires software independence: David Wagner was not able to find
any place in the standard which requires the voting system to be software
independent, as part of the normative text (rather than the discussion/introductory
material). He feels a requirement needs to be added. Subcommittee was
in agreement to add statement because the VVSG that is delivered to
the EAC must say all voting systems shall be SI, period. John Wack suggestion
was to add this requirement in the auditing chapter (being retitled
Security Architecture) as a high level requirement with subrequirements
on how to achieve SI, currently with VVPR. A glossary term for SI also
needs to be included - using Ron's original definition.
VVPR should be mandatory: David Wagner feels there should be a requirement
specifying that all voting systems must produce a voter-verified paper
record. This is responsive to the TGDC resolution requiring software
independence, combined with the decision that systems producing a voter-verified
paper record are the only kind that will be currently supplied in the
standard, and the innovation class will provide a way to path for certifying
other kinds of voting systems. After discussion it was agreed that this
requirement would NOT be included. VVPR is included as a "may"
requirement for meeting software independence so that it leaves room
for other means of meeting SI for innovation class designs. It must
be made clear that any system in the innovation class must meet all
requirements in the VVSG and that SI must be met, but it can be met
in a new way. The proposal was to add a discussion to the requirement
to say the intent of the requirement is that all non innovation class
submissions must comply with VVPR but innovation class submissions can
meet the requirement of SI by other means. David Flater suggested adding
the caveat that any other way they system claims to be SI will be put
under intent scrutiny to make sure it meets SI.
STS Meeting Tuesday, June 19th.
adjourned at 12:30 p.m.
to the Help America Vote Act of 2002, the TGDC is charged with directing
NIST in performing voting systems research so that the TGDC can fulfill
its role of recommending technical standards for voting equipment to the
EAC. This teleconference discussion served the purposes of the CRT subcommittee
of the TGDC to direct NIST staff and coordinate its voting-related research
relevant to the VVSG 2007. Discussions on this telecon are preliminary,
pre-decisional and do not necessarily reflect the views of NIST or the
to NIST HAVA Page
Last updated: July 25, 2007
Point of Contact
policy / security notice / accessibility statement
NIST is an agency of the U.S. Commerce Department