Joint Technical Guidelines Development Committee (TGDC)
Human Factors and Privacy Subcommittee (HFP) &
Security and Transparency (STS) Teleconference *
February 9, 2007
Meeting Minutes

Participants: Secretary Gale, Allan Eustis, Wendy Havens, Benjamin Long, Barbara Guttman, Nelson Hastings, John Kelsey, John Cugini, David Flater, David Baquis (U.S. Access Board), Philip Pearce, Whitney Quesenbery, John Wack, Tricia Mason, Ron Rivest, David Wagner, Sharon Turner Buie


1. Administrative Updates (Allan Eustis and John Wack)

2. Software Independence and Accessibility for the voter (Sharon and Nelson) -- Review of Initial white papers (We also need a better term than "end to end" accessibility, such as "complementary accessibility".)

3. Software Independence and implications to usability of audits (Sharon and Nelson) SI without innovation classes imply paper records. Is there anything we can do to improve the usability of audits of the paper?

4. Other issues (Safety -- need to hand off to safety expert?)

5. Next meeting topic: usability benchmark paper

[Next HFP teleconference is scheduled for: Friday, February 23, 2007 at 11 AM ET]


[* Pursuant to the Help America Vote Act of 2002, the TGDC is charged with directing NIST in performing voting systems research so that the TGDC can fulfill its role of recommending technical standards for voting equipment to the EAC. This teleconference discussion served the purposes of the STS and HFP subcommittees of the TGDC to direct NIST staff and coordinate its voting-related research relevant to the VVSG 2007. Discussions on this telecom were preliminary and do not necessarily reflect the views of NIST or the TGDC.]

Administrative Updates: 2/9/07 Testimony to the EAC by David Alderman of NIST on NVLAP accreditation progress has been posted to:

Software Independence and Accessibility for the voter: Whitney mentioned that starting point for this discussion has been that all aspects of the voting process need to be accessible or accommodated in an appropriate way. "Complimentary accessibility" seems a better term than "end to end" and avoids confusion with STS cryptographic use of the term "end to end". Ron agreed that verification for all voters sits at the intersection of security and usability. STS is looking for guidance and collaboration from HFP on the right path forward. Whitney pointed out we need to determine exactly what we are verifying.

(1) You need to be able to audit the election.
(2) Voter needs to be able to review ballot.
(3) Voter needs to be able verify vote is counted.

We are focusing on (2). John Cugini offered 4 possible technical approaches/solutions for discussion. He stressed that SI solutions are oriented towards blind voters. These are not mutually exclusive:

(a) Use audio as is (review of audio ballot sufficient)
(b) Introduce paper into process (such as EBM or OCR) and have device read back audio
(c) Use of audio tape to directly record what the voter is hearing
(d) Innovation class solution (E.g. independent "witness" system like Scytel reads back and records vote )

(a), (c), and (d) are paperless solutions. All have pluses and minuses. Some are not ready for prime time.

Discussion initiated by Ron Rivest on what of the four truly meet software independence. Tampering could occur with (a). Demanding SI for all voters may not feasible.(i.e. (a) may still be acceptable; (b) meets independent verification (IV) requirement. It does require an independent reader device; (c) also in (IV) class.

Dave Wagner reviewed his understanding of four technical approaches/solutions. First three appeared SI to him. (d) did not appear as SI. Whitney also reviewed her understanding of the four technical solutions from accessibility stand point and production of durable records.

Philip Pearce asked if any of the approaches violated private and independent verification requirement for all voters. John Kelsey said they all are private but the main issue is security of each approach. Whitney agreed and reviewed independence of each method; Issues of sampling size with some approaches. Philip Pearce posed question of how much responsibility we have to decide the "best" method. What are sufficient techniques?

David Baquis added four different challenges that need to be considered in this matrix:

1) those with low vision that do not routinely use audio
2) those with dexterity/mobility disabilities
3) those with cognitive disabilities

Whitney referenced a recent presentation at on cognitive disabilities presented at recent access board meeting. (See: Point here is that cognitive disabilities are on a spectrum with respect to barriers. We should not gloss over usability issues.

Whitney and participants reviewed the approaches (a)-(c) for each of the disabilities brought up by David Baquis. A magnifier could be used for those with low vision who do not use audio support routinely.

John Kelsey synthesized issue as a determination what appears on the screen and does not make it on to the papers that are relevant to the vote the voter he/she was casting. Also there are issues here related to alternative languages. Issue here is "one ballot versus many ballots "of alternative language ballots on a specific machine affecting privacy of vote. There are election procedures here to mitigate the loss of voter privacy.

David Wagner had concerns with co-mingling of ballots. Sharon Turner Buie pointed out that paper ballots are co mingled as they are read into the machine. Concern here by Wagner of one Chinese voter at that polling place. Turner Buie pointed out that ballot is tabulated irrespective of the language used to vote. (John Gale agreed.) Only in a recount would you possibly be able to detect the voter's identity. (Wagner expressed ongoing concerns related to privacy issues.)

Quesenbery brought up dexterity issue and transporting paper as a barrier to independence requirement for the voter. Scanner device was discussed. Does assistance with the paper ballot remove independence for the voter? We will get push back here.

Secretary Gale noted that state and local election officials operate a voting system, not just voting equipment. Trained election officials and poll workers operate the equipment within a system context. Election officials do their independent testing and assessment of the equipment to ensure that it is working accurately and effectively before an election. You do not want to discuss these issues in a vacuum separate from an election system. Our focus here seems to be to drill down to require the individual voter to do the assessment and evaluation. Meanwhile studies indicate that 30% or less of the voters verify their vote. If the average voter has trust in the certification and verification of the voting system, they are going to accept probability that the system is working correctly. That is they do not need to be responsible for reading the installation manuals etc. An analogy here: when we fly a commercial airplane we trust the system. We do not feel the need to read the service manuals or review the qualifications of the pilot because others do that. Are we making voting complicated for the average voter because we do not trust the election officials or the voting system to operate correctly? Should we not focus on assuring that the voter has ease of access and has a sense of confidence in the casting of their ballot? If we don't trust the software, then we should have independent audit verification of random precincts that use DRE equipment with VVPATs. Those selected precincts are going to have to count the VVPATs and compare them with the electronic result. That way, election officials are addressing the verification issues for all the voters in the precinct without passing the responsibility on to the individual voter.

Rivest agreed that the voter verification step is important from a security viewpoint if it is done by a sample of the voters. The idea of a random audit of the precincts is a good one especially with VVPATs.

Philip Pearce commented that, in our discussion, we should be asking the question for whom the voting equipment is working accurately and appropriately. Is it allowing voters, whenever they wish to do so, to ensure that the equipment allows for private and independent verification of their votes?

Secretary Gale agreed with this assessment, and commented that the voter had to have trust in the system's checks and balances. We seem to be in new arena of voter responsibility independent from the system. We should also realize that no equipment is going to be perfect just as no election is perfect. You have to rely on the system as a safety net back up.

Wagner noted that we are talking about voting systems producing non electronic records that will be used for permanent records and allowing voters to verify their votes. We want to eliminate barriers to the disabled voters' ability to verify his/her vote. However we are not giving the voter the right to follow the vote through every step in the process. We simply want to preserve the security.

Summarizing, Quesenbery commented that elections are indeed complex. We are not looking for full accessibility of the process; we are looking for equivalence. If the paper ballot is the ballot of record, we are looking for means to ensure that all voters can review that ballot. We do not require that all systems operate in the same way. We look at how voting system architectures satisfy the requirements for all voters. This makes it difficult to write guidelines to ensure that voters with disabilities can fully participate. The next step requires an answer to the question of what makes the ballot auditable. For example there are definite challenges in auditing an audio tape.

Discussion ensued on (a) thru (d) above relative to whether they satisfied as a sufficient SI technique. Each was reviewed. Rivest provided context for SI in terms of errors that are capable of causing changes in the election outcome without any detection. (Will an audit detect something wrong with some probability?) In (a) a sighted a voter could listen to the audio and determine if it is different than what is printed on the paper. This is an SI capability. Qeusenbery noted that we have to answer two questions. Is the approach sufficient to meet SI and is it sufficient to meet accessibility. (a) Meets accessibility. John Kelsey suggested a formal procedure to encourage (enabled) voters to vote on accessible audio ballot voting systems.

Quesenbery and Cugini determined that VVSG 2007 has requirements that audio and video interfaces work simultaneously. Rivest noted Selker's research indicating that sighted voters who used audio voted more accurately. Cugini cautioned that we are writing an equipment standard and can only recommend procedures.

Concluded that (a) might be SI, but is certainly accessible. (b) Produces a paper ballot. There is a way to read it for blind and low vision (magnify). There are transportation challenges for (a) and (b). The final (c) is accessible and auditable. But the ease of auditability is a factor.

There are issues with the first three. The fourth is still an unknown. We need a chart with all the issues laid out. We have maybe's for SI at this point.

Point is to scope out all cases before presentation and acceptance/rejection by full TGDC. HFP and STS Sub groups will need to agree first; need to get away from the theoretical to the practical. Rivest offered the opinion that (a), (b) and (c) have workable SI solutions. Gale noted complexity with (b) for transportation issues.

Cugini asks that we not downplay (d). There are prototypes of this architecture that help solve (b) issues.

Action item: NIST will document in a list/table format the four technical solutions with auditability and usability concerns clarified. The table will be sent out to STS and HFP for review before next joint meeting.

Software Independence and implications to usability of audits: This will be covered at the next HFP meeting. We will also plan a joint meeting to review auditability issues as well as resolving issues from this telcon.

[Next HFP teleconference is scheduled for: Friday, February 23, 2007 at 11 AM EST]


Teleconferences from 2004, 2005, 2006 and upcoming in 2006.


Link to NIST HAVA Page

Last updated: July 25, 2007
Point of Contact

Privacy policy / security notice / accessibility statement
Disclaimer / FOIA
NIST is an agency of the U.S. Commerce Department