|
Technical
Guidelines Development Committee (TGDC) AGENDA
Attendees: Alexia Scott-Morrison, Alice Miller, Allan Eustis, David Baquis (U.S. Access Board), Elle Colver (EAC), John Cugini, John Wack, Mat Masterson (EAC), Nelson Hastings, NIST Statisticians, Sharon Laskowski, Wendy Havens, Whitney Quesenbery Administrative Updates (Allan Eustis):
Significant changes to Usability, Accessibility, and Privacy since the May 2007 TGDC Meeting (John Cugini):
Benchmarks White Paper (Sharon Laskowski): Sharon reviewed in detail the benchmarks white paper - a final draft of the paper should be available by next Wednesday. This paper will be presented at the July 3rd plenary meeting. The goal of the white paper is to be able to present to the TGDC and EAC, in plain language, the overview and basic understanding of the usability benchmarks. The white paper summarizes the research conducted to develop the test, metrics, and benchmarks in response to a TGDC resolution and describes the resulting performance-based requirements included in the current draft of the VVSG. Supporting documentation of test data and test materials are included appendices. The report defines the test: the ballot and the instructions used and the test participants. It defines how the system will be measured: effectiveness, efficiency, and satisfaction (efficiency and satisfaction numbers will be reported but will not be used as benchmarks to pass/fail a system). The paper describes tests used to validate the results and processes used to analyze the results. The paper also explains what was done to show that we could get repeatable results from the test protocol. Sharon then explained how the benchmarks were set, explaining that the confidence interval was a repeatability measure. The research described in this white paper shows the validity and repeatability of our testing methodology and allowed HFP to determine benchmarks and specify usability performance requirements for voting systems. This methodology is technology-independent. Any system can be tested to these benchmarks. The current explanatory white paper still needs to be shortened to about 5 pages before next week. Sharon has begun working on a more technical document that will provide a thorough analysis and will hopefully be ready when the VVSG is delivered to the EAC. Anyone wishing to analyze the data to review how the benchmarks were obtained will be able to do so using the raw data that will be made available. After the VVSG is delivered to the EAC, time will be spent researching test methods. Tests have to be run to get reproducibility by the test labs. The flexibility of the test protocol has to be determined -- to ensure that this methodology can be reproduced in any VSTL, we will be performing research to determine how precisely the test conditions must be followed as part of our work on VVSG test method development. It needs to be determined how test methods are presented to the test labs. Other: It was proposed and accepted that at the next HFP subcommittee meeting members of the STS subcommittee would give an overview of the materials in the STS chapters (Chps 4-12). Whitney feels that a pre-brief will better help members get prepared for the July 3rd meeting. It was reminded that the CRT subcommittee has posted a change log <http://vote.nist.gov/TGDC/crt/index.html> for review. NEXT HFP MEETING: Friday, June 29, 2007. Meeting adjourned at 12:00 p.m. [* Pursuant to the Help America Vote Act of 2002, the TGDC is charged with directing NIST in performing voting systems research so that the TGDC can fulfill its role of recommending technical standards for voting equipment to the EAC. This teleconference discussion serves the purposes of the HFP subcommittee of the TGDC to direct NIST staff and coordinate its voting-related research relevant to the VVSG 2007. Discussions on this telecon are preliminary and do not necessarily reflect the views of NIST or the TGDC.]
**************** Link
to NIST HAVA Page Last updated: July 25, 2007 Privacy
policy / security notice / accessibility statement |