CRT Teleconference
November 30, 2006, 11:00 a.m. EDT

Agenda

1) Administrative updates (Allan E.)

2) Proposed agenda for the December 4-5 TGDC plenary (http://vote.nist.gov/TGDCagenda120406.htm) (Allan E. and John W.)

3) Overview of CRT plenary presentations (Alan G. and David F.)

4) Other Items

Attendees: Allan Eustis, Stephen Berger, Sharon Laskowski, Nelson Hastings, Dvid Flater, Alan Goldfine, Britt Williams, Max Etschmaier, Philip Pearce

The Meeting convened at 11:03 EST

1) Administrative updates (Allan E.)

AE reminded participants of Sunday night informal reception at Gaithersburg Hilton beginning at 7 pm in the Rockville Room.

2) Proposed agenda for the December 4-5 TGDC plenary

The Agenda and meeting materials were sent out on a CD to all members. All updated meeting materials are also available on the web at: http://vote.nist.gov/TGDC/TGDCpresentations120406.htm.


3) Overview of CRT plenary presentations

AG and DF reviewed their presentation slides for the upcoming plenary. See:
http://vote.nist.gov/TGDC/TGDCpresentations120406.htm.

AG noted papers of ME and discussion of MTBF/ alternative accuracy metrics; Need for implementation of quality control program. AG noted an accepted industry quality standard of ISO 9000/9001.

SB brought up concerns of advocating ISO 9000/9001 especially implementation dates and concern over the ability of vendor to what he says he will do; Potential horror stories her. AG will note possibility of this scenario in his talk. Process does need supplementing. SG noted that ISO 9000 adds little value without specifying what should be in quality program.

Participants discussed costs to implement an ISO 900 certification framework. Is this a NISTissue or a policy issue for EAC?

ME agreed that with ISO 9000 vendors also need a quality handbook.

SB brought up linkage between reliability and usability. Is there mistake tolerance convergence? Do we need to handle this?

DF noted source of confusion between reliability (applies to equipment) and usability (applies not to equipment but to operator).

SB wants to make sure requirements cover the potential human error and transposition of races. ME noted the value of functional failure analysis here. DF noted that this is also an issue of system integrity. There are requirements in VVSG 2007 to deal with system tolerance.

DF noted that you want to design system to achieve reliability. Discussion ensued on conflict between reliability benchmarks in theory and in practice.

DF noted inclusion of voting system variations and the arrival at conformity assessment. There was a discussion of open ended testing beyond conformity assessment.

BW initiated discussion of testing of optional features by ITAs. Historically this is not called out in the standards. DF noted that optional features are outside of conformity assessment to the standards.

BW noted that you do not want to have states repeating testing of VVSG done at the Federal level. Discussion continued on "fitness for use" evaluations. Can or should a voting system contain additional features?

DF noted that you will be writing tests to the requirements. BW noted that voting systems should conform to vendor documentation. SB noted that you want the system resistant to unauthorized use.

BW illustrated issues here with VVPAT spools not covered in VSS 2002. The VVPAT systems are tested to the vendor documentation. DF noted the issue of a vendor simply re writing the documentation to conform. AG noted the issue should be raised to EAC.


BW indicated the need of supplementing the test reports with concerns of the lab within the standard requirement r of the test reports. States require this within their reports. DF brought up litigation issues here. He gave an example of a vendor specific feature that met general requirements but misbehaved later in a costly way to the end user. (Good intentions but not conformity assessment). There is a relevant discussion paper in the meeting written materials.

DF noted that California Volume Test will be discussed at the meeting.

4) Other Items

Stephen Berger offered for review three resolutions (below) that he planned to introduce at the December TGDC plenary. AE proposed and SB agrees that me all three resolutions would be forwarded to all TGDC members in advance of the plenary. A copy of the resolutions would also be sent to EAC Commissioner Davidson. No edits were made in the proposed resolutions during the telcon call.

 

U.S. Election Assistance Commission
Technical Guidelines Development Committee

Resolution for consideration by the TGDC at their plenary meeting, December, 2006

Resolution #__-__, Offered by: Stephen Berger

For a variety of historical reasons the voting systems used in the US have developed to be response to the varied needs and desires of state and local election officials and others involved in the selection and use of voting systems. As a result the current systems may be characterized as:

i) Flexible,
ii) Adaptable,
iii) Individual, and therefore Non-Standard modification for jurisdictions, and
iii) Economical.

These characteristics have resulted in a mismatch between the systems and the use environment. Specific areas of contrast:

i) The flexibility and adaptability and the level of technical support and training generally available to jurisdictions to properly configure the systems for use,
ii) Flexibility and adaptability also come in conflict with resistance to human error and mistakes,
iii) Flexibility and adaptability are in conflict with a well designed security strategy and the security generally desired for elections,
iv) The modification of systems for individual states and jurisdictions blocks certain economies of scale, the development of various support tools for use by many jurisdictions and the ability to easily build expertise among all users of a system.
v) Budgetary constraints and pricing pressure have resulted in system quality levels that meet contractual requirements but fall short of general expectations for quality.

In order to assure that the 2007 revision is maximally helpful the NIST staff are requested to study and prepare a report how the proposed changes to the VVSG address these issues and specifically:

i) Result in systems that are appropriate to the level of technical support and training generally available to jurisdictions.
ii) Will significantly reduce the number of human errors and mistakes.
iii) Will result in a system that implements a system strategy that meets specified security threats. This requirement will require specific identification of the threat model being addressed.
iv) Will bring greater uniformity to voting systems and allow development of general use tools in support of election administration.
v) Will raise the system quality level at appropriate points but equally resist adding cost where there is not sufficient value added.


U.S. Election Assistance Commission
Technical Guidelines Development Committee

Resolution for consideration by the TGDC at their plenary meeting, December, 2006

Resolution #__-__, Offered by: Stephen Berger

NIST is requested to prepare a report surveying problems experienced in the 2004 and 2006 elections and analyzing these experiences for trends, causal factors and patterns of problems.

The report should then compare the changes made with the introduction of the 2002, 2005 and proposed change for the 2007 standard.

This report should then answer the following questions:

i) In what ways was the introduction of the 2002 version of the VSG helpful, harmful or irrelevant to addressing the issues identified? Were these impacts due to the standard itself, transitional factors related to its introduction or other simultaneously introduced factors?
ii) In what ways would the 2005 version of the VVSG have been helpful, harmful or irrelevant to addressing the issues identified? What transitional issues can be foreseen for the introduction of the 2005 guidelines?
iii) In what ways would the 2007 version of the VVSG have been helpful, harmful or irrelevant to addressing the issues identified? What transitional issues can be foreseen for the introduction of the 2005 guidelines?


U.S. Election Assistance Commission
Technical Guidelines Development Committee

Resolution for consideration by the TGDC at their plenary meeting, December, 2006

Resolution #__-__, Offered by: Stephen Berger

NIST is requested to prepare a report analyzing the relevance and effectiveness of recent and current proposed changes of the voting system certification process, specifically addressing the role and contribution of the VVSG, to recent election problems. The report should analyze the effect of recent and proposed changes with the purpose of identifying the most effective means of bringing improvement to problems and concerns with current voting systems and election administration.

Meeting adjourned at 12:25 pm EST.

 

 

 

************

Link to NIST HAVA Page

Last updated: July 25, 2007
Point of Contact

Privacy policy / security notice / accessibility statement
Disclaimer / FOIA
NIST is an agency of the U.S. Commerce Department