Human Factors Test Suite Version 2.0-1 for the Usability, Accessibility, and Privacy Requirements of the VVSG-NI, Version 2.0

Notes to Reviewers

April 1, 2009

This represents a test suite for the human factors requirements in Part 1 Section 3, Usability, Accessibility, and Privacy Requirements, of the next iteration of the Voluntary Voting System Guidelines (VVSG-NI). When the VVSG-NI is approved by the Election Assistance Commission (EAC), the test suites will be available for use by voting system testing laboratories as a common basis for testing voting systems to determine conformance to the VVSG-NI.

Test suite reviewers are advised to first read and understand the VVSG-NI, especially the sections relevant to the test suites under review, before reviewing the test suites. Part 1 Section 3 can be found here. A complete version of the VVSG-NI in HTML, MS-Word, or PDF formats can be found here. Additionally, the human factors test suite is issued in HTML and contains links troughout to VVSG-NI requirements corresponding to the tests.


Please send comments on the test suites, by July 1, 2009, to:

You may provide comments directly in your email and/or send attachments in MS-Word or PDF. In general, please tell us the features you like and provide us with comments, corrections, and suggestions on how to improve the test suites. Please provide the following items:

If including suggestions for changes to the tests, a description of the suggested change including an adequate justification for the change, or a draft replacement for the test including the justification and any other necessary documentation or commentary All comments will be considered. After all comments have been received and incorporated into the test suites, a new version of the test suites will be posted on the NIST web site.

This document and associated files have been prepared by the National Institute of Standards and Technology (NIST) and represent draft test materials for the Election Assistance Commission's next iteration of the VVSG. It is a preliminary draft and does not represent a consensus view or recommendation from NIST, nor does it represent any policy positions of NIST.



1 Background of VVSG Testing
2 Structure of this Document
3 Tester Qualifications
4 Measurement Equipment Needed
5 General Rules and Background Assumptions for Testing
5.1 Rules for All VVSG Testing
5.2 Rules Specifically for Usability Testing

VVSG Requirements - TOC

3.2 General Usability Requirements
3.2.1 Performance Requirements Overall Performance Metrics Manufacturer Testing
3.2.2 Functional Capabilities Editable Interfaces Non-Editable Interfaces
3.2.3 Privacy Privacy at the Polls No Recording of Alternative Format Usage
3.2.4 Cognitive Issues
3.2.5 Perceptual Issues
3.2.6 Interaction Issues Timing Issues
3.2.7 Alternative Languages
3.2.8 Usability for Poll Workers Operation Safety
3.3 Accessibility Requirements
3.3.1 General
3.3.2 Low Vision
3.3.3 Blindness
3.3.4 Dexterity
3.3.5 Mobility Controls within Reach
3.3.6 Hearing
3.3.7 Cognition
3.3.8 English Proficiency
3.3.9 Speech

Combined Requirement Test Methods

Voting Performance Protocol (VPP)

Usability Testing by Manufacturer

Editable Ballot Session

Non-Editable Ballot Session

Privacy of Voting Session

Privacy of Cast Vote Record (CVR)

Language Clarity

Ballot Design

Default Characteristics

Font Characteristics

Use of Color

Scrolling and Feedback

Accidental Activation

Response Time

Inactivity Time

Alternative Languages

Operational Usability for Poll Workers

End-to-end Accessibility

Accessible Ballot Verification and Submission

Partial Vision

Audio-Tactile Interface

Audio Volume

Non-Manual Operation