Official websites use .gov
A .gov website belongs to an official government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.
DRAFT Test Assertions for VVSG 1.0 Section 3.1.1 (Tabular Format)
[Tabular Format]
April 18, 2014
3.1.1 The vendor shall conduct summative usability tests on the voting system using individuals representative of the general population. The vendor shall document the testing performed and report the test results using the Common Industry Format. This documentation shall be included in the Technical Data Package submitted to the EAC for national certification. | TA311-1 The manufacturer SHALL conduct realistic summative usability tests on the portion of the voting system used by the voter during the voting session that does not include accessibility features. |
| TA311-2 The summative usability tests SHALL be performed upon a completely functioning ("finalized") product. | |
| TA311-3 The summative usability tests SHALL use individuals representative of the population under test not requiring use of accessibility features (as required in section 3.2, VVSG 1.0). | |
| TA311-3-1 The population under test SHALL consist of a mix of voters including users of different ages, genders, ethnicities, levels of education, voting experience, and geography. | |
| TA311-3-2 The population under test SHALL consist of a mix of voters who are eligible to vote in the U.S. | |
| TA311-3-3 The population under test SHALL NOT consist of voters who are, or have been, a poll worker, a voting machine manufacturer, a voting machine developer, in the marketing or sales of voting systems, or involved in any other position that is part of the voting process. | |
| TA311-3-4 The population under test SHALL NOT consist of voters who are involved with a usability or market research business/company. | |
| TA311-4 The manufacturer SHALL report the total number of participants tested and demographics of the participants. | |
| TA311-5 The manufacturer SHOULD describe their recruiting strategy. | |
| TA311-6 The manufacturer SHOULD detail any compensation given to participants. | |
| TA311-7 The manufacturer SHALL describe how the voters were screened and selected. | |
| TA311-8 The manufacturer SHOULD note any differences between the users profiled as recruits and the users who participated in the actual study. | |
| TA311-9 The manufacturer SHALL ensure that at least eight test participants are able to complete the testing session. | |
| TA311-10 The manufacturer SHOULD initially target at least 10 - 12 participants in order to ensure that at least 8 individuals are able to complete the testing sessions. | |
| TA311-11 The manufacturer SHOULD ensure that at least 30 test participants are able to complete the testing session. | |
| TA311-12 The manufacturer SHOULD include detailed tables of participant demographics as an appendix to the test report. | |
| TA311-13 The manufacturer SHALL document the test results using the Common Industry Format. | |
| TA311-14 The manufacturer SHALL report the test results using the Common Industry Format. | |
| TA311-15 The manufacturer SHOULD use the Modified CIF Template for manufacturers as a template and guidance for the semantics, content and testing. | |
| TA311-16 Manufacturers MAY define their own testing protocols for the summative usability tests. | |
| TA311-17 The Technical Data Package submitted to the EAC for national certification SHALL contain the summative Usability Test Report. | |
| TA311-18 The summative usability tests SHALL measure and report metrics for efficiency, effectiveness, and satisfaction as defined in the ISO/CIF standard. | |
| TA311-19 The summative usability tests conducted by the manufacturer SHALL be realistic and SHALL apply to the development of the test ballot and test script, voting operations and the testing environment. | |
| TA311-20 The test ballot used in the summative usability tests SHOULD look like a real ballot. | |
| TA311-21 The test ballot used in the summative usability tests SHOULD be of medium complexity. | |
| TA311-22 The test ballot used in the summative usability tests SHOULD enable testing of all functionality for the voter interface under test. | |
| TA311-23 The summative usability tests conducted by the manufacturer SHOULD use the NIST test ballot. | |
| TA311-24 The manufacturer SHALL ensure that the summative usability documentation/report is reported in the TDP and conforms to the formatting and content requirements of the Common Industry Format (CIF). |
Created April 30, 2014, Updated August 25, 2016