U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Information Technology Laboratory

Voting

Human Factors Public Working Group

Share

READ-ONLY SITE MATERIALS: Historical voting TWiki site (2015-2020) ARCHIVED from https://collaborate.nist.gov/voting/bin/view/Voting

Purpose

This working group is for the discussion and development of guidance for voting system human factors-related issues, including for accessibility and usability. The guidance will inform the development of requirements for the Election Assistance Commission (EAC) Voluntary Voting System Guidelines (VVSG).

Objectives

  • Identify gaps/issues with current accessibility and usability requirements for voting including priority gaps/issues that are clearly within the VVSG scope.
  • Collect resource materials and develop recommendations to address priority gaps/issues.
  • Ensure consideration and implementation of accessibility and usability in all election processes identified by other working groups.
  • Ensure integrated and balanced consideration of accessibility and usability within the work of cyber security, interoperability, and testing constituency groups.

VVSG 2.0 Human Factors Deliverables

DRAFT User-Centered Design/Usability Testing Guidance and Templates

All-UCD-UTEST-2018-12-14.zip

DRAFT VVSG 2.0 Requirements Guidance

DRAFT_VVSG_2.0_REQ_GUIDANCE_2019-6-20.zip

Supporting Reference Material

Surveys and Reports on Voting Accessibility Issues

  1. "Disability, Voter Turnout, and Voting Difficulties in the 2012 Elections" report by Lisa Schur and Doug Kruse, Rutgers University

  2. "Defining the Barriers to Political Participation for Individuals with Disabilities" report by Thad E. Hall and R. Michael Alvarez http://elections.itif.org/reports/AVTI-001-Hall-Alvarez-2012.pdf
  3. "Experience of Voters with Disabilities in the 2012 Election Cycle", National Council on Disability, October, 2013 
  4. "The Blind Voter Experience: A Comparison of the 2008, 2012, and 2014 Elections", National Federation of the Blind, December, 2014 https://nfb.org/images/nfb/documents/word/2014blindvotersurvey.docx
  5. "The Experience of Blind and Visually Impaired Voters Who Used Online Ballot Delivery Systems during the November 2014 General Election", National Federation of the Blind, January, 2015 https://nfb.org/images/nfb/documents/word/2014blindvoteronlineballotdeliverysystemsurvey.docx
  6. "Voting Access Barriers: 2016 Survey & Webinar Discussion - Priority Issues" See attachment at the bottom of this web page.
  7. Voter Registration
  8. Vote by Mail

White papers

  1. 2011 TGDC Voluntary High-Level Goals for Remote Electronic Voting Systems http://www.nist.gov/itl/vote/upload/VoluntaryHigh-LevelGoalsForRemoteElectronicVotingSystems-final.doc

  2. 2011 TGDC Usability and Accessibility Working Group White Paper on VVSG 2.0 and Beyond: Issues and Gaps in the Usability Accessibility Requirements http://www.nist.gov/itl/vote/upload/VVSG-2-0-UA-issues-and-gaps-12-21-10-final-3.doc
  3. A roadmap for future usability and accessibility guidance (DRAFT) NIST and Center for Civic Design http://civicdesign.org/projects/roadmap/
  4. Principles for Remote Ballot Marking Systems (DRAFT) NIST and Center for Civic Design http://civicdesign.org/projects/remote-ballot-marking/
  5. Exploring a revised structure for Usability and Accessibility in the VVSG (DRAFT) NIST and Center for Civic Design

Research

  1. Accessible Voting Technology Portal http://www.nist.gov/itl/vote/accessiblevoting/index.cfm
  2. EAC-funded Accessible Voting Technology Initiative (AVTI) <https://www.eac.gov/payments-and-grants/accessible-voting-technology-initiative> July 2015 TGDC presentation on AVTI http://www.nist.gov/itl/vote/upload/Dawkins-TGDC-EAC-AVTI-R-D-Final-web.pdf

Info about usability testing/testing with users for VVSG

  1. Modified CIF Template For Voting System Manufacturers for reporting results of user testing http://www.nist.gov/itl/vote/upload/CIF_Template_Manufacturers-VVSG-1-1.docx

  2. Guidelines on How to Complete the Modified CIF Template for Voting Manufacturers http://www.nist.gov/itl/vote/upload/Guidelines_CIF_Template_Manufacturers-VVSG-1-1.doc

Logistics

Bi-weekly teleconferences to solicit and discuss technical issues for the development of the next iteration of the Voluntary Voting Systems Guidelines begin on Thursday, May 19, 2016, 4:00PM EDT / 1:00PM PDT (1 hour).

Dial: 877-972-8541, Access Code: 9771600

We may have a large number of people on each call, so to make things move along efficiently, please:

  • DO NOT announce your name when joining (we will do a round of introductions)
  • Keep your phone on MUTE at all times unless you are speaking
  • DO NOT put the phone connection on hold, which can cause music to play that makes discussion impossible, resulting in shutdown of the call and redialing

If you cannot attend, you can email the HFCG mailing list at vvsg-human-factors [at] nist.gov (vvsg-human-factors[at]nist[dot]gov) or sharon.laskowski [at] nist.gov (sharon[dot]laskowski[at]nist[dot]gov) with your comments relevant to the agenda. We will also archive information on this twiki.

Next Meetings

Biweeky, 4PM EDT/1PM PDT; 877-972-8541, Access Code: 9771600

Meeting Archive

Thursday, June 2, 2016, 4PM EDT/1PM PDT

Goal for this call: Plot course for future calls: identify and prioritize areas where U&A guidance is needed

Agenda:

  1. Introductions (5 minutes)
  2. Update on scope discussion: the EAC is aware of your interest in including standards for electronic pollbooks and remote ballot marking interfaces (5 minutes)
  3. Biweekly meeting goals (3 minutes)
    1. To refine structure for VVSG U&A standard
    2. To identify gaps in previous requirements, and opportunities for new guidance
    3. To address those gaps
  4. Structure for U&A (Principles & Guidelines document attached)
    1. Seeking input on core guidance and system-specific guidance
      1. Core U&A guidance (15 minutes)
  • E.g., color, contrast, font, plain language, plain interaction
  • Universal design (i.e. adjustable/customizable interfaces)
  • Ubiquitous across devices and interfaces
  • Widely accepted usability design
  • Baseline accessibility requirements
  • Includes WCAG as appropriate
  • Voting specific core guidance (5 minutes)
  • Ballot navigation
  • Overvote/undervote handling
  • Review of ballot selections, etc.
  • System-specific guidance (20 minutes)
  • (Based on today’s call we will expand this list and send it to the group for feedback via email.)
  • Devices, Interfaces, interactions
    • Small form factor (e.g., ballot content amount, ballot layout, minimum screen size)
    • Screen orientation (portrait or landscape)
    • Electronic forms (web-based and pdf)
    • Ballot selection representation (e.g., QR code, optical character recognition (OCR))
    • Touchscreen gestures (e.g., navigation, zoom, coordination with screen readers)
    • System feedback (e.g., mobile device vibrations)
    • Personal assistive technology (PAT) (e.g., screen readers, braille readers)>Onscreen keyboard use
    • Magnification and zoom
  • Meta topics: (2 minutes)
  • Over the course of these calls, let’s consider how our vision of the polling place has changed
  • Wrap-up (5 minutes)
  • June 16—update and review gaps and opportunities and prioritize discussion topics
  • June 30—begin topic discussions
Thursday, May 19, 2016 4PM EDT/1PM PDT

Goal for this call: Discuss scope and general approach

Agenda:

  1. Introductions
  2. How these calls will be structured: each call will have a set of issues and questions for participants. Specific topics will be identified for each call. For the next 2 calls we will choose some topics
  3. Brief overview of approach to developing next set of usability and accessibility requirements (see draft attachment at bottom of this page: “Developing Standards for Emerging Voting System Technologies U&A”)
  4. Discussion to help determine overall approach to new VVSG:
    • Scope: Voting systems and any equipment that “touches” the voting system in the polling place. For this group, it is where the voters, pollworkers, and election officials interact with the system.
    • HAVA and VVSG 1.1 call for at least one “accessible” machine per polling place. Currently, this often results in one machine in the corner that pollworkers might not be very familiar with and few voters realize that they could benefit from the accessibility features. We plan to write requirements that make a broader assumption. We would assume that any electronic system in the polling place is designed for all voters, accessible to the greatest extent possible. This would also simplify the organization of the usability and accessibility requirements as they would apply to all electronic voting systems in the polling place.
    • Accessibility standards such as the WCAG 2.0 are more mature than 5-10 years ago. We plan to refer to these in the VVSG. We are thinking about citing them directly. For any web application, WCAG 2.0 applies. We can do more than just cite WCAG in general, we could also look at how they apply in a voting specific context and also point out where they might be more stringent for voting, e.g., high levels of contrast.
    • Emerging technology (mobile devices) and voting research has resulted in numerous possibilities about where and how a voter can mark and cast their ballot. For example, a voter could use personal assistive technology to mark their ballot at home and mail it in or take the marked ballot to a polling place. The marked “ballot” itself could be just a list of selections and/or a QR code. Should we include these other models of voting in the VVSG? How? We are trying to figure out how to include standards for emerging technologies.
Kickoff meeting: FRIDAY JANUARY 29 at 1PM EST/10AM PST

Agenda:

  1. Introductions (25 participants)
  2. EAC TGDC meeting Feb. 8-9
  3. Role of this public working group (PWG)
    1. Constituency group to provide accessibility and usability input to Pre-election/Election/Post-election PWGs
      1. i. Process model being created by above election cycle PWGs
    2. Scope of next generation VVSG
  4. Issues for next generation VVSG
    1. Accessibility for all systems, no separate “accessible system”
    2. Use of existing standards (e.g. WCAG 2.0)
    3. Monolithic set of requirements vs. component-based subsets
    4. Outside of the polling place, personal devices, personal assistive technology, remote ballot marking
    5. Structure and usability of the VVSG itself (e.g. set of core requirements, high level principles)
  5. http://vote.nist.gov for more information
  6. Human Factors

Officers

Chair

Name: Diane Golden

Email: diane.golden [at] ataporg.org (diane[dot]golden[at]ataporg[dot]org)

Affiliation: Association of Assistive Technology Act Programs

Agency Lead

Name: Sharon Laskowski

Email: sharon.laskowski [at] nist.gov (sharon[dot]laskowski[at]nist[dot]gov)

Affiliation: NIST

Members

Membership Information:

Participation in this Working Group is open to all interested parties. There are no membership fees.

  • List of mailing list members (TBD)

Latest Working Documents

Enter information about latest working documents here.

Resources

Enter information about important resources here such as meeting rooms, teleconferencing numbers, access codes, URLs, etc.

Email List

Email List Name: vvsg-human-factors

To join the list or find more information about list policies and related procedures, please visit the VVSG Working Group Lists page.

Relevant Documents

Reference relevant documents here.

Charter:

Definition:

Policies and Procedures:

Reference Architecture:

Work To-Date:

Technology Roadmap:

Vocabulary:

Publications:

 

Attachment

Date

Comment

2-Principles-and-Guidelines.docx

2016-06-01 - 20:45

 

2016_Voting_Access_Barriers_Survey_Summary.docx

2016-02-05 - 18:57

 

All-UCD-UTEST-2018-12-14.zip

2019-06-20 - 16:11

 

Ballot_secrecy_requirements_from_HF_gap_analysis.docx

2017-11-09 - 16:03

 

DRAFT_VVSG_2.0_REQ_GUIDANCE_2019-6-20.zip

2019-06-20 - 17:10

 

Developing_Standards_for_Emerging_Voting_System_Technologies__UA.pdf

2016-05-13 - 18:53

Overview of approach to developing next set of usability and accessibility requirements

NIST_HF_PWG_meeting__2017-10-05.docx

2017-10-19 - 17:44

 

NIST_HF_PWG_meeting__2017-10-19-final.docx

2017-10-25 - 15:17

 

NIST_HF_PWG_meeting__2017-11-16-final.docx

2017-11-28 - 20:15

 

NIST_HF_PWG_meeting__2017-11-2_final.docx

2017-11-09 - 16:03

 

NIST_HF_PWG_meeting__2017-11-30.docx

2017-12-12 - 20:23

 

NIST_HF_PWG_meeting__2017-12-14_final.docx

2018-01-10 - 21:05

 

NIST_HF_PWG_meeting__2018-1-11_final.docx

2018-01-22 - 15:28

 

NIST_HF_PWG_meeting__2018-1-25-final.docx

2018-02-08 - 18:10

 

NIST_HF_PWG_meeting__2018-2-8-final.docx

2018-02-20 - 21:32

 

NIST_HF_PWG_meeting__2018-3-8-final.docx

2018-06-12 - 18:04

 

NIST_HF_PWG_meeting__2018-4-5-final.docx

2018-06-12 - 20:07

 

NIST_HF_PWG_meeting_notes_20170907-final.docx

2017-10-19 - 17:44

 

NIST_HF_PWG_meeting_notes__20170824-final.docx

2017-10-19 - 17:44

 

VVSG2-HFP-ALL-18-2-6a.zip

2018-02-08 - 18:10

 

VVSG_2-Glossary-HFP_only-18-01-16a.docx

2018-01-25 - 21:28

 

VVSG_2-GuidanceBrief-Plain_Language-draft-18-01-16.docx

2018-01-25 - 21:27

 

VVSG_2-GuidanceBrief-Plain_Language-draft-18-01-30pwg.docx

2018-02-12 - 20:52

 

VVSG_2-GuidanceBrief-Text_Size-draft-18-01-16.docx

2018-01-25 - 21:27

 

VVSG_2-GuidanceBrief-Text_Size-draft-18-01-27pwg.docx

2018-02-12 - 20:52

 

VVSG_2-HFP-Requirements_Summary_for_Commments.docx

2018-02-20 - 21:32

 

VVSG_2-Principle_2.2-draft-18-1-2_PWG.docx

2018-01-10 - 21:05

 

VVSG_2-Principle_5.1-draft_17-10-17.docx

2017-10-26 - 16:29

 

VVSG_2-Principle_5.1-draft_17-10-24.docx

2017-10-26 - 16:31

 

VVSG_2-Principle_5.2-draft_17-10-17.docx

2017-10-19 - 17:28

Draft VVSG requirements

VVSG_2-Principle_5.2-draft_17-10-24.docx

2017-10-26 - 16:24

 

VVSG_2-Principle_6.1-draft_17-10-30-pwg.docx

2017-11-03 - 19:25

 

VVSG_2-Principle_6.1-draft_17-11-9-PWG.docx

2017-11-09 - 16:03

 

VVSG_2-Principle_6.2-draft_17-10-29-pwg.docx

2017-11-09 - 16:03

 

VVSG_2-Principle_7.1-draft_17-11-15-pwg.docx

2017-11-15 - 20:23

 

VVSG_2-Principle_7.1-draft_17-11-27-pwg.docx

2017-11-28 - 20:15

 

VVSG_2-Principle_7.2-draft_17-11-27-pwg.docx

2017-11-28 - 20:15

 

VVSG_2-Principle_7.2-draft_17-12-11.docx

2017-12-12 - 20:23

 

VVSG_2-Principle_7.3-draft-17-12-12_PWG.docx

2017-12-12 - 20:23

 

VVSG_2-Principle_7.3-draft-17-12-19_PWG.docx

2018-01-10 - 21:05

 

VVSG_2-Principle_8.1-draft-18-1-16_PWG.docx

2018-01-22 - 15:28

 

VVSG_2-Principle_8.1-draft-18-1-2__PWG.docx

2018-01-11 - 15:37

 

VVSG_2-Principle_8.2-4-draft-18-1-2_PWG.docx

2018-01-10 - 21:05

 

VVSG_HF_20160519mtgSummary.docx

2016-06-01 - 20:45

 

 

Voting TWiki Archive (2015-2020): read-only, archived wiki site, National Institute of Standards and Technology (NIST)

ARCHIVE SITE DESCRIPTION AND DISCLAIMER

This page, and related pages, represent archived materials (pages, documents, links, and content) that were produced and/or provided by members of public working groups engaged in collaborative activities to support the development of the Voluntary Voting System Guidelines (VVSG) 2.0. These TWiki activities began in 2015 and continued until early 2020. During that time period, this content was hosted on a Voting TWiki site. That TWiki site was decommissioned in 2020 due to technology migration needs. The TWiki activities that generated this content ceased to operate actively through the TWiki at the time the draft VVSG 2.0 was released, in February of 2020. The historical pages and documents produced there have been archived now in read-only, static form.

  • The archived materials of this TWiki (including pages, documents, links, content) are provided for historical purposes only.
  • They are not actively maintained.
  • They are provided "as is" as a public service.
  • They represent the "work in progress" efforts of a community of volunteer members of public working groups collaborating from late 2015 to February of 2020.
  • These archived materials do not necessarily represent official or peer-reviewed NIST documents nor do they necessarily represent official views or statements of NIST.
  • Unless otherwise stated these materials should be treated as historical, pre-decisional, artifacts of public working group activities only.
  • NIST MAKES NO WARRANTY OF ANY KIND, EXPRESS, IMPLIED OR STATUTORY, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT AND DATA ACCURACY.
  • NIST does not warrant or make any representations regarding the correctness, accuracy, reliability or usefulness of the archived materials.

ARCHIVED VOTING TWIKI SITE MATERIALS

This wiki was a collaborative website. NIST does not necessarily endorse the views expressed, or concur with the facts presented on these archived TWiki materials. Further, NIST does not endorse any commercial products that may be mentioned in these materials. Archived material on this TWiki site is made available to interested parties for informational and research purposes. Materials were contributed by Participants with the understanding that all contributed material would be publicly available.  Contributions were made by Participants with the understanding that that no copyright or patent right shall be deemed to have been waived by such contribution or disclosure. Any data or information provided is for illustrative purposes only, and does not imply a validation of results by NIST. By selecting external links, users of these materials will be leaving NIST webspace. Links to other websites were provided because they may have information that would be of interest to readers of this TWiki. No inferences should be drawn on account of other sites being referenced, or not referenced, from this page or these materials. There may be other websites or references that are more appropriate for a particular reader's purpose.

 

Voting systems
Created August 28, 2020, Updated February 5, 2021