1 1 TECHNICAL GUIDELINES DEVELOPMENT COMMITTEE (TGDC) MEETING 2 FRIDAY, JULY 9TH, 2004 3 1225 New York Avenue, N.W., Suite 1100 4 WASHINGTON, D.C. 20037-2519 5 6 COMMITTEE MEMBERS: 7 Dr. Arden Bement, Chairman Hon. Donnetta Davidson 8 Alice Miller Sharon Turner-Buie 9 Helen Purcell James Harding 10 James Elekes Anne Caldas 11 H. Stephen Berger Dr. Brittain Williams 12 Paul Craft Dr. Ronald Rivest 13 Dr. Daniel Schutzer (via telephone) Whitney Quesenbery 14 Others: 15 Allan Eustis, Project Leader Voting 16 System Standards Craig Burkhardt, Chief Counsel for Technology 17 U.S. ELECTION ASSISTANCE COMMISSION: 18 DeForest Soaries - Chairman 19 Gracia Hillman - Vice-Chair Ray Martinez 20 Paul Degregorio - Federal Officer for TGDC 21 Taken by: LaDonna M. Woods a court reporter 22 - 0 - 2 1 (The meeting was called to order at 9:05 o'clock, a.m.) 2 CHAIRMAN BEMENT: I'm Arden Bement, Director 3 of the National Institute of Standards and Technology, and 4 Chairman of the Committee of Guidelines and Development 5 Committee. 6 I hereby call to order the first meeting of 7 this Committee today, July 9th, 2004. 8 Public Law 107, the Help America Vote Act, 9 establishes the Technology and Guidelines Development 10 Committee. 11 HAVA charters the members of this Committee 12 to assist the Election Assistance Commission in the 13 development of voluntary voting assistance guidelines. 14 As Chairman, I am appointing Allan Eustis, 15 project director, and Craig Burkhardt, counsel and 16 parliamentarian. 17 Allan currently is Project Leader for NIST's 18 Voting System Standards efforts. 19 Craig is General Counsel for the Department 20 of Commerce's Technology Administration. 21 Both Allan and Craig are here to make sure 22 the Committee operates smoothly. They will provide 3 1 comments later on today. 2 At this time, I'd like to call on 3 Mr. Burkhardt to determine the status of our members, and 4 also, to establish a quorum. 5 MR. BURKHARDT: Thank you, Dr. Bement. I'm 6 Craig Burkhardt, and as Dr. Bement mentioned, I'm the Chief 7 Counsel for Technology matters at the Department of 8 Commerce, and I certainly bring you greetings from the 9 Commerce Secretary, Don Evans, and welcome those of you who 10 have journeyed to Washington, D.C. to be here. 11 I'm on the team, as Dr. Bement will shortly 12 describe to you, to provide advice and support to the TGDC 13 and the EAC on a number of the items that you will be 14 pursuing over the next nine months. 15 There are, in addition, lawyers working 16 directly with the EAC. And one of the tasks which those 17 lawyers had to do was to collect from many of you, a 18 financial disclosure form, which is required by law. 19 I will tell you the efforts that have been 20 made to very quickly distribute those forms, collect them 21 from you, and distribute and to review them, have been 22 extraordinary in their timeliness, given the fact that all 4 1 this has taken place in less than a week, when normally 2 these things can take up to a month or two. 3 I need to announce that there are still yet 4 four people who the review process has not been completed 5 for by the lawyers working with the EAC. Temporarily, they 6 will not be included on the roll calls for various formal 7 motions and votes that will be taken. 8 I will tell you that, actually, as we speak, 9 that review process is ongoing. So during the day, we may 10 be adding one or more of these four people to the roll call 11 officially. 12 The four people who I need to advise should 13 restrain yourself from voting and from actively 14 participating in the conversations today until you are 15 added to the roll call would be Ms. Turner-Buie, 16 Dr. Rivest, Dr. Schutzer, and Mr. Gannon. 17 The rest of you will be called on roll calls 18 and will also help make the quorum. And as the other four 19 are finished in the review process, if you will, I may have 20 announcements from time to time during the day. You are 21 certainly welcome to stay here, observe, and participate in 22 that particular manner. And when we come to the 5 1 self-introductions part of the agenda, you will be 2 encouraged to participate in that portion of the agenda. 3 In order to conduct this meeting, we need to 4 have eight voting members who, of course, have finished the 5 review process. So I am going to go through, and I'm going 6 to read a roll call to establish our quorum. 7 I might also say that when we take votes on 8 motions, which are of significant impact, we will take a 9 roll call, by which Dr. Bement will introduce me, and I 10 will just rapidly go through the roll call, and ask you to 11 cast an "aye" or a "nay" vote for formal recording, because 12 as a federal committee, we have to vote in public. 13 A secret ballot may apply in the voting 14 booth, but it does not apply at the deliberations we'll be 15 engaging in before this Committee. 16 So, for purposes of roll call, I'll say your 17 last names, and please announce by simply saying "present" 18 that you are here. 19 Dr. Bement? 20 CHAIRMAN BEMENT: Present. 21 MR. BURKHARDT: Ms. Davidson? 22 MS. DAVIDSON: Present. 6 1 MR. BURKHARDT: Ms. Miller? 2 MS. MILLER: Present. 3 MR. BURKHARDT: Ms. Purcell? 4 MS. PURCELL: Present. 5 MR. BURKHARDT: Dr. Hardy? 6 DR. HARDY: Present. 7 MR. BURKHARDT: Mr. Elekes? 8 MR. ELEKES: Present. 9 MR. BURKHARDT: Ms. Caldas? 10 MS. CALDAS: Present. 11 MR. BURKHARDT: Mr. Berger? 12 MR. BERGER: Present. 13 MR. BURKHARDT: Dr. Williams? 14 DR. WILLIAMS: Present. 15 MR. BURKHARDT: Mr. Craft? Not present yet. 16 And Ms. Quesenbery? 17 MS. QUESENBERY: Present. 18 MR. BURKHARDT: Dr. Bement, I would advise 19 you that a significant quorum exists for the conduct of 20 business for the Technical Guidelines Development 21 Committee, and you may wish to make that declaration as 22 Chairman. Thank you so much. 7 1 CHAIRMAN BEMENT: Thank you very much. I so 2 declare. 3 First of all, I would like to thank the 4 Commission for making available to us these wonderful 5 facilities and also being present with us this morning as 6 we go through our first meeting. 7 SPEAKER: Speak louder, please. 8 CHAIRMAN BEMENT: I will try. One thing I 9 really need everyone to hear is what we need to do if we 10 should have an emergency, which we hope will not occur. 11 There are three stairwells outside this door 12 which will provide egress, and there will be people to 13 assist you, if necessary. They're fairly easy to find near 14 the elevators. Also, the restrooms are just outside that 15 door to the left. I think you'll be able to find them 16 readily. 17 As was mentioned, Dr. Schutzer is joining 18 us by teleconference, and Mr. Craft, and Dr. Rivest are 19 trying to make it here by plane. I guess one flight was 20 cancelled, but they'll join us during the course of our 21 meeting. 22 This Committee's initial set of 8 1 recommendations for these guidelines are due to the 2 executive director of the Election Assistance Commission in 3 April of 2005, in accordance with HAVA's nine-month 4 deadline. 5 In the interim, the 2002, Voting System 6 Standards adopted by the Federal Election Commission serve 7 as the first set of voluntary voting system guidelines 8 under HAVA. 9 In accordance with HAVA, the U.S. Election 10 Assistance Commission and I have appointed 14 individuals 11 to serve with me on this Committee. 12 Those members include: 13 Two members of the Standard's Board;. 14 Two members of the Board of Advisors; 15 Two members of the Architectural and 16 Transportation Barrier Compliance Board under Section 502 17 of the Rehabilitation Act of 1973, which forever more will 18 be referred to as the Access Board; 19 A representative of the American National 20 Standards Institute; 21 A Representative of the Institute of 22 Electrical and Electronics Engineers; 9 1 Two representatives of the National 2 Association of State Election Directors, selected by such 3 Association, who are not members of the Standards Board or 4 Board of Advisors, and who are not in the same political 5 party. 6 In addition, the Election Commission has 7 appointed four individuals with technical and scientific 8 expertise relating to voting systems and voting equipment. 9 A majority of the members of the Technical 10 Guidelines Development Committee shall constitute a quorum. 11 At this time, I would entertain a motion to 12 adopt that the latest revised version of Roberts Rules of 13 Order be adopted to govern the Technical Guidelines 14 Development Committee and subcommittee proceedings. 15 SPEAKER: So moved. 16 CHAIRMAN BEMENT: I have a motion. Let's 17 hear a second. 18 SPEAKER: Second. 19 CHAIRMAN BEMENT: All in favor say, aye. 20 COMMITTEE COLLECTIVELY: Aye. 21 CHAIRMAN. BEMENT: Opposed? 22 Our first meeting has an ambitious agenda 10 1 with a specific outcome that are necessary for us to 2 accomplish in order to move forward with the Committee's 3 HAVA responsibilities. 4 Specifically, as a Committee, we must agree 5 on a procedural road map for standards development, as well 6 as a preliminary work plan. 7 In addition, we will also receive briefings 8 from NIST scientists on HAVA-related issues, and work 9 currently in process. 10 The time required to accomplish the agenda 11 item means that the Committee cannot take public comment at 12 this meeting. However, there will be other opportunities 13 for the public to comment and provide testimony at future 14 meetings, as well as electronically. 15 At this point, I should make known to those 16 who may be in the back-up rooms that to provide e-mail 17 comment, the e-mail address is voting@NIST.gov. I'll 18 repeat that again, voting@NIST.gov. 19 In fact, public comment that this Committee 20 receives electronically and in open meetings will be 21 critical to the standards development process. 22 I believe that this Committee must strive 11 1 for five distinct deliverables to the EAC in the next nine 2 months. 3 These include the following: 4 First, a list of publicly noted requirements 5 for the voluntary voting assistance standards: 6 Second, recommendation through standards 7 that currently exist and changes, if necessary. 8 Third, an assessment of best practices that 9 can be made available to the election community for us in 10 the 2006 election cycle. 11 Four, a recognition and statement, thereof, 12 of those areas where there are no current standards under 13 development. 14 And, five a prioritized calendar for future 15 standards development relative to each of the four previous 16 deliverables. 17 I can now announce that Mr. Craft has been 18 added to the roll call. Congratulations. 19 MR. CRAFT: Thank you. 20 CHAIRMAN BEMENT: At this time, I would like 21 to entertain a motion to adopt the agenda for the July 9th 22 meeting of the TGDC, which was distributed this morning. 12 1 MR. ELEKES: So moved. 2 CHAIRMAN BEMENT: Is there a second? 3 MR. HARDING: Second. 4 CHAIRMAN BEMENT: All in favor say, aye. 5 COMMITTEE COLLECTIVELY: Aye. 6 CHAIRMAN BEMENT: All opposed. 7 Before I ask Chairman Soaries and the 8 Election Assistance Commission to comment, I would like to 9 find out that the success of our work will be that of a 10 critical finding from Congress and the Administration in 11 the next fiscal year and beyond. 12 I think at this time, we can be optimistic 13 that adequate funding money will be available to us in 14 fiscal year '05, but we won't know until we get it. 15 Now, I would like to call on the Election 16 Assistance Commissioner Soaries, as well as other 17 members and other Commissioners to address the Committee on 18 any remarks they may have. 19 CHAIRMAN SOARIES: Thank you, Dr. Bement, 20 and let me thank all of your for your presence here, which 21 signals your commitment to participate in this important 22 Committee. 13 1 Let me thank my colleagues on the Election 2 Assistance Commission for the hard work they've done since 3 January to advance the cause of implementing the Help 4 America Vote Act, and the overall mission of election 5 reform in this country. 6 Let me thank members of the public, who are 7 here. We would have been so disappointed, we set up chairs 8 for you, and we lowered the temperature of the room for 9 you. So the fact that you're here is quite encouraging, 10 but moreover, it's encouraging to know that there are 11 people who are watching our process as we attempt to 12 improve elections for all Americas. 13 Before I give my charge to the Committee, 14 I'd like to introduce my colleagues on the Commission. 15 The Vice-Chair of our Commission is a very 16 dynamic leader in her own right, who has brought grace and 17 charm and elegance to what otherwise would be a very boring 18 and drab set of male Commissioners. 19 She has taken on many responsibilities, not 20 the least of which being the organizing of, and being 21 liaison to the Standards Board and the Board of Advisers. 22 The work of this Committee will ultimately 14 1 be vetted through those two Boards. They are defined by 2 the statute, and she is our designated Federal officer to 3 those Boards. And she has brought great energy and insight 4 to our work, and I'd like to introduce for her brief 5 comments, Vice-Chairman Gracia Hillman. 6 VICE-CHAIR HILLMAN: Thank you. I just want 7 to welcome everybody and say thank you. And to pick up on 8 the Chairman's comments, to say that I will be facilitating 9 the recommendations that come from this Committee to be 10 reviewed and commented on by the Standards Board and the 11 Board of Advisors. And as you heard previously, we do have 12 members from each of those bodies on this Committee to also 13 help with the liaison work. 14 It is really great to see this. For six 15 months we have been trying to figure out how we were going 16 to put together the resources, and within the time frame, 17 to see the TDGC in operation. So, it's really nice to see 18 that we were able to bring everybody together for the 19 meeting. I thank you all for being here. 20 CHAIRMAN SOARIES: You can tell that 21 Vice-Chair Hillman has been the one among us who has really 22 specialized in the bureaucracy. You can assess how 15 1 bureaucratized a person has become with the ease with which 2 they use initials to describe organizations. 3 We have called the TGDC everything. But she 4 said it, it flowed so easily that you can tell she has 5 really specialized in the bureaucracy. 6 The next Commissioner, who I'd like to 7 introduce, is the young attorney from Texas, who has work 8 in the Whitehouse, has a wealth of experience with local 9 election officials, having represented many counties and 10 their efforts to address the elections and other issues. 11 He brings youth and vision and a dynamic to 12 the Commission that makes us a better Commission. He has 13 taken on responsibility for seeing to it that the states 14 receive their requirements' payments as perhaps the largest 15 portion of his portfolio. 16 All of you know that our embryotic stages 17 require that the Commissioners not only set policy, but 18 also do administrative work. And this Commissioner is 19 chiefly responsible for our being able to report that the 20 requirements' payments began to flow to the states within 21 the statutory time that was prescribed by HAVA; and that 22 within six months of our having been established as a 16 1 Commission, that money began being distributed. 2 He is a proud father, and as I mentioned, a 3 very dynamic addition to this Commission. And I would like 4 to call on him for his remarks now. Commissioner Ray 5 Martinez. 6 COMMISSIONER MARTINEZ: Thank you, 7 Mr. Chairman. I appreciate that very kind introduction. 8 I am delighted to be here this morning. I join 9 my colleagues, the Chair and Vice-Chair Hillman in 10 Welcoming the formation of The Technical Guidelines 11 Development Committee. 12 I think this is going in and out now. I 13 think I'm trying to project as loudly as I can. Hopefully, 14 everybody can hear me. 15 I do want to pick up on the comments that 16 our Chairman just made; and that is the fact that we have 17 over the course of the last two months or so, this 18 Commission distributed close to $1 billion now of payments 19 to state and local governments -- or to state governments, 20 I should say, under Title 2 of the Help America Vote Act, 21 The Requirements Payments. This is in addition to Title 1, 22 Early Out Monies, that were distributed prior to the 17 1 formation of this Committee -- of the EAC, I should say. 2 The reason to pick up on those is the simple 3 fact that we have a lot of money that is Federal money 4 unprecedented, historical role that the Federal Government 5 is now playing in the administration of Federal elections. 6 And we have money going out to states to improve the 7 Federal Election Administration process, and the states are 8 waiting for the Election Assistance Commission, and its 9 very important statutory advisory committees like the TGDC, 10 to offer guidance and advice on what to do with regard to 11 the purchase of voting systems and related questions. 12 There's no doubt in my mind that the 13 formation of this committee is a major milestone for the 14 Election Assistance Commission. There is much work to be 15 done, and certainly I'm most supportive of Dr. Bement and 16 our partnership with NIST, and the very fine folks that 17 work there as we do the important work. 18 It seems like every time we come to a 19 milestone, I say this is perhaps the most important thing 20 the EAC will be doing. I'll be saying that a lot because 21 we're doing, I think, so many important things, but we're 22 doing it in a partnership with organizations like NIST, 18 1 with our state and local election partners, with advocates 2 for disability rights, advocates for voting rights and 3 civil rights, with the entire community of people who are 4 interested in improving the election administration 5 process. 6 So I welcome you all today, and I look 7 forward to the work of this Committee. 8 CHAIRMAN SOARIES: Thank you, Commissioner. 9 I saved the final Commissioner for last, not 10 because he's the best cook, he's certainly not the best, 11 but the shortest, I saved him for last because our presence 12 here today is directly attributable to work he has done 13 more than any of the four of us. 14 As I mentioned, we divided up our 15 administrative focus into portfolios, and the next 16 Commissioner took personal responsibility for the 17 development of the Technical Guidelines Development 18 Committee, which Commissioner Hillman easily says "TDGC". 19 The Election Assistance Commission divided 20 itself into five focus areas, the administration, 21 communications, Standards and Advisory Boards, grants and 22 research, and election resources. 19 1 And consistent with the mandate of HAVA, we 2 perceive that the ultimate resource for voting in this 3 country is the setting of comprehensive standards that 4 would be useful for testing, and useful for offering 5 guidelines to all of the states. 6 The Commissioner, who has the most 7 experience in actually running an election, is Commissioner 8 Paul Degregorio. He served as the Director of Elections 9 for St. Louis County for many years. He worked with the 10 International Federation for Elections, and monitored 11 elections all over the world. 12 So he brought to this Commission so much 13 practical hands-on experience, that consistent with that, 14 we ask Commissioner Degregorio if he would not take 15 personal responsibility to oversee the election resources 16 commitment of our agency. 17 He has worked tirelessly to ensure that this 18 Committee was named, that we work very closely with NIST to 19 accomplish a meeting as soon as possible, and he is our 20 designated Federal officer between the Commission as 21 liaison to this Committee to ensure that there are no gaps, 22 and to ensure that we have a healthy, productive working 20 1 relationship. 2 All three of us respect Commissioner 3 Degregorio's expertise. We bring passion; he brings 4 experience. And I'm proud to introduce him, and I thank 5 him for the work that resulted in our being here today. 6 Commissioner Degregorio. 7 COMMISSIONER DEGREGORIO: Thank you, 8 Mr. Chairman. I'm not a preacher, but I do pray, and I 9 said a lot of prayers to get us here today. 10 Let me just say, the past six months with 11 EAC has been some of the most challenging but enjoyable six 12 months of my life, because of my three colleagues here. 13 The Chairman articulated very well what 14 we've all been doing, but you know it's been team work. 15 Yes, I bring my expertise to the table, but my colleagues 16 have brought their's to the table, too, and we have worked 17 as a strong team, a bipartisan team, over the past six 18 months to do everything we can to implement the Help 19 America Vote Act. 20 This is a historical day for this Committee 21 to be meeting, and to begin its work, and to begin its work 22 over nine months. 21 1 I just had my first granddaughter about ten 2 days ago, and I watched my daughter as she went through her 3 nine months of pregnancy. And my wife had four daughters, 4 so I went through that experience. And now this will be a 5 nine-month experience, too. So I know, you know, during 6 pregnancies there's always ups and downs and many things. 7 I know that this is a great team that's been 8 assembled here. The 15 people here represent some of the 9 best in the United States of America, some of the best 10 professionals, best election officials. It's an honor for 11 me to be associated with you, and I hope to be of 12 assistance to you, to be the liaison with this group, with 13 the Commission. 14 It is my hope and prayer that all of you 15 work as a team as we have the past six months, and that we 16 could be your guide in how you proceed to do this work, 17 because I think this country needs unification, democratic, 18 republican, and it also needs leadership in this particular 19 area. 20 We recognize that there is not much you can 21 do to change things between now and November in terms of 22 election equipment or setting standards, but you will 22 1 certainly set the stage for the future. 2 Commissioner Martinez worked very hard to 3 get the money to the states, and the states are clamoring 4 for guidance from you all on what equipment they should 5 buy, or what standards they should use when they buy this 6 equipment. 7 Vice-Chair Hillman worked very hard over the 8 past few months to get money for this Committee for next 9 year. And we're hopeful that her efforts will be 10 successful. And we're confident, actually, that they're 11 going to be successful, so this Committee can have the 12 funds, so NIST can have the funds do its work and support 13 this Committee. 14 The Chairman has provided great leadership 15 over the past six months, but particularly on this issue. 16 He's made public statements; he's encouraged us as 17 Commissioners to focus on this issue, and we have. And 18 he's provided great leadership. 19 I look forward to working with all of you. 20 I know Dr. Bement will be a great leader of your team; and 21 anything that I can do for you, please feel free to contact 22 me. I'll be sitting through your meetings when you have 23 1 them, but here available with EAC, if you should need me 2 for any reason. Thank you. 3 CHAIRMAN SOARIES: Thank you, Paul. Let me 4 now begin with clearly recognizing the fact I'm a Baptist 5 preacher, and you gave us until 10 after 10 to do our 6 business. We will not be with you all day, and we did want 7 to communicate with you some very specific points so that 8 you would really understand the significance that we place 9 on your work, and the quality and relationship that we 10 pursue. 11 I want to begin, though, by thanking the 12 National Institute of Standards and Technology. Dr. Bement 13 and his wonderful team have been marvelous, not only in the 14 building of this Committee, but in the work of the EAC in 15 general. 16 Even before the EAC was confirmed, you 17 should know that the National Institute of Standards and 18 Technology, under Dr. Bement's leadership, began working on 19 products and on research that would undergird the standard 20 setting process through public forums, through human 21 factors' studies. 22 NIST has really preceded the EAC in the 24 1 implementation of the Help America Vote Act. And I'd like 2 to thank Dr. Bement publicly and personally for his 3 commitment to the EAC, to the passion with which he and his 4 team has pursued election reform. 5 Allan Eustis has been an invaluable resource 6 to us. Craig Burkhardt has done everything except move 7 tables in this office. Susan Zevin, before she left. It's 8 just been a marvelous experience. And then, of course, 9 Dr. Bement's capable deputy has stepped in 10 because Dr. Bement now has added responsibilities due to 11 recent changes in his professional life. 12 There's been just a seamless flow of 13 communications and coordination between the EAC and NIST, 14 and we anticipate that that will do nothing but grow. 15 Now, let me focus on the Technical 16 Guidelines Development Committee. I want to thank all of 17 you personally for your willingness to serve. 18 I don't know all of you as well as I know 19 some of you, but my assumption about each of you is that 20 none of you needed more work to do. I don't think you were 21 praying and asking God to give you another assignment. I 22 want to make it clear that your presence here today is only 25 1 indicative of the fact that large groups of people have 2 high respect for you and your work, for many of you have 3 been designated by various institutions mandated by HAVA. 4 But it also reflects the great character of this country 5 because you are volunteers. 6 We were able to buy your plane ticket, we 7 got you a bed at a hotel, and we give you a couple dollars 8 to eat lunch at McDonalds. I think it's Subway today. But 9 this is a volunteer effort. 10 If you look at America from a step away, 11 you'll discover that America is, in fact, a great country 12 because people do things just like without remuneration, 13 without compensation, you'll get no stock options, you'll 14 make no overtime. You're giving of your lives, you've 15 sacrificed time away from your families, you've given up 16 time away from your jobs to help America vote. 17 The fact is when the story of this country 18 is told, it will have to include the fact that we were not 19 just a great economic system, a great political system, but 20 we were a network of people who were willing to do just 21 what you do, serve on boards, serve on committees, and give 22 up their own individual resources and intellectual capacity 26 1 to make this happen. I just want to thank each of you, 2 because none of you had to serve. 3 This has become perhaps a more high 4 visibility and high profile mission than it would have been 5 five years ago. Had we been meeting here in 1999, we 6 perhaps, would have had much less pressure on us. We would 7 have had less media attention. Our outcomes would have 8 been less visible. 9 Now, not only do you give of your time, but 10 you put your name on a list next to a list of names of 11 people who you know will be criticized because there will 12 be large groups of people who will not like the outcome of 13 your work and we know that now. 14 I want to thank you on behalf of the four 15 Commissioners who could not make the critical decisions 16 that we have to make, if you did not express a willingness 17 to serve. Thank you in advance for the hard work and the 18 hours that you'll invest in this process. 19 Our challenge in many ways is your challenge 20 because we are now in this together. What we've discovered 21 since coming to this task is that voting in this country 22 was something that we really took for granted for a long 27 1 time. That we began voting as a country in colonies, 2 selecting delegates to represent the interests of the 3 colonies as we discussed the future concept that we now 4 consider to be America. And voting through the years has 5 evolved managed by the states with the growth of the 6 country. 7 It's not as if we waited until we had 250 8 million people and then decided to vote. We voted our way 9 from the 18th Century to the 21st Century. And that voting 10 has, in large measure, been something that we assumed to be 11 normal, but we never looked at it quite the way you've been 12 asked to look at voting before. 13 Because voting has been so normal and so 14 regular, and it has been an election for something by 15 somebody somewhere every year since we got started as a 16 country, many assumptions have grown with voting. 17 If you sit where we sit, and now you do sit 18 where we sit, we've discovered that many of those 19 assumptions were more mythology than reality. 20 One of the myths that has persisted in the 21 country is that the Federal Government is much more 22 involved in voting than the Federal Government has been. 28 1 Another myth that has evolved is that the machines upon 2 which we vote have gone through the kind of vetting that 3 our microwave ovens, and our cars, and other utensils have 4 gone through. 5 We are here today to turn that myth into 6 reality, and to create standards that make us as confident 7 in the equipment that we use to vote, and the voting 8 systems that have embraced our democracy as we do all the 9 other devices that we use in normal life. 10 The Help America Vote Act really changed all 11 of that, and created a historic role for the Federal 12 Government in the administration of Federal elections. 13 Our job as a Commission is to be a national 14 clearinghouse, providing information to the states. And 15 that information has to be guided by the advice that we get 16 in large measure from persons like yourself. 17 We are faced with a challenge exacerbated by 18 much more media attention given to voting devices than ever 19 before in human history. 20 One of you, with whom I visited recently, 21 told a group at my hearing that the media has done such a 22 fine job in one jurisdiction, that there's almost no need 29 1 to do any public education on the types of voting devices 2 used because the media has done all of that. The media has 3 described with great, great detail what a punch card looks 4 like, what a legal machine looks like. And in that sense, 5 the media has provided a public service. 6 The media has also come to some conclusions 7 and made its decisions without the benefits of real facts, 8 and without access to real science, that I think has made 9 our jobs even harder. 10 Our challenge over the next nine and a half 11 months is to pursue the process that has integrity, so that 12 the products that we produce from this process have equal 13 integrity. 14 I'd like to charge this Committee to be 15 committed to principles that I know all of you share; but 16 sometimes it's harder in a group to maintain the commitment 17 to certain values that you possess as individuals. I'm 18 going to charge you as Chair of the EAC, to work hard to 19 rise above any partisan, geographical, or philosophical 20 differences by keeping the focus on the mission. 21 I want to charge you with keeping open minds 22 as we have had to, because there are numerous people who 30 1 have already made up their minds as to what you should do. 2 They will call you, they will e-mail you, they will lobby 3 you, they will threaten you, and they will try to convince 4 you that only they, only they, know exactly what the 5 outcome of your work should be. 6 I want to charge you with having open 7 exchange. We've had as a Commission, wonderful experiences 8 disagreeing without becoming disagreeable. You will never 9 read about or hear about any disagreements we've in public, 10 because all of our disagreements are in private. 11 I would urge you to attempt as best you can, 12 to keep the process on the one hand open and transparent, 13 but on the other hand, to keep it unified sufficient to 14 jelling as a team, and fostering a sense of team effort. 15 I also want to charge you to do your work in 16 nine months. One, because the law requires it; and two, 17 because we do have a sense of urgency as it relates to this 18 mission. It would be better to produce a product in nine 19 months, and have another product deferred, than to get too 20 caught in the quagmire of perfection, and not come up with 21 anything at the end of nine months. 22 Dr. Bement will talk in much more detail 31 1 about these topics, but we are encouraged, again, because 2 so much work has been done by some very important people, 3 even prior to today. 4 The NIST Human Factors Report is a report 5 that the four Commissioners embrace enthusiastically. It 6 was a report that identified major challenges as it relates 7 to usability. It was a report that had objectives that we 8 think should guide this Committee, at least in the area of 9 usability. And it's a report that we would urge you to 10 take very seriously and consider as you decide on your 11 priorities and your timetables. 12 The IEEE Project 1583 is very important 13 work. I'd like to thank that organization for the work 14 they did as it relates to considering standards and options 15 for standards, and for establishing really, a precedent for 16 participatory inclusive standard setting process. I think 17 this Committee would do well to embrace the products in the 18 form that they're in, and to consider what the implication 19 and requirements. 20 We would be remiss if we did not thank and 21 recognize the National Association of State Election 22 Directives for the fine work that they did. The concept of 32 1 voting system standards did not really emanate from any 2 vision in the halls of government, at least the Federal 3 Government. Rather the notion of voting system standards 4 really emanated from the passion and the vision of state 5 election directives. 6 We have among us Alice Miller, who is a 7 former national leader of NASED. We have two 8 representatives of NASED at the table, Mr. Craft and Dr. 9 Williams, and I think we all owe each of you a debt of 10 thanks. 11 Many of you around the table who are not in 12 NASED contributed to the NASED process. The National 13 Association of Secretaries of State and others contributed 14 from 1975 going forward to a process where volunteers 15 without compensation, without even reimbursement, traveled 16 around the country, went to meetings, and created the only 17 voting system standards to which Dr. Bement referred a 18 little earlier, that we have today. 19 Were it not for NASED, were it not for that 20 group of people, we would not even have the FEC 2002 21 standards, which started in 1990, but the work started long 22 before 1990. And NIST, in its former life, contributed to 33 1 that process also. 2 We're indebted to the work of NASED, and 3 therefore, I think we owe it to history, we owe it NASED, 4 and we owe it frankly to common sense, to really consider 5 how much of the 2002 Federal Elections Commission standards 6 can be embraced, and use that as a starting point, and not 7 throw out the baby with the proverbial bath water, thinking 8 we have to start from scratch. 9 I also want to generally say that we are 10 ultimately looking for standards that are 11 performance-based, standards that can measure success as it 12 relates to voting, standards against which new voting 13 devices can be measured; and standards that embrace 14 existing and emerging technologies, recognizing that it 15 will never be our responsibility to dictate to states what 16 kind of voting devices they use. Rather it is our 17 responsibility to establish standards against which states 18 can make informal decisions. 19 If we were in church, I'd say that's the 20 Devil. 21 MR. EUSTIS: Could the people that are 22 around the conference call please push their mute buttons, 34 1 make sure your mute is on. Thank you. 2 CHAIRMAN SOARIES: Having said that, let me 3 share with you our commitment as a Commission. And I can 4 speak for all four of us, even though we've never had a 5 public meeting to vote on what I'm about to say. That's my 6 disclaimer. 7 We are committed in the first instance to 8 making sure you have money. Paul mentioned Commissioner 9 Hillman's work. She's taken responsibility for helping us 10 craft and market our budgets. You may not know that we 11 have received support from the Office of Management and 12 Budget, and the Whitehouse to amend the President's '05 13 budget, to include $10 million for research for this 14 undertaking that had not been this individual submission. 15 We're going to continue educating members of 16 Congress. We are forbidden from lobbying. We have no 17 bypass authority, which means we can't go directly to 18 Congress with funding requests, but we can educate Congress 19 and brief Congress, and make sure the member understand the 20 importance of research. Research without money is like a 21 car without gas. And we know that not all things can be 22 solved with money, but this is one issue that does require 35 1 money. 2 The second commitment we make is the 3 commitment to communicate on your behalf, and in support of 4 your work, to the public ensuring that the public knows as 5 much as the public can know about this process. It is not 6 going to help us if you were perceived to be a private 7 group going off in a clandestine manner to pursue some 8 secret agenda. 9 We have responsibility through our public 10 appearances, through our access to the media, to 11 communicate with passion and clarity, the work of the TGDC 12 to ensure that the country understands that while nine 13 months may seem like a long time, and may not have much 14 impact this year, that nine months is an aggressive amount 15 of time, and that you're not dragging your feet, or taking 16 this task lightly. We will do that to the best of our 17 ability, and the communications link that exists between 18 the EAC and NIST will, I think, serve you well. 19 Finally, we're committed to a partnership. 20 We're not going to distance ourselves from you, thus 21 leaving you out on a political limb to die. Nor are we 22 going to attempt to micromanage you. We have a wonderful 36 1 working partnership with NIST. We have complete confidence 2 in the leadership of Dr. Bement, and we believe that this 3 Committee is enhanced by the unusual degree of competence 4 and credibility that NIST brings to the enterprise. 5 I have nongovernment agencies all of my 6 adult life, and without fear of contradiction, I can say 7 this: The National Institute of Standards and Technology 8 and Dr. Bement probably have much credibility as any 9 government agency anywhere has in anything. And NIST has 10 done a fine job, has distinguished itself as being capable, 11 professional, competent, and diligent towards this task. 12 We are committed to a partnership with you. 13 The partnership means we may sometimes disagree, we may 14 make requests of you that make you uncomfortable, likewise, 15 it means that we will fly together or die together, and 16 that's our commitment. 17 I do have one favor that I need to ask of 18 you, and in your binders, you'll find a report that's done 19 by the Brannon Center for Justice and the Leadership 20 Conference on Civil Rights. 21 As you know, most of your work, if not all 22 of your work, will focus on the nine-month mission to come 37 1 back with recommendations, that we will then take to the 2 Standards Board and Board of Advisors and vet with the 3 public. 4 However, you can appreciate that much of our 5 work has to include focus on what we can do to make a 6 positive impact on the November 2004 elections. We have 7 not been alone in our concerns about the quality of the 8 elections' process and the effectiveness of the voting 9 outcomes. 10 This report was prepared at the request of 11 the Leadership Conference on Civil Rights, which is a group 12 that we met with early on. 13 Dr. Bement and I talked about this. Our 14 response to this report has been to say that we would like 15 to ask the members of this Committee to take a look at it, 16 and based on your experience, your expertise, give us some 17 sense of whether or not you think there is any way you 18 would like us to include this in the kind of guidance or 19 options that we provide local elected officials. I say 20 that because next week, we will begin the process of 21 releasing a very comprehensive tool kit that has models and 22 examples of best practices and administrative procedures 38 1 for this November. 2 As we both solicit and receive information 3 and input from organizations around the country, we need 4 the fact that you are on our team now to contribute to our 5 process as we deliberate what our options are. So, if you 6 would take time over the next few days and read this 7 report. 8 I'm not asking you to formally include this, 9 Dr. Bement, in your deliberations, but because you are who 10 you are, and you know what you know, and you do what you 11 do, and now you're connected to us joined at the hip, we 12 value your input, and before we respond to the contents of 13 this report, that would be great. 14 In closing, we had this discussion at the 15 Standards Board, and I'm sure the Vice-Chair wants me to 16 say this, so I'll say it. At the end of this process, what 17 many people question is this: Why should you do all this 18 work to create standards, which at the end of the day are 19 voluntary to the states? Why should you fly from across 20 the country, give up your time and talent, to produce a 21 document that is not mandatory? 22 HAVA uses some very interesting language. 39 1 It uses standards language; it uses guidelines language; 2 and it uses recommendation's language. We are committed as 3 a Commission because we signed on knowing that we did not 4 have regulatory authority, and that we could not mandate 5 standards. 6 We all recognize today that every voting 7 jurisdiction in this country has been waiting for guidance 8 and waiting for resources to help that jurisdiction conduct 9 free, fair, and effective elections. So we are confident 10 that the voluntary guidelines process can work. 11 The fact is that down the road as we 12 discover that we're wrong, then we'll make a recommendation 13 to Congress. 14 In the meantime, we see our setting 15 standards and offering them as guidelines as being real 16 analogist to raising teenagers. If you're going to raise a 17 teenager, you know that parents have standards, and then 18 the kids take them as guidelines. 19 So having said that, some of us have been 20 somewhat successful with that, we've negotiated then. And 21 we believe that states and jurisdictions are waiting to 22 work with us and to receive our guidance. 40 1 So having said that and being 2 uncharacteristically 20 minutes early, I'd like to conclude 3 by thanking you again, and I hope and pray that this 4 process will create the America that all of us have assumed 5 exists, and that can upgrade our democratic practice so 6 that we can be the beacon of light, and the standard for 7 voting for the entire world. Thank you. 8 CHAIRMAN BEMENT: Thank you, Chairman 9 Soaries, and also other members of the Commission. You've 10 not only charged us, but you've supercharged us. I hope we 11 can live up to your expectations. I want to thank you for 12 working with me and putting together this wonderful 13 Committee. And I think with the team work that's already 14 underway, we're going to make you proud of us. 15 I also especially want to thank Commissioner 16 Degregorio for all the work that he's done with NIST and 17 with members of our team in getting us to where we are 18 today. It's really been a miracle in many respects that we 19 now have this Committee that is empowered to vote and do 20 work. And, in fact, a lot of it went together last 21 evening. So thank you very much, Paul. 22 I think one or two people have come on the 41 1 telephone since Dr. Soaries begin his remarks. I wonder if 2 they'd take a moment and you could identify yourself, those 3 who have joined us on teleconference. 4 MR. EUSTIS: Turn your mute back off. 5 MR. SCHUTZER: I am Dan Schutzer. 6 CHAIRMAN BEMENT: Yes, Dan, thank you. 7 Anyone else? 8 MR. EUSTIS: Turn the mute back on. 9 CHAIRMAN BEMENT: I guess to segue, I have 10 raised eight teenagers, and I have to say that any 11 guidelines that they might accept are totally 12 discretionary. 13 Now at this time, I'd like to ask each of 14 the Committee members in attendance to introduce 15 themselves, speak briefly about their backgrounds, and also 16 comment briefly on why they agreed to serve on the 17 Committee. 18 I would also ask that during the course of 19 the day as we get into a discussion, to assist our 20 stenographer, our recorder, and to be sure we get a clear 21 transcript of our deliberations, if you would identify 22 yourself as you make comments, that would be very great. 42 1 Paul, let's start with you. 2 MR. CRAFT: I'm Paul Craft. I'm Chief of 3 the Bureau of the Voting Systems Certification in the State 4 of Florida, and for the last 12 years, had the honor and 5 privilege of being part of a very small group of 6 individuals within the National Association of State 7 Elections, who have been working as volunteers, and with 8 great passion to attach what we're carrying forward today. 9 It is frankly such a thrill and honor to be 10 here, that I'm rather humbled by the Chairman's gratitude 11 at our service. There's really nowhere else I would rather 12 be. I think this is probably the most important work in 13 the nation at this time. 14 As many of you who know me know, I actually 15 would like to push beyond the legal scope of this 16 Committee. Florida very strongly believes, and has taken 17 the position in the last 12 years, that existing standards 18 should not be voluntary, they should be mandatory. 19 Obviously, that's outside our scope, but we hope to work 20 here and lay out good enforcement standards which Congress 21 can bring at a later date, if they wish. 22 CHAIRMAN BEMENT: Let me interrupt for just 43 1 a moment. Chairman Soaries, I did want to respond to your 2 request to provide comments on the recommendations from the 3 Brannon Center. Clearly, this is valuable input for our 4 deliberations. It would be -- the recommendations here 5 would be highly valuable in our considering some of the 6 security aspect of voting. 7 I would ask individuals certainly to respond 8 to the request. We need to demonstrate that we can operate 9 realtime, as well as by procedure. On the other hand, I 10 would caution you that you would be responding as 11 individuals, not as members of the TGDC, so this is your 12 individual input. Thank you. 13 MS. QUESENBERY: My name is Whitney 14 Quesenbery, and I guess I'm pretty honored to be in such 15 experienced company. I guess I'm one of the voter 16 volunteers. I got interested in this in 2000, like so many 17 people did, a lot of people. 18 I am the President of the Usability 19 Professional Association, a trade association of people who 20 work with industry and government making the products and 21 web sites, and the regulations that we often use, easier 22 for people to use and more effective for people to use. 44 1 In 2000, a lot of people got very excited 2 about the issue of usability in voting, and I was one of 3 the few people left standing as a volunteer. So we started 4 an education process within our organization that contained 5 some of the other human factors, just to keep members aware 6 of what was going on. One thing led to another, and now 7 we're getting with the IEEE Committee on the 8 Usability/Accessibility section of that standard. 9 I'm really glad to see that we're beginning 10 to consider people, and not just machines as we think about 11 voting, because it is ultimately a human process, and not a 12 systems' process. 13 I think of it as more than voting -- there 14 are more than voters in that system. There are lots of 15 people who were working around election officials who are 16 candidates themselves. And all of those people need to be 17 considered as we think about standards. So, I think that's 18 all I have to say. 19 CHAIRMAN BEMENT: Thank you, Whitney. 20 MR. WILLIAMS: I'm Britt Williams from the 21 State of Georgia, and I represent NASED on this Committee. 22 You've got my bio in the materials that were sent to you, 45 1 so I won't repeat that. 2 I will say that this is the fourth body like 3 this that I've been involved with to write standards for 4 voting systems. I was a consultant for the FDC in the 5 development of the standards in 1990, again in 2002. I'm 6 on a board with the IEEE to develop standards. Those other 7 three were non-funded. It was entirely voluntary. In 8 1990, it took us four years to develop a set of standards. 9 In 2002, it took three years. 10 The IEEE project has been in business for three 11 years, and it's not in any danger of closing any time soon. 12 So, I'm going to be real curious to see what we can do with 13 funding in nine months. 14 MS. DAVIDSON: My name is Donnetta Davidson, 15 and I guess you can say elections have been my life, and I 16 won't tell you how many years. I started out in the 17 county. I probably am one of the very few people that was 18 sent here that has been a county clerk, the Secretary of 19 State, and a candidate. And all of those things are very 20 important with, you know, what we're doing. 21 I was part of the birth of NASED, part of the 22 birth of the original standards. And, obviously, my goal 46 1 is to hopefully set standards that will improve the process 2 as we always like to do, make it a better process of voting 3 in the United States. 4 I think that nearly every state -- I know I 5 fought in Colorado years ago to put the standards in where 6 it was a mandate. They have to be approved by the 7 independent testing authorities before we can even test 8 them in our state. So I think many of the states have done 9 this, and I think that's a -- if we can't get it where it's 10 a mandate nationally, that leaves the states to move 11 forward to do their own mandate. And I think that's a 12 great thing that we ought to encourage, Paul. I agree with 13 you. So I look forward to it, and I feel it's a real honor 14 to be serving with all of you here. 15 MR. BERGER: I'm Steve Berger. I'm here 16 representing the IEEE. It's a great honor to be here, and 17 it's an honor for the IEEE to be included. 18 Professionally, I don't have a long history 19 in elections. I certainly haven't served as an election 20 official. Most of my career has been in product 21 development, basically serving the function of advising 22 management when a new design was ready to be introduced and 47 1 marketed. 2 In many ways that's what we're about, helping 3 voting officials know when equipment was ready for 4 implementation. I say sometimes that I spent a lot of 5 years breaking things and making it look like it was 6 someone else's fault. 7 Along the way, I served on the two Access 8 Boards of advisory committees, and disability access is a 9 particular interest of mine, and I'm gratified to see 10 there's going to be a focus here. 11 I'd like to say in terms of the IEEE, and as 12 much as Whitney said, I view my role as a conduit. There's 13 a lot of people in this country, particularly in the 14 engineering community, that want to contribute. I would 15 just offer the invitation that I think it would be entirely 16 appropriate for this Committee to ask different bodies of 17 experts to lend a hand, to take specific assignments and 18 maybe lend their expertise. 19 In my experience, a lot of people want to be 20 involved, and would be glad to serve in that way. We 21 certainly view our effort as being part of this process and 22 under what this Committee does and the EAC more generally. 48 1 MS. PURCELL: Thank you. Good morning, I'm 2 Helen Purcell, and I'm also very glad to be with this 3 Committee. 4 My county is Maricopa County, Phoenix, 5 Arizona, and we have a million three registered voters, and 6 that requires a tremendous amount of administration putting 7 together for the election. I have been an election 8 official there for 15 years. 9 I have seen some very good things happen. But 10 then as a result of the controversies of the 2000 election, 11 I think that we have all seen that we either could do a 12 better job of the job that we do, we can also do a better 13 job of educating the public as to what we do, and get a 14 response back from them. 15 It is a system that should be very user 16 friendly. I don't know that we advertise that we are a 17 user friendly group of people, but we certainly need to do 18 that. And I hope that with the work that we can do here, 19 we can make some headway in a lot of areas. 20 MR. HARDING: Good morning. I'm J.R. 21 Harding from the great state of Florida. Perhaps that's 22 one of the reasons why we're here. 49 1 I am a member of the Access Board. My life 2 is not a voting expert, while I did experience the life of 3 (inaudible) the need for name recognition. 4 My perspective, this is perhaps that of the 5 disabled perspective. Most of my adult life and part of my 6 adolescent, I started with trying to run away from whom I 7 had become. But as I grew and matured, worked, labored, 8 and learned, I realized this is part of who I am, and that 9 through my adult life, I've really become a bit of a 10 full-time advocate representing from the state of Florida, 11 nationally and statewide, from transportation to work, 12 labor, to educational issues. 13 I just kind of feel my role in life is 14 pushing the envelope for independence and self-sufficiency 15 in our world. And I think voting is a cornerstone of that 16 self-sufficiency in terms of expressing and shaping our 17 view of our America and our cities. 18 It's from that advantage point that I'm glad 19 to be here and offer what little bit I can to ensure that 20 everyone has the opportunity to vote and reliably. Thank 21 you. 22 MR. ELEKES: I'm Jim Elekes. I should 50 1 state in full disclaimer I am from New Jersey. I have 2 three things in common with Chairman Soaries: We both 3 reside in New Jersey; we both reside in Somerset County; 4 and we are both debonair males. 5 I am an associate college professor in 6 public administration and policy. My experience in voting 7 over the years has been to work to ensure that there is 8 barrier free access to the polling places, as well as 9 appropriate assistance at polling places, volunteers 10 trained to assist persons with disabilities in exercising 11 their right to vote. 12 I have also worked as a beta tester for a 13 number of organizations who produce software speech 14 services, and braille for persons who are low vision or 15 blind. 16 As a representative for the U.S. Access 17 Board, I share Mr. Harding's goal as the lynchpin, voting 18 is a form of expression, and the ability to proactively 19 participate in one's community. And whatever contributions 20 can be made, or input as a tester that I can make to ensure 21 the standards are capable of meeting the diverse and 22 specialized needs of persons with disabilities, as well as 51 1 perhaps being a liaison to the blind and visually impaired 2 community, I look forward to the challenge of meeting the 3 goal of this Committee. 4 MS. MILLER: Good morning. I'm Alice 5 Miller. I'm the executive director for the D.C. Board of 6 Elections and Ethics. And let me first say I am the mother 7 of two teenagers, neither of which accept my rules as 8 guidelines or standards. 9 From that, I have been working elections for 10 about 17, 18 years. I started off in the legal field on 11 (inaudible) the staff of the attorney general counsel of 12 the elections board. Somehow or another, I ended up as 13 executive director, and I'm still trying to figure that 14 out. 15 From that, what I've learned is that we've 16 watched voters by all definitions. I've watched them as a 17 candidate, watched them as a voter, watched them as a poll 18 worker learning the process, and from that angle. 19 I know that there has to be a comfort level 20 with what everybody is doing to not only ensure the 21 person's vote is counted as they have voted, and the poll 22 workers are comfortable with ensuring that their 52 1 instructions are where they need to be to ensure the 2 voter's vote is also accurately done and accounted. 3 I'm actually probably one of the least 4 technical people on this Committee. I still question how 5 you all thought it was a very good idea to place me here. 6 So it's from that perspective that I am indeed honored to 7 be here, and I do intend to give my all and to contribute 8 what I can to help with the guidelines development in this 9 process. 10 MS. TURNER-BUIE: I am Sharon Turner-Buie, 11 director of elections from Kansas City, Missouri. May I 12 start a little bit differently than everyone else by 13 commending the Commission on your office space. I know for 14 a commission to focus on transparency -- you certainly get 15 that feeling as soon as you get off the elevator, you get 16 feeling of transparency, which is so important in this 17 process. 18 I am very humbled to be asked to serve on 19 this Committee. It's a very intimidating thought because 20 to be surrounded by engineers and technical experts with 21 the science background, etcetera, etcetera. People don't 22 normally associate that with elections administration, but 53 1 it certainly is an important part of it, because we need it 2 for the task that is at hand. 3 I would like to say that as an election 4 administrator, we had problems and concerns prior to 5 Elections 2000. What Elections 2000 did, what the HAVA Act 6 did was give us a voice. Now we not only have a voice, but 7 we have an audience. And with that voice, I hope to be 8 able to serve as a link between the architects of these 9 voluntary voting systems standards and guidelines and, 10 ultimately, the people that use these systems in the 11 election, certainly the people that work on election day 12 are crucial to the process and ultimately the voters. 13 So it is so important that we remain focused 14 on the ultimate users as we are surrounded by all of these 15 engineers, scientists, and technical people so we can keep 16 the human element close at heart. 17 I would like to disagree with one member of 18 the Committee, and that being Ms. Miller. She is not the 19 least technical person here. I have that distinct honor. 20 But hopefully we will be able to contribute some necessary 21 input as we move forward. 22 MS. CALDAS: My name is Anne Caldas, and I 54 1 represent the American National Standards Institute. 2 Actually, that was a nice segue into what we do, because 3 there are many experts in this country who populate 4 standards writing organizations, including 200 that ANSI 5 accredited, including IEEE. 6 And I agree with Mr. Berger that many will 7 be willing to assist, and I don't think we need to think 8 about starting from scratch. 9 American National Standards are developed in 10 an environment that is open-balanced, fair, and 11 characterized by due process. ANSI welcomes the 12 opportunity to participate in this project, and to help the 13 Election Assistance Commission and the TGDC in the areas 14 both of standardization and conformity assessment. 15 I'm honored to be here. ANSI is willing to 16 give whatever resources we can to help implement the Help 17 America Vote Act. Thank you very much. 18 CHAIRMAN BEMENT: Well, I'll bring up the 19 rear. I'm Arden Bement. I'm the Director of NIST, and 20 acting Director of the National Science Foundation. And 21 people ask me all the time how can you carry out both those 22 responsibilities, and it's because there's a hidden secret, 55 1 and that is that NIST excels in spite of it's director. It 2 not only has great breadth but also great depth. 3 I think one of the very best decisions I've 4 ever made was to appoint Dr. Semerjian as deputy director 5 of NIST. He has stepped in immediately without a loss of 6 movement, and has done a superb job in continuing the 7 day-to-day operations of NIST. 8 As far as my enthusiasm in chairing this 9 Committee, it's both a learning experience as well as a 10 human experience. A learning experience because this is 11 very important as well as a challenging area to be working 12 in at the moment. 13 It's a human experience because I've got 14 enough time to meet people who have a passion about voting 15 and want to improve the system, and recognize that voting 16 is the bedrock of our democracy. 17 It's that citizenship and that sense of 18 civic responsibility that I've tried to live up to in my 19 career, and I just feel very proud to be associated with 20 all of you. 21 You've heard a lot of great words about 22 NIST, and I want to call on Allan Eustis in a moment to 56 1 tell you all the great assistance they're going to provide 2 when you carry on your impossible assignments over the next 3 nine months. 4 MR. EUSTIS: Could we ask Dr. Schutzer if 5 he's still with us, that he give us a little back ground? 6 CHAIRMAN BEMENT: I'm sorry. Dan, are you 7 still with us? 8 DR. SCHUTZER: I'm still with you. I can 9 only be on for another hour or so. I apologize for not 10 being with you. I'm please to be -- I believe I have no 11 voting experience except as a voter. I believe that my 12 main background and interest was that I worked with banking 13 and I worked in many of our systems that you assume are 14 kinds of technologies, including our ATM technology. 15 I worked a long time in many different 16 engineering and standards organizations. And I've even 17 worked for the government as a previous career there in the 18 military defense systems. And most recently, the last 19 position I had there, which is on the opposite side of the 20 coin, I was the technical director for Naval Intelligence. 21 So I know a little bit about how to break 22 and penetrate systems, as well as how to protect them. I 57 1 really look forward to what sounds to me like it's going to 2 be a fascinating and very important function. 3 CHAIRMAN BEMENT: Thank you, Dan. I now 4 call on Allan Eustis, and then Craig Burkhardt, and then 5 Alice McKenna. 6 MR. EUSTIS: Thank you, Dr. Bement. Good 7 morning. I'm Allan Eustis. I'll be short, because I am. 8 Along the lines of the short remarks that were made 9 earlier, I would like to say that I learned and you will 10 learn with Commissioner Degregorio and myself sharing this 11 trait in common, that you don't mess with the short people. 12 But other than that, I am extremely 13 gratified that we are well ahead of schedule. I would 14 recommend to the Chair that after we finish our remarks, 15 myself, Mr. Burkhardt, and Ms. McKenna, that we break at 16 that time, which will probably be about 20 of, a quarter of 17 11. And using the discussing, I will extend the break from 18 30 minutes to 40 minutes. 19 For the public in attendance, there is a 20 cafe downstairs in this building. And there's a Subway, if 21 you walk over to -- there's a McDonald's too, I think. 22 Basically, if you want to go to Subway, you can walk to the 58 1 corner of 12th, go south on 12th, and it's on the left. 2 Either buy your food and eat it there, or bring it back. 3 For all of the members of the Technical 4 Guidelines Development Committee who have given us your 5 names, do not get up from the table when we break because 6 we need to collect money from you. The days of the ten 7 cent lunches when I went to grade school are gone 8 unfortunately. If it was, I would pay for it all. 9 I thank you all for being here. I am going 10 to use a term that Dr. Bement and Commissioner Degregorio 11 used, this is, this is really a dream team. I couldn't 12 think of a better group of individuals with diverse talents 13 as yours to come together for problems which almost seem 14 insoluble. Hopefully we will come up with recommendations. 15 Along that line, I do believe that in nine 16 months you will give birth to recommendations, but that 17 will be unusual in the fact that we will not have to 18 involve gynecologists in the process, that I'm aware of. 19 So we will do our work; we will make the 20 Chairman and the Commissioners, I hope, proud of our 21 product. I believe we will. And myself and Craig 22 Burkhardt are here to assist. 59 1 But let me say that the reason that this 2 meeting is taking place as smoothly as it is is because of 3 my assistant, Mary Floyd, who you'll all get to know quite 4 while. If Mary hadn't been here, we wouldn't be here. So 5 thank you, Mary, from the bottom of my heart. It really is 6 amazing that we did this in less than three weeks. 7 I would like to thank Paul Degregorio, and 8 say that I've enjoyed working with you, and look forward to 9 continuing our work relationship. I've enjoyed working 10 with all the Commissioners, which goes back to January 11 actually, before when he came to our symposium. So I think 12 we will move forward and we have made great strides, and we 13 have a lot of very important business to accomplish. 14 By the way, for the public that's here, and 15 for those not in attendance but reading the transcript, 16 early next week -- and you'll find in your packet a 17 template for our Technical Guidance Development Committee 18 web page. The website is Http://vote@NIST.gov/ and this is 19 case sensitive, capital TGDC.htm. On that page will be the 20 charter, members, public testimony, hearings of 21 transcripts, comments and position statements that have 22 been submitted through voting@NIST.go, which is the e-mail 60 1 address and any other relevant news to be totally 2 transparent as we move forward. 3 This afternoon, we will discuss a little bit 4 about the subcommittees, the potential subcommittee 5 structure that may be adopted by this Committee. I want 6 you all to know that there will be subject matter experts 7 available to each Committee from NIST, and some of those 8 people you will hear from this afternoon. 9 And again, I think a lot of where we are 10 right now is from the starting position. With a lot of 11 background knowledge it's due to folks like Dr. Laskowski, 12 David Alderman, Mark Skall, Lynne Rosenthal, and Jeffery 13 Horlick, and you'll meet some of the others, perhaps during 14 the break you'll get the chance to meet them as well if you 15 haven't. We are here to assist in any way possible to 16 officially accomplish the tasks that we need to accomplish. 17 I appreciate the trust of both Dr. Bement and 18 Dr. Semerjian in my work. It's made my life a lot easier, 19 and I think that we'll continue to move forward and do 20 quite well. The final person -- actually they're two 21 people, Dr. Zevin recently retired, and I think she's met 22 many of you on the Committee. It's been a lot of work to 61 1 lay the foundation for where we are. 2 Our general counsel, as appointed by 3 Dr. Bement, so I call on Craig to make a few brief remarks, 4 and then keep this moving on. I thank you for all the help 5 he has given me. 6 MR. BURKHARDT: Thank you, Allan. We're now 7 going to go into a brief presentation on the legal aspects 8 of your service on this Federal Advisory Committee. I hope 9 that this will actually be the first and last presentation 10 agreed with (inaudible) because what we want to do is we 11 want to keep a lawyer in on the procedural things, in the 12 background, which would allow you to move forward very very 13 repeatedly in technical deliberations, then eventually the 14 writing of your policy and the proposals for any overall 15 agenda. 16 Essentially, as a Federal Advisory 17 Committee, you are in the business, if you will, to provide 18 advice to the Federal Government and senior officials in 19 the Federal Government. 20 Shortly, Alice McKenna, who is an expert on 21 Federal Advisory Committee laws is going to give you a 22 presentation on some of the do's and don'ts on how to 62 1 conduct yourselves as a Committee, and as subcommittees, 2 which we'll be talking about in a moment. They're also 3 personal activities. There are a lot of Federal Advisory 4 Committees (inaudible) and generally, speaking function 5 without any legal difficulties or complications. 6 So, we will try to be very client-centered 7 in managing your activities as opposed to erecting barriers 8 in your way. 9 Keep a couple of things in your mind. 10 Dr. Bement has agreed to present to you the subcommittee 11 recommendations, which he has to split this group up into 12 three-working groups, subcommittees, which are functioned 13 over the next six months. 14 A couple of things to keep in mind as we're 15 getting together as an overall preview, such as we have 16 today. We are regulated by certain rules that require us 17 to give public notice in the Federal Register seven days in 18 advance. And in taking back this meeting, (inaudible) 19 public availability requirements. 20 We're going to be talking about 21 subcommittees. The subcommittees will typically be four or 22 five people. One thing we need to understand about the 63 1 subcommittees right from the get-go is they have no 2 independent authority. These are essentially going to be 3 smaller working groups that are going to study specific 4 subject matter issues and bring proposals back to this 5 group. 6 So this is the only group that actually has 7 authority to vote and adopt proposals, which would 8 eventually go back to the Federal Election -- to the 9 Election Assistance Commission and it's members. 10 The subcommittees will operate slightly 11 differently than this Committee. When I say slightly 12 differently, all I mean is that notices, which will be 13 given in sometimes immediate circumstances. The intent is 14 to keep the subcommittee's activities very transparent and 15 very public to properly serve the vendor community, the 16 election administration community, as well as the testing 17 community and the press, because we want to be open. 18 That's the preference of the Election Assistance Commission 19 as well as NIST. We hope that it will be in your 20 preferences as well. 21 A couple of the things that Alice talked 22 about that you will want to keep in mind is that the 64 1 subcommittees will have a little flexibility. So these 2 will be smaller working groups, but you'll have to be 3 somewhat more nimble, you'll need to react quickly to 4 information that comes, and the ability to have meetings by 5 telephone conference and so forth. 6 I want to press and the public who was listening 7 in to know that we will be giving notices by web site. And 8 the web site as Mr. Eustis mentioned is vote.NIST.gov, and 9 that could mean that there will be duplicate postings of 10 all notices of subcommittees' activities on the Election 11 Assistance Commission web site, which is www.EAC.gov. 12 We will not be unfair to the public at large 13 or the press. We will give as advanced notice as we can. 14 But I do advise those of you who are in the room, and those 15 who might be listening in who have great interest, to 16 regularly check in on those web site postings, keep track 17 of what will be going on over the next six months, which 18 will essentially be subcommittee hearings, subcommittee 19 deliberations, and eventually subcommittee formulation of 20 proposals, which will come back to this overall Committee 21 probably in about six months. 22 As Dr. Bement will mention, we probably 65 1 won't be getting back together again as an overall 2 Committee until sometime in January or February of next 3 year. 4 So I will not go into any other details on 5 the subcommittees because DR. Bement will be presenting a 6 detailed proposal to you this afternoon. But those are the 7 things that I want you to be a aware of as you listen to 8 Ms. McKenna's presentations on the Committee, which we 9 comprised, and the subcommittees, which will be comprised. 10 Alice McKenna will now come forward. She is 11 a senior attorney at the Department of Commerce. She's an 12 expert on Federal Advisory Committee activities and the 13 regulations that govern the activities of those committees. 14 Alice? 15 MS. MCKENNA: Thank you, Craig. Good 16 morning. 17 As Craig said, I'm Alice McKenna from the 18 Department of Commerce Office of General Counsel. We are 19 the parent agency to the parent agency of NIST. 20 I have a handout that's going around the 21 room. I'll please ask the DFO to take two copies because 22 one copy is going to need to go in the public file, the 66 1 other is going to be maintained on this Committee as is 2 require by the FACA. 3 While Craig is also handing out the handout, 4 I'll tell you that I have served as an election officer, 5 and I know what it's like to spend 14 hours on your feet, 6 and then at the end of the 14 hours, have to do a meeting. 7 And while NIST is a scientific and computer 8 and mathematics based organization, I have to tell you that 9 as an attorney for the Department of Commerce, I am not. I 10 want to a large midwestern university that specialized in 11 electrical engineering and computer science, and I majored 12 in political science and German literature. 13 If I had been any good in math at all, I 14 would have gone to the electrical engineering department 15 and had myself a much higher paying job that I have as a 16 person with two Bachelor's of Arts Degrees from the 17 University of Illinois. 18 You all now have a handout, and I will turn 19 your attention to that. I'm going to hit the highlights. 20 These are some basic bedrock principles that define how 21 advisory committees work under the Federal Advisory 22 Committee Act, which is a statute I've come to specialize 67 1 in in the last 13 years or so at the Commerce Department. 2 The main principle of any advisory committee 3 is it is simply that. It is advisory only. Most 4 committee's, unless otherwise directed by Congress, have no 5 operational functions. They don't make and implement 6 government decisions. They advise the government and the 7 government relies on the advice in one way or another. 8 If there is any question about whether a 9 committee is operated in an advisory capacity, or whether 10 it is operating within the scope of its charter as dictated 11 by the statute, those questions should be referred to the 12 designated federal officer, who will then contact the 13 relevant government officials. 14 As I just stated, your advisory duties are 15 sort of limited by the statute. The general purpose is to 16 assist the Executive Director of the Election Assistance 17 Commission in the development of voluntary voting system 18 guidelines. 19 The Committee is statutorily directed to 20 provide its first set of recommendations to the Executive 21 Director not later than nine months. That work's out to 22 April 2005. After that, the Committee shall cause to have 68 1 published any guidelines on voluntary voting systems that 2 are adopted by the Commission. 3 Your role as members, you're appointed, you 4 serve at the pleasure of the Commission and the Director of 5 the National Institute of Standards and technology. 6 There is one important don't. You do not 7 advise Congress or engage in what we call grassroots 8 lobbying. These activities are first, outside the scope of 9 your charter, and for us, for the government and agencies 10 involved, may also implement other prohibitions against 11 grassroots lobbying. None of this, however, applies to 12 what you do as private citizens. I'm talking strictly in 13 your capacity as Committee members. 14 There's two kinds of Committee members under 15 the ethics statues. Some of you are appointed as "Special 16 Government Employees" because you serve as an expert in an 17 individual capacity. Others of you have been, however, 18 appointed in what we call a "representative capacity" to 19 bring to the table the views of a particular group or 20 organization. The Commission -- the Election Assistance 21 Commission is responsible for identifying who is who. 22 The ethics rules, I'm not an ethics' 69 1 attorney. I have a short summation of how the ethics rules 2 apply to the two different groups. 3 "Those who are serving in an individual, 4 expert capacity, are Special Government Employees, and are 5 subject to the Federal criminal conflict of interest 6 statutes and rules." 7 This is summarized very generally. "As such 8 they are prohibited from participating in particular 9 matters that may have a direct and predictable effect on 10 their financial interests or on those of a spouse, minor 11 child or general partner. They also cannot be registered 12 as a foreign agent, and I'm sure you all know that. 13 It's a requirement -- when somebody works 14 for a foreign government or a wholly-owned subsidiary of a 15 foreign government, there's a requirement to register with 16 the Justice Department and with the government under the 17 Foreign Agents Registration Statue. 18 People who serve in a representative capacity 19 are not subject to criminal conflict of interest statues; 20 nevertheless, they are bound by general rules including the 21 prohibition on use of government affiliation, resources or 22 information. 70 1 Advice on any questions on standards of 2 conduct matters for both Special Government Employees and 3 representatives will be provided by the Election Assistance 4 Commission, and through the offices that support it. 5 Meetings of your Committee. There is a core 6 requirement in your charter. The Committee shall not act 7 in the absence of a quorum or a majority of the members, 8 not having a conflict of interest in. If you have a 9 conflict of interest, you must recuse yourself from 10 discussion of the matter. 11 A quorum is defined as a simple majority of 12 all people who don't have a conflict. Half will be enough. 13 Committee meetings only occur when called by 14 a federal officer, usually the DFO. All meetings are open 15 to the public, and public notice must be given to the 16 public 15 days in advance, including web site notices. 17 While the Federal Advisory Committee Act 18 provides for limited exceptions to the open vetting 19 requirement, those have to be decided on advice approved by 20 the government agency involved. 21 Meetings should allow a reasonable 22 opportunity for the public to make comments. The public 71 1 may also file written statements with the Committee at any 2 time. 3 All materials that are made available to the 4 Committee, my handout prepared for the Committee, handouts 5 prepared by the Committee, minutes and transcripts of 6 meetings will be filed in files that are kept by the 7 government and available to the public at any time upon 8 request. This would be like in a reading room format. 9 An exception to that would be if it's 10 material that's exempt under the Freedom of Information 11 Act. 12 Meetings can be in person, via 13 videoconference or conference call. The crucial element 14 for things like video conference, and conference calls, the 15 public must be afforded access to the deliberations by 16 having a site where the public can attend, a room where the 17 public can come in and listen to a speakerphone or 18 something like that. 19 There are some things that don't rise to the 20 level of a meeting. One of those is where Committees 21 members meet and have a conference call to compare 22 schedules, things like that. Those are administrative, 72 1 housekeeping matters, and are not necessarily required to 2 be public. 3 We'll talk briefly about subcommittees that 4 may be necessary to accomplish the Committee's function. 5 Subcommittees are a very important function of the 6 Committee. 7 Subcommittees, again, are sort of a pyramid 8 being the start basically of the pyramid, feeding into the 9 middle level, being this Committee, and then the top being, 10 the government agency. 11 Subcommittees that meet to advise the 12 government directly may be in violation if they haven't had 13 their meetings noticed 15-plus days in advance in the 14 Federal Register. 15 The Committee must deliberate on what the 16 subcommittee gives them to make sure that it complies with 17 the FACA prior to advising the government based on the 18 subcommittee's work. 19 Committees or subcommittees that contemplate 20 using people that are not at the table today, but other 21 people, we would have to look at that separately. 22 Any documents that are provided to the 73 1 Committee by the subcommittee, also have to go into the 2 public file. 3 Some purely administrative matters, that all 4 that's standing between you and a break. Administrative 5 support for the Committee is provided by the Election 6 Assistance Commission. 7 As NIST representatives, you know, members 8 are not compensated for their services, but travel, 9 including per diem in lieu of subsistence, may be paid for 10 upon request. 11 All Government-funded travel must be at the 12 Government's request and must involve the provision of a 13 direct service to the Government. 14 Election Assistance Commission attorneys 15 provide legal advice to the Commission, including the DFO, 16 with the exception of standards of conduct issues. The 17 government attorneys do not serve as legal advisors to the 18 individual Committee members, we only advise the Committee 19 as a whole. 20 If you have any questions about what to do, 21 or if there is something that comes up, please, direct your 22 questions to the Committee's Designated Federal Officer. 74 1 If that person can't answer you on the spot, that 2 individual is invited to contact Craig or myself to get the 3 answer, and that DFO will then report back to you. 4 At this time do you have questions? 5 MR. ELEKES: You note that the documents 6 would be placed in a central location. Should a person 7 with a disability require alternative format, is there 8 consideration to provide those documents in alternate 9 electronic format such as audio or braille? 10 MS. MCKENNA: I'll refer that question to 11 Allan Eustis. 12 MR. EUSTIS: Yes, we will provide that 13 through my office. You can make the request through 14 e-mail, vote@NIST.gov, or through any of the Commissioners 15 or myself. We will provide that in a commonly accepted 16 format, either disk or document. We will do that in 17 whatever is recognized as the most useful format for that, 18 and we'll take recommendations as well on other formats. 19 CHAIRMAN BEMENT: J. R. 20 MR. HARDING: Thank you, Mr. Chairman, J.R. 21 Harding. A follow-up on that. Perhaps on the bottom of 22 all of our agenda's, for alternate format needs, I request 75 1 that a phone number and a contact person would be 2 identified to make those formal requests. 3 MR. EUSTIS: So noted. 4 MR. HARDING: I've got that language for 5 you. 6 CHAIRMAN BEMENT: Any other questions by 7 members? 8 While we've earned ourselves a little bit of 9 slack time, we do have members of the public here today. 10 I would like to invite them, if they would, to introduce 11 themselves. 12 Let's just start in the front row and just 13 move to the right and to the back. 14 PAUL WOLTE: Paul Wolte, Sequoia Voting 15 Systems. 16 DAVID: David from the staff of the Access 17 Board. I'm here to provide support to Mr. Harding. 18 DR. LASKOWSKI: Dr. Sharon Laskowski from 19 NIST. 20 MR. ALDERMAN: David Alderman from NIST. 21 MR. SKALL: Mark Skall from NIST. 22 MS. ROSENTHAL: Lynne Rosenthal from NIST. 76 1 MS. CHIN: Candace Chin. Senate. 2 MS. SCHNEIDER: Elizabeth Schneider 3 MR. AUBLE: Daniel Auble, Common Cause. 4 MR. WACK: John Wack, NIST. 5 MR. WHITE: Douglas White, NIST. 6 MR. KELSEY: John Kelsey, NIST. 7 MR. SMOLKA: Richard Smolka, Election 8 Administration Reports. 9 CHAIRMAN BEMENT: Thank you very much. Well 10 we have now earned our break. It's a quarter to 11. We 11 will reconvene at 11:30. 12 (Recess from 10:45 to 11:30) 13 DR. RIVEST: (...) In terms of my business 14 of serving on this Committee, I think that voting is a 15 great national important civic duty. It's my duty to help 16 participate on my Committee. (Inaudible). 17 CHAIRMAN BEMENT: Thank you, Ron. I would 18 like to call your attention to a tab in your book called 19 "Organization." And what we're going to be talking about 20 for the next 30 minutes or so is organizational 21 recommendations, which is essentially our procedural 22 roadmap of how we're going to be conducting business. And 77 1 I'll go through this piece by piece so we'll have an 2 opportunity to have discussions as we go. 3 The first part, we pretty well covered this 4 morning all the way through the first paragraph. 5 The second paragraph under Committee 6 structure, there's a recommendation that the business of 7 the TGDC is to accomplish the three subjects, specific 8 subcommittees comprised only of TGDC members. 9 Each subcommittee will have a member chair. 10 For technical assistance, each subcommittee will have a 11 NIST executive, familiar with the subject matter assigned 12 to it. I propose that such committees are the Subcommittee 13 on Security and Transparency; the Subcommittee on Human 14 Factors and Privacy; and the Subcommittee on Core 15 Requirements and Testing. 16 In the back pocket of your notebooks is a 17 signup sheet which briefly describes the recommended scope 18 of each subcommittee. 19 MR. HARDING: Mr. Chairman, do you require a 20 motion for those three subcommittees? 21 CHAIRMAN BEMENT: We're going to get to that 22 in a little bit. So I would ask you, as we go along if 78 1 you're already familiar with this, go ahead and indicate 2 your personal choices. 3 Now, because of financial disclosure forms 4 for TGDC members have not been totally completed by this 5 meeting, it is not appropriate at this time to announce 6 subcommittee chairs, but that will be done very shortly. 7 And in selecting the members for the subcommittee, I'll pay 8 particular attention to your first choices, try to get as 9 much distribution as we can. 10 Any comments or discussions on that part of 11 our Committee structure? 12 All right. Going on to Committee 13 operations. The duties of subcommittees will be to engage 14 in information gathering and analysis, including but not 15 limited to conduct that probably carries the analysis of 16 existing best practices, specifications, and standards. 17 As you know, there is a body of standards 18 already in place. Subcommittees will prepare proposals to 19 the entire TGDC, and only those (inaudible) will be 20 forwarded to the Election Assistance Commission, subject to 21 prior constraints. 22 The subcommittees' meetings can occur in 79 1 person or by telephone conference. (Inaudible) subject to 2 availability of funding essential subcommittee activities 3 accomplished in concert with the staff. The National 4 Institute of Standards and Technology will provide 5 technical assistance, including first, the conduct of 6 public hearings to accept oral and written testimony. 7 Second, the acceptance of analysis and 8 written materials by NIST web capabilities. 9 Third, the survey and analysis of applicable 10 best practices, specifications, and standards. And, 11 actually, identification of areas requiring writing and 12 viewing of a revised best practices, specification, and 13 standards. And the prioritization projects. Writing 14 proposals, best practices, specifications or standards, and 15 communication of project selection and progress to the 16 TGDC. 17 Subcommittee work product will be shared on 18 a regular basis will all TDGC members, and subcommittee 19 activities and communications must be in compliance with 20 legal requirements as approved by the Chair. 21 Now, some of you have broad interests in 22 almost every activity that the Committee is engaged in and 80 1 may like to be active on more than one subcommittee, and we 2 can certainly accommodate that, subject to your 3 availability. But recognize that any products from the 4 subcommittees will be fully vetted and discussed at the 5 Committee level before it can go forward. 6 Also, since we will be making arrangements 7 for the public to be involved, either by teleconference or 8 by other means, such as in other languages that monitor 9 subcommittee activities, if you so wish. 10 So I think there will be ample opportunity 11 to maintain on an ongoing basis, the familiarity with all 12 activities of the Committee, even though, we have to parse 13 it into these three subcommittees in order to carry out the 14 work of the full Committee. 15 Having said that, are there any other 16 comments or questions you might have on the function of 17 the subcommittees? If not, at an appropriate time and this 18 will likely be January/February 2005, the TGDC will hold a 19 multi-day session during which work product of the 20 subcommittees will be presented in the form of a series of 21 resolutions. 22 The resolutions will be will bated for 81 1 potential amendments and adoption. Adopted resolutions 2 will thereafter be refereed to this staff for technical 3 assistance and clean up. And this staff will return all 4 adopted resolutions, after providing technical assistance 5 and clean up, to the TGDC for final review and approval. 6 In April of 2005, adopted resolutions and 7 related material will be presented to the EAC Executive 8 Director in the form of the statutory mandated first set of 9 recommendations. 10 Any comments or questions on that? 11 Work product form and priorities provisions 12 with HAVA effectively require the TGDC to submit the first 13 set of recommendations to the EAC Executive Director in 14 April of 2005. The form of the recommendations will vary 15 for each subject matter, depending on the existence of 16 current best practices, specifications, or standards. For 17 example, some products may consist of reviewing quality to 18 test the standards already developed by other 19 organizations. 20 Recommendations related to such projects 21 will consist of suggested standards, capable of immediate 22 implementation. The subject varies such as human factors, 82 1 where there are viewing systems specifications or standards 2 related to voting systems. Recommendations may consist 3 primarily of best practice guidelines and protocols for 4 validation of independent testing and government testing 5 activities. 6 And going on, HAVA requires election 7 authorities to be in compliance with certain requirements 8 in January 2006. I recommend to the TGDC and its 9 subcommittees, your first priority projects that will be 10 useful to the manufacturing testing laboratory and election 11 administration communities. In other words, that's a 12 caution that we need to stay focused and prioritized, and 13 not take a frontal approach to the whole field, but perhaps 14 approach it by segment. 15 If there are any general comments or 16 additions or corrections to these procedures, we can take 17 these up now or during the time when we entertain motions. 18 So let's move on. 19 I would like to entertain a motion that 20 three subcommittees be formed for gathering and analyzing 21 information -- 22 MR. HARDING: So moved. 83 1 CHAIRMAN BEMENT: Let me read the whole 2 motion. The subcommittees shall be comprised only of TGDC 3 members. The subcommittees shall propose resolutions to the 4 TGDC on best practices, specifications, and standard. The 5 subcommittees shall be named: Security and Transparency, 6 Human Factors and Privacy, and Core Requirements and 7 Testing. 8 Do you wish to move that? 9 MR. HARDING: Yes, I do. 10 CHAIRMAN BEMENT: We have a motion. Is 11 there a second? 12 MR. ELEKES: Second. 13 CHAIRMAN BEMENT: We will take a roll call 14 on those motions. If you will read the roll. 15 MR. BURKHARDT: Okay. We will designate 16 this as Committee Resolution Number 1. I will repeat the 17 motion for purposes of the court reporter to check her 18 record. 19 The motion is as follows: That three 20 subcommittees be established to gather and analyze 21 information. The subcommittees shall be comprised only of 22 TGDC members. The subcommittees shall propose resolutions 84 1 to the TGDC on best practices, specifications, and 2 standards. Subcommittees shall be named: Security and 3 Transparency, Human Factors and Privacy, and Core 4 Requirements and Testing. 5 I will now call the roll call. I will only 6 be calling the names of people whose financial disclosure 7 documents have been approved. 8 You will answer "aye" or "nay" or state if 9 you belive you have a conflict of interest. 10 Dr. Bement? 11 CHAIRMAN BEMENT: Aye. 12 MR. BURKHARDT: Davidson? 13 MS. DAVIDSON: Aye. 14 MR. BURKHARDT: Miller? 15 MS. MILLER: Aye. 16 MR. BURKHARDT: Purcell? 17 MS. PURCELL: Aye. 18 MR. BURKHARDT: Harding? 19 MR. HARDING: Aye. 20 MR. BURKHARDT: Elekes? 21 MR. ELEKES: Aye. 22 MR. BURKHARDT: Caldas? 85 1 MS. CALDAS: Aye. 2 MR. BURKHARDT: Berger? 3 MR. BERGER: Aye. 4 MR. BURKHARDT: Williams? 5 MR. WILLIAMS: Aye. 6 MR. BURKHARDT: Craft? 7 MR. CRAFT: Aye. 8 MR. BURKHARDT: Quesenbery? 9 MS. QUESENBERY: Aye. 10 MR. BURKHARDT: Mr. Chairman, that's 11 yes 11 votes, no nay votes. I believe that you can declare that 12 as having been adopted. 13 CHAIRMAN BEMENT: I declare that the 14 resolution has been adopted. 15 I would like now to entertain a motion that 16 the Chair survey the interest of TGDC members, and 17 thereafter appoint members and chairs of the subcommittee. 18 Is there a motion? 19 MR. BERGER: So moved. 20 CHAIRMAN: Is three a second? 21 MS. PURCELL: Second. 22 CHAIRMAN BEMENT: Any discussion on the 86 1 motions? If not, please call the roll. 2 MR. BURKHARDT: This will be designated 3 Committee Resolution Number 2. I'll repeat for purposes of 4 the court reporter, the motion at this time: 5 That the Chair survey the interests of TGDC 6 members, and thereafter appoint the members and chairs of 7 the subcommittees. 8 Roll call. Bement? 9 CHAIRMAN BEMENT: Aye. 10 MR. BURKHARDT: Davidson? 11 MS. DAVIDSON: Aye. 12 MR. BURKHARDT: Miller? 13 MS. MILLER: Aye. 14 MR. BURKHARDT: Purcell? 15 MS. PURCELL: Aye. 16 MR. BURKHARDT: Harding? 17 MR. HARDING: Aye. 18 MR. BURKHARDT: Elekes? 19 MR. ELEKES: Aye. 20 MR. BURKHARDT: Caldas? 21 MS. CALDAS: Aye. 22 MR. BURKHARDT: Berger? 87 1 MR. BERGER: Aye. 2 MR. BURKHARDT: Williams? 3 MR. WILLIAMS: Aye. 4 MR. BURKHARDT: Craft? 5 MR. CRAFT: Aye. 6 MR. BURKHARDT: Quesenbery? 7 MS. QUESENBERY: Aye. 8 MR. BURKHARDT: Mr. Chairman, the voting is 9 11 ayes, 0 nays. I believe you can declare Committee 10 Resolution Number 2 passed. 11 CHAIRMAN BEMENT: I declare that Resolution 12 Number 2 passed. 13 I would like now to entertain the motion 14 that resolutions prepared by subcommittees be considered by 15 the TGDC. Resolutions adopted by the TGDC shall be 16 referred to NIST for technical assistance and editing. 17 Upon the return from NIST, the TGDC shall review the 18 resolutions to confirm they conform to its intent. 19 MR. HARDING: I would move that. 20 CHAIRMAN BEMENT: Okay. There is a motion. 21 Is there a second? 22 MR. ELEKES: Second. 88 1 CHAIRMAN BEMENT: Any discussion? Very 2 well, please call the roll. 3 MR. BURKHARDT: This will be designated 4 Committee Resolution Number 3. I will repeat it for 5 purposes of the court reporter. It is designated as 6 follows: 7 The motion is that: 8 Resolutions prepared by subcommittees be 9 considered by the TGDC. Resolutions adopted by the TGDC 10 shall be referred to NIST for technical assistance and 11 editing. Upon the return from NIST, the TGDC shall review 12 the resolutions to confirm that they conform to its intent. 13 Roll call. Bement? 14 CHAIRMAN BEMENT: Aye. 15 MR. BURKHARDT: Davidson? 16 MS. DAVIDSON: Aye. 17 MR. BURKHARDT: Miller? 18 MS. MILLER: Aye. 19 MR. BURKHARDT: Purcell? 20 MS. PURCELL: Aye. 21 MR. BURKHARDT: Harding? 22 MR. HARDING: Aye. 89 1 MR. BURKHARDT: Elekes? 2 MR. ELEKES: Aye. 3 MR. BURKHARDT: Caldas? 4 MS. CALDAS: Aye. 5 MR. BURKHARDT: Berger? 6 MR. BERGER: Aye. 7 MR. BURKHARDT: Williams? 8 MR. WILLIAMS: Aye. 9 MR. BURKHARDT: Craft? 10 MR. CRAFT: Aye. 11 MR. BURKHARDT: Quesenbery? 12 MS. QUESENBERY: Aye. 13 MR. BURKHARDT: Mr. Chair, the votes are 11 14 yes, 0 no. I recommend that you declare the resolution has 15 been adopted. 16 CHAIRMAN BEMENT: I declare Resolution 17 Number 3 is adopted. 18 Next, I'd like to entertain the motion that 19 adopted resolutions and appropriate explanatory materials 20 comprise the "first set of recommendations" mandated by the 21 Help America Vote Act. 22 MS. DAVIDSON: So moved. 90 1 CHAIRMAN BEMENT: There's a motion. 2 MR. HARDING: Second. 3 CHAIRMAN BEMENT: Any discussion on this 4 motion? If not, call the roll. 5 MR. BURKHARDT: I'm designating this motion 6 as Committee Resolution Number 4. 7 The motion for purposes of confirmation for 8 the court reporter is as follows: 9 Adopted resolutions and appropriate 10 explanatory materials comprise the "first set of 11 recommendations" mandated by the Help America Vote Act. 12 Roll call. Bement? 13 CHAIRMAN BEMENT: Aye. 14 MR. BURKHARDT: Davidson? 15 MS. DAVIDSON: Aye. 16 MR. BURKHARDT: Miller? 17 MS. MILLER: Aye. 18 MR. BURKHARDT: Purcell? 19 MS. PURCELL: Aye. 20 MR. BURKHARDT: Harding? 21 MR. HARDING: Aye. 22 MR. BURKHARDT: Elekes? 91 1 MR. ELEKES: Aye. 2 MR. BURKHARDT: Caldas? 3 MS. CALDAS: Aye. 4 MR. BURKHARDT: Berger? 5 MR. BERGER: Aye. 6 MR. BURKHARDT: Williams? 7 MR. WILLIAMS: Aye. 8 MR. BURKHARDT: Craft? 9 MR. CRAFT: Aye. 10 MR. BURKHARDT: Quesenbery? 11 MS. QUESENBERY: Aye. 12 MR. BURKHARDT: Mr. Chair, the votes are 11 13 yes, 0 no. I recommend that you declare Resolution Number 14 4 as adopted. 15 CHAIRMAN BEMENT: I so declare. 16 That completes the formal resolutions. 17 There's time now for discussion of the major issues and 18 timelines in our road map and anything else that you may 19 wish to discuss or any other motions that you may want to 20 put forward. 21 According to our agenda, I think the 22 prioritization time frame in terms of the three most 92 1 important recommendations is fairly within the next nine 2 months when we will have our first set of preliminary 3 recommendations, and that will be a priority effort. But 4 then, of course, ongoing, there will be activities toward 5 that up to and beyond 2006. There may be some discussion 6 about that. 7 As far as considering all aspects of voting 8 (inaudible) before we post voting. As well as the 9 interplay between people, technology, and process there may 10 be some discussion on that. What do we mean by holistic 11 approach? 12 The phase and (inaudible) of our work plan, 13 which is pretty well laid out in the organization right 14 now. If you'd just (inaudible) and then any next steps 15 that might be of interest to the Committee. In some 16 respects I think that (inaudible) of engagement were pretty 17 well covered this morning as far as the requirements under 18 the Sunshine Act that we can take up any questions then as 19 well. 20 And following our discussion, we will then 21 begin our background briefings in this (inaudible). Is 22 there anyone who may wish to entertain any of these 93 1 segments? 2 MS. DAVIDSON: I would just like to offer 3 since there are going to be subgroups, ANSI does have a 4 database of existing American National Standards and 5 standards that are not American National Standards but that 6 are recognized. 7 We can certainly make that data available 8 and produce specific reports, depending on the area of 9 interest of the subcommittee, so we can identify standards 10 that are existing or in process. And I would like to make 11 that offer across to all subcommittees, since I will be on 12 one and there will three. 13 CHAIRMAN BEMENT: Thank you very much. 14 MR. HARDING: Mr. Chairman, J. R. Harding. 15 In our motion 3 as well as 4, we spoke to specific 16 timelines, January/February of 2005, April of 2005, in 17 terms of the deliverable to this group. Related to the 18 deliverable perhaps of a work product in a subcommittee, 19 are there any timelines/expectations or just in that 20 January/February time frame and again in that April time 21 frame? 22 CHAIRMAN BEMENT: The only thing that comes 94 1 to mind J. R. Is that I think it's important that we get as 2 much up on the website as early as we can in terms of gaps 3 that we can identify in existing standards which need to be 4 addressed. Because normally in the standard development 5 process and (inaudible) you're talking about what, 12 to 18 6 months of effort on the flip side. 7 MR. HARDING: Yes. 8 CHAIRMAN BEMENT: So that getting the 9 standard development organizations attuned to the thinking 10 of the Committee earlier would be very helpful. 11 Clearly in your January meeting, when we 12 formulize our recommendations, that will also be an 13 important stage to help us through that process. 14 If there are questions that occur to you as 15 we go along throughout the afternoon, and as we hear the 16 NIST presentations, we will take time to have a discussion. 17 Oh, yeah, Steve. 18 MR. BERGER: I might just offer a thought 19 for our groups. I think one of the things we need to be 20 conscious of is the time required for other parties to 21 implement this. One of my particular concerns, given my 22 background is, the time vendors would need to respond to 95 1 the direction we might take. 2 So in some of the areas that I know we will 3 being addressing as soon as we could identify the direction 4 that we will address them, so the vendors could start doing 5 background and have things like information on usability, 6 information on reliability of the products and process 7 along with the speed they will need for ultimate 8 implementation. That's exactly what you said that making 9 sure that we signal that on the web sites so outside 10 parties can start making their points. 11 CHAIRMAN BEMENT: As we get to the NIST 12 briefings, which will be pretty much on the procedures for 13 development standards, any question that you might have 14 (inaudible) how performance based standards are developed 15 as compared to process standards, product standards, any 16 other kind of standard and, basically, we will try and 17 enter into that discussion. 18 So why don't we start with the first 19 presentation, which will be by Sharon Laskowski, who did a 20 lot of the yeomen work. 21 MR. HARDING: Mr. Chairman, is this found 22 within our background tab or is it still -- 96 1 CHAIRMAN BEMENT: Yes, it's in the 2 background. 3 Sharon was lead author of NIST HAVA Mandated 4 Human Factors Report to Congress. And she's also a 5 scientist for NIST Information Technology Laboratory. 6 During her briefing, she'll discuss the findings of her 7 report, "Improving the Usability and Accessibility of 8 Voting Systems and Products." 9 DR. LASKOWSKI: Allan, how do you want us -- 10 do you want us to stand here for our presentations or? 11 MR. EUSTIS: Yes, maybe J. R. Doesn't mind 12 if you put your stuff on the table; is that right? 13 DR. LASKOWSKI: And how much time, now that 14 we're on a slightly different schedule? I could try to 15 keep it to 10 minutes or an hour. 16 MR. EUSTIS: I would say try to keep it to 17 15 minutes, and then if you can finish earlier than that, 18 there could be time for some questions from the TGDC. 19 DR. LASKOWSKI: Okay. I'm going to read the 20 slides for anyone that might have vision problems and 21 describe any diagrams. 22 Does anyone in the audience need to look at 97 1 me while I'm speaking? If so, feel free to move or just 2 give me a cue so that I can be sure to be looking towards 3 you. Okay. 4 Let me first just take off on the running 5 joke of this morning, I have three teenagers, now that my 6 son just turned 13 two weeks ago, and they view standards 7 just as guidelines. But what I found is critical that they 8 need some notion of how I'm going to evaluate their 9 performance against those standards and guidelines, and 10 that's sort of the theme in probably not just my talk, but 11 in the other NIST talks that you'll hear about. 12 I'm talking about improving the usability 13 and accessibility of voting systems and products based on 14 what was put together in the Human Factors report. Okay. 15 When we talk about human factors, it's 16 important to look at all the users in this system. And 17 I've got this diagram called "Voting Echo System" that was 18 develop at a workshop a few weeks ago at the Usability 19 Professional Association on Voting Systems that had about 20 14 people, rather international crowd. That was a day 21 workshop. 22 It was very productive, but one of the first 98 1 things we did is sit down and say, okay, who do we have to 2 be concerned about? I think this approach to who are the 3 users of the voting system is important not just for the 4 usability and accessibility, but also when you look at any 5 aspect of the system, security, elsewhere. 6 Any implementation has to be evaluated in 7 terms of the user of the system, if it's going to work 8 properly, have people have confidence that it's working. 9 In the middle circle, we have voters, of 10 course, because they're the critical user here. We also 11 have poll workers and around the perimeter of that we've 12 got voter registration workers, challenger judges, 13 observers, warehouse workers, support staff, election 14 officials, postal workers, if there's absentee, to be 15 delivering absentee, etcetera. 16 And then around the outside perimeter, we've 17 got voter advocacy groups, elected officials, legislators, 18 the press, candidates, vendors. All those people are part 19 of a voting system, and we need to keep that in mind as we 20 design and develop standards that are testable. 21 So just to set the stage, the human factors 22 usability perspective on voting systems with respect to the 99 1 voters is comprised of the cognitive and physical nature of 2 the voters, the physical environment, the psychological 3 environment, the voting product itself. And usability is 4 determined by the demands of the system and the voters 5 ability to perform under those demands. 6 Now, do you know how to measure 7 accessibility and usability? There are a couple of 8 definitions here. 9 Accessibility is the degree to which the 10 system is available to and usable by individuals with 11 disabilities. 12 So not just access but usability, can they 13 actually cast a vote the way they intended if they have a 14 disability, or a language issue? 15 Usability -- this is the standard 16 definition -- is a measurement of the effectiveness, 17 efficiency, and satisfaction achieved by a specified set of 18 users performing specific tasks with a given product. 19 Standard metrics, for example, for 20 effectiveness, efficiency, and satisfaction are things like 21 that counting errors that causes a vote that is cast not as 22 intended, or a vote not cast, that is (errors prior to 100 1 success) for the voting systems -- well, if they do 2 eventually cast the vote as intended, errors might 3 contribute to -- prior to success, contribute to taking 4 more time using the system. But that is also a critical 5 aspect because we don't want long lines at the polling 6 places. You can measure subjective satisfaction as a 7 measure of errors as well. 8 The process for user-centered design process 9 goes from design to measurement. You start with a 10 user-centered design when a system is being built. You do 11 diagnostic usability evaluation to advise the design 12 process as you iterate and build a bit of pride, and then 13 at the end, you do a performance test called usability 14 testing. Now, what's the state of usability of 15 U.S. voting systems. In general, voting systems have not 16 been measured for usability, nor have they been developed 17 using a user-centered designed process. So we really do 18 not know the degree to which voters cast their vote not as 19 they intended due to confusion with the user interface. 20 There's been a number of observations by 21 people at either Cal Tech and MIT Voting Technology 22 Project, Paul Hermanson Digital Government Funded Project 101 1 on Voting, others such as Doug Jones and many usability 2 professionals who note that there do appear to be some 3 potential problems with the designs that we see that are 4 causing errors. 5 Now, when we developed standards through 6 usability and accessibility, the design standards or how 7 the product is designed; for example, specified font size 8 or ballot instructions. 9 Performance standards are how the product 10 actually functions. So specified functional characters are 11 no over voting is allowed by the system. You can test that 12 usually by demonstration. There will be users and a lot of 13 voters to go through the system to test that. 14 Counting errors, and measuring performance, like, time 15 to cast the vote, failure in casting vote as intended, that 16 requires measuring with users against benchmarks. 17 How do we know if the system passes or fails when 18 you measure these things? You need to have realistic 19 benchmarks that go along with the standards in order to 20 test the performance standards. 21 And doing that kind of testing, you also need 22 sample ballots of different complexities if we're going to 102 1 follow this ITA process that we see that's currently in 2 place. They've got to have some standards ballots to test 3 with. And you need well-defined test protocols and user 4 groups. 5 So, if we want to measure for this qualification 6 testing and certification testing that goes on, well, right 7 now we don't have a protocol for measuring the usability of 8 voting systems. We don't need a high degree of usability 9 because we want low errors. As opposed to if you were using 10 a word processor, you have some errors that are a little 11 frustrating; but we expect better from our voting systems. 12 Following design guidelines don't necessarily 13 ensure usability. We can look to the usability engineering 14 field to provide measurement methods; but again, not 15 necessarily to the degree we need specifically for voting. 16 I've got a line graph on the left: Informal 17 evaluation, that depends on usability professional 18 evaluating, they have a good sense of what the design is, 19 what minimizes errors, probably not necessarily repeatable, 20 a lot of variation, depending on the expert who is doing 21 the review. 22 On the right of the timeline: Rigorous research 103 1 and experience, a very complex but reliable kind of human 2 factors researchers do. However, that's very costly, so 3 what we'd like to do is get a conformance testing up to a 4 point that we've got reproducible reliable results from our 5 testing, so the ITA's can use their tests, but that is 6 still feasible, and it's not too costly. And that's a 7 research issue. So if we want conformance standards for 8 usability, we need to address this problem if we're going 9 to develop conformance testing on certifying these systems. 10 What about current voting standards and testing? 11 Well, the current VSS -- Voting System Standards -- do have 12 some accessibility standards, but only a usability 13 appendix. They're not shallow statements, they're 14 recommendations because you're not being tested against it 15 in the ITA process, there's not much incentive to build to 16 recommendations such as those. 17 Again, if we really want to test for usability 18 and accessibility, we need standards that are clear and 19 testable, and we need good procedures to do that testing. 20 IEEE P1583 has draft standards, Task Group 3 21 usability and accessibility has been making some good 22 progress. One thing that if you're not involved in a 104 1 voluntary consensus standards in standard development 2 organizations, you should be aware that that works best 3 when you're got a lot of vendors such as Microsoft, large 4 vendors who can put a lot of capability into this volunteer 5 effort, other organizations, large organizations with a 6 good bottom line, so that they have resources to bring to 7 bear to send teams of people to develop conformance tests 8 along with the standards. 9 IEEE has gotten a marvelous set of volunteers, 10 but they don't have a lot of resources. Participants don't 11 have a lot of travel money and, Steve, correct me if I'm 12 wrong, that's just being realistic. 13 STEVE: That's absolutely true. 14 DR. LASKOWSKI: If you're going to be serious 15 about performance based standards, you have to address that 16 issue that you need resources to develop benchmarks for 17 testing conformance. 18 The Human Factors Report. Our report recommends 19 an approach that will produce measurable voting system 20 standards for usability and accessibility. It doesn't need 21 a lot of research, but it does need some. It does need 22 expertise in conformance test development, some applied 105 1 research to develop their user testing protocols I've 2 alluded to. And because of the sensitivity of voting 3 systems, we want to make sure we have good neutral third 4 parties that develop these benchmarks. 5 So there's no cheap, quick fix here. But there's 6 some things we can do in the interim to require some 7 usability testing so that we could avoid major usability 8 blunders. And that's probably a good place to start, but 9 there is no guarantee. 10 I guess I have time to quickly go over the 10 11 recommendations. 12 I've summarized, and you can read the report for 13 much more explanation and detailed sentences for each of 14 the recommendations. 15 The 10 recommendations are: 16 Performance-based usability standards. They're 17 high enough levels so that they're not what I call 18 technology agnostic. We want then independent of the 19 technology being used, because if you base your standards 20 on very exact technology, when new technology or approaches 21 come along, you've got to redo your standards. So you 22 don't want them to be very specific. But if you've got 106 1 performance-based benchmarks, they are pretty much 2 technology agnostic. 3 One thing we also noted is that we need a 4 complete set of user-related functional requirements. Over 5 voting is a functional requirement. Pulling those out 6 means usability testing gets easier, because you don't need 7 to do a lot of usability testing with users, you can 8 specify the testing to test against functional 9 requirements. One thing we've noticed is those things are 10 kind of haves been intermixed, so it would be helpful to 11 pull those out. 12 You want to avoid low-level design 13 specifications. Use only product design requirements that 14 have been validated as necessary for operation of the 15 system. 16 We do need to apply research to support the 17 development of benchmarks for usability and accessibility 18 standards. 19 We should review the current requirements form 20 the Access Board, the current VSS, the draft IEEE standards 21 for possible adoption, because there are good nuggets in 22 all of those. 107 1 Ballot design guidelines. Butterfly ballot comes 2 to mind. There's a lot of good work out there on good 3 presentation from the America Institute of Graphic Arts, 4 AIGA, and other groups, and we're pulling those together. 5 They wouldn't be standards for good conformance testing at 6 the ITA level because ballots are designed at the state and 7 precinct level. But I think those would help election 8 officials a great deal in producing better ballots and 9 avoiding bad design. 10 And, again, I think that's a matter of 11 collecting what's currently out there and pulling it 12 together in a better format. 13 Also, guidelines for facility and equipment 14 layout; also guidelines for how to design and usability 15 tests for the vendors; and usability testing, good 16 usability based documentation and training materials. 17 Users beyond the voters here, and all that 18 supports a better voting system process. 19 Vendors should incorporate a user-centered design 20 approach. There's lots of literature out there, standard, 21 user-centered approaches in systems engineering. 22 We need a good set of conformance tests for 108 1 voting products against the applicable accessibility 2 requirements. 3 And, finally, we need valid, reliable, 4 repeatable, reproducible processes for usability 5 conformance testing of voting products against the 6 usability standards described in the first recommendation 7 with agreed upon usability pass/fail requirements. That 8 is: It passes benchmark, yes or no. 9 I would say the most critical need is a set of 10 usability standards that are performance-based with 11 objective measures and conformance test procedures. Then 12 we can certify against usability, and this is the only way 13 to guarantee high levels of usability. 14 And just to summarize a very shortened version of 15 a roadmap here. In the short term, we can do a lot to 16 encourage usability and user-centered design. 17 Some usability testing at various levels. At the 18 vendor level, at the state level with actual ballots; for 19 example, to mitigate possible problems that may occur when 20 you look at real ballots. 21 There's a lot that can be done just educationally 22 to bootstrap people to this way of thinking. 109 1 Long term: We should start with using the best 2 of IEEE and other standards. Get some good ballot design 3 guidance. The parallel, to start developing good user test 4 procedures, collect user data to define performance 5 baselines, and develop performance standards and 6 conformance tests, that would bring us a lot further than 7 we currently are. 8 I guess we have a minute or two for 9 questions. 10 CHAIRMAN BEMENT: Yes, thank you, Sharon, 11 very much. 12 Are there any questions for Sharon? 13 MR. ELEKES: Yes, Mr. Elekes, Access Board. 14 As you were going through your list, you said accessibility 15 and facility, I didn't catch the point. 16 DR. LASKOWSKI: Yes, we do need some 17 accessibility, both design standards and -- the performance 18 standards are with respect to usability. With respect to 19 facilities, I think we need a -- not these don't go through 20 the ITA's because facility set-up is done at the polling 21 place. 22 We want to make sure we have good facility 110 1 layout, including good accessibility written in such a way 2 the documentation is plan language, and easy to read for 3 our poll workers so that we make sure that they all have a 4 good understanding of how to set things up properly. 5 MR. ELEKES: On that point, it may just be a 6 replication of work, because about a year and a half ago, 7 the U.S. Department of Justice came out with a check sheet 8 and a diagram for barrier-free access for polling places. 9 Now, with the election equipment that might be 10 built and specified, would those parameters have to be 11 modified, or is that a consideration for the human factor 12 to have the equipment fit into the recommendations as 13 published by the Justice Department. 14 DR. LASKOWSKI: Well, there's accessibility 15 recommendations at the polling place of the set up of 16 equipment. There are other considerations; for example, if 17 there's audio use, are they far enough apart for privacy, 18 also, on screen glare, it's not accessibility in terms of 19 the disabled user, but just in general, is your lighting 20 set up properly, and your polling booth set up so the 21 screen is easily readable. So it does depend on the 22 equipment itself. And so I think there's a set of 111 1 guidelines for accessibility and a design kind of 2 guidelines that would be specific to the technology based 3 upon the recommendation for equipment. 4 MR. HARDING: Yes, J. R. Harding. Based on 5 the earlier comments by our chair, would there have been 6 room within this discussion for (inaudible) testing 7 authority. Does that contribute to that process of 8 understanding the accessibility or the requirements. 9 DR. LASKOWSKI: I guess I'm not sure I 10 understand your question. The ITA's currently don't test 11 for usability. I think they're waiting for guidance on 12 what conformance tests ought to be. I believe on Task 13 Group 3, there is at least one person from one of the test 14 laboratories. Isn't that right? 15 MS. QUESENBERY: Not active, they're not 16 active. There's someone from one of the test labs who's on 17 the list and has occasionally chimed in, but she hasn't 18 been reactive in writing. 19 DR. LASKOWSKI: We don't see any immediate 20 contributions from those parties at this point. But we 21 certainly would want to be inclusive here. 22 MS. QUESENBERY: If I could chime in, this 112 1 is Whitney Quesenbery. One of the things we've been doing 2 is looking at other standards. There's an ISO standard 3 that's being developed for usability testing of walk up and 4 touch kiosks, and that seems like that work would be 5 relevant. They're also trying to define a test procedure. 6 We're trying to leverage some of that work as well. 7 So there are some standards being developed 8 in other fields that are applicable, although, not directly 9 applicable to voting. 10 MS. TURNER BUIE: Sharon Turner-Buie. It 11 appears that all of the guidelines and standards that are 12 set up will be developed using an ideal situation as a 13 benchmark. And you look at usability/accessibility. We 14 will adopt the standards based on an ideal set of poll 15 situations. 16 Will these guidelines also contemplate or 17 provide standards for those facilities than are less than 18 ideal, that will be encountered by the elections 19 administrators? 20 DR. LASKOWSKI: I think there's two levels 21 of test. One is the (inaudible) test, which, of course is 22 in a vacuum in some sense and it's independent of the 113 1 different kinds of environments that are depending on the 2 state and precincts. 3 If you look at some of the other details or 4 recommendations that are on the map, one of the 5 recommendations is also for a for a usability testing 6 that's done by election officials for their actual ballots, 7 knowing what their situations are like in the polling place 8 to get some idea of potential problems. And I think that's 9 about -- so not necessarily guidelines, but certainly 10 guidelines on how to do some usability testing and setup to 11 avoid problems due to the environment of the polling place 12 itself. 13 Now, of course, if you violate some 14 temperature range on the equipment, you know, then all bets 15 are off. 16 MR. WILLIAMS: I'd like to clarify 17 something. This is Britt Williams. When you talk about 18 the ITA's and the FAC testing and all, we tend to talk at 19 the federal level. The FAC standard actually defines three 20 levels of testing, not only the ITA level, but it defines a 21 level of testing that should be done at the state level, 22 and a level of testing that should be done at the local 114 1 jurisdiction level. 2 DR. LASKOSWKI: Yes, there's also the 3 certification acceptance testing -- 4 DR. WILLIAMS: Yeah, that's what I'm talking 5 about certification at the state level. 6 DR. LASKOWSKI: Right. When we're talking 7 about the standards like the VSS, I've concentrated mainly 8 to report on the ITA testing that's part of qualification; 9 but there are some recommendations before that do say you 10 need to do some of this testing locally as well with 11 respect to usability. 12 DR. WILLIAMS: And I would expect whatever 13 standards we develop to make take same approach. 14 MR. CRAFT: Paul Craft. We have dabbled 15 recently in usability and accessibility standards for 16 beginning with the publications of 2001 standard. We 17 brought in both a standard set of ballot layout 18 requirements, and we also have been experimenting actually 19 using what's a very preliminary rough of information 20 available standards for audio ballots for accessibility and 21 developed a procedure on that. 22 The biggest problem from our perspective is 115 1 developing metrics. We approached it taking those things 2 that we clearly knew were issues and those things that we 3 learned about of the test methodology. It's quite doable. 4 DR. LASKOWSKI: Oh, it is. I think it is, 5 and I think that's something for the subcommittee on the 6 Usability, Human Factors and Privacy to consider in making 7 progress on those. 8 MR. HARDING: Just for those of us who 9 aren't quite as familiar with the whole voting application, 10 could we get one more review of the way you had the three 11 level of tiers, I guess, ITA's and the national votes, 12 because I believe the disabled community would really like 13 to know where that certification or standard process -- 14 where folks can really contribute and participate in their 15 communities at the local, the state versus federal, and how 16 might we then ultimately agree to that standard. 17 CHAIRMAN BEMENT: Care to address that, 18 Britt? 19 DR. WILLIAMS: Sure, be happy to. This is 20 Britt Williams. The testing that's done at the national 21 level is the testing of the features that are going to 22 ultimately wind up in the system. 116 1 Now, when you get to the state level, 2 primarily what you're looking at at the state level is 3 compliance with state requirements, state laws, state 4 rules, and state regulations that are not addressed at the 5 national level. 6 When something comes to us from the ITA's, 7 we assume that it's accurate, for instance. We assume that 8 it's functional within the functional definitions and the 9 standards and so forth. And we test for usability at the 10 state level. The ITA's do not do any usability testing 11 that regards election officials, how difficult it is to run 12 an election with, that sort of thing. It's a checklist: 13 Does it have this feature? Yes. Does it have this 14 feature? Yes. 15 It does not look at how gracefully that 16 feature is implemented. We do that at the state level. We 17 bring in some local election directors, have them look at 18 the system, and give us their opinions how easy it would be 19 or difficult it would be to use the system. 20 The federal level does not look at 21 affordability. They don't care what it costs. They don't 22 even know what it costs. We very much care what it costs 117 1 at the state level. Then when you get down to the local 2 level, in most states, the local level is the procuring 3 level. They're the ones that wrote the procurement policy. 4 So for acceptance testing, there generally 5 are two dimensions. One is you're verifying that what got 6 delivered at that local level is, in fact, the same thing 7 that was qualified at the federal level and certified at 8 the state level. And then the second thing you're 9 verifying is that it complies with the procurement 10 documentation. So that's generally a description of the 11 three levels. 12 MR. CRAFT: Paul Craft. Another perspective 13 on that is the Florida approach where we really -- we do 14 not recognize the federal qualification process at all. We 15 have our own state level qualification/certification 16 program. 17 We do use some of the work product out of 18 the federal labs. But our view of it is the certification 19 testing evaluates the system to make sure that it meets the 20 established standard. And once again in our case, as with 21 the federal program, our legislature has directed us to 22 leave the cost and the usability pieces in large part to 118 1 the local jurisdiction. 2 The question of how difficult is it, and 3 what kind of resources does it take to lay out a ballot and 4 stage an election, that has been directed to the cost 5 benefit analysis of the local jurisdictions. 6 Our state would level casting the votes and 7 assuring the system is properly operated according to 8 documentation can be reliably used and the performance 9 expected. And then we look to the county government to do 10 their acceptance testing and to, in their procurement 11 process, specify what their standards for the system is, 12 and then do acceptance testing to verify those standards 13 are being met. 14 Then, of course, the third level of testing 15 is the logic and accuracy testing which relies on both the 16 two higher levels of testing, actually verifies that the 17 programming of the systems in this user's hands has been 18 properly done. And then you follow back on that with an 19 audit process where we can verify the system against our 20 benchmarks of what we have tested for. 21 CHAIRMAN BEMENT: Do you want to add 22 anything to that, Sharon? 119 1 DR. LASKOWSKI: I just thank you very much 2 for giving me this opportunity to talk about the NIST 3 report. 4 CHAIRMAN BEMENT: Okay. The next person, Ed 5 Roback, is probably one of the most overworked division 6 directors at NIST. 7 Ed is responsible for the security division, 8 which writes the federal standards for information 9 processing for the entire federal government, and a lot 10 more, smart cards, and whatever. He'll provide his 11 background. 12 MR. ROBACK: Mr. Chairman, members of the 13 Committee, it's been an honor to be here this morning and 14 be with you. 15 SPEAKER: Can you speak a little louder? 16 MR. ROBACK: All right, a little louder. 17 What I thought I'd do is give you a little sense of what we 18 do out at NIST in the area of computer securities, since we 19 will obviously be helping to support the activities of the 20 TGDC. 21 First, I'd mention that NIST does, under the 22 Federal Information Security Management Act of 2002, set 120 1 the computer standards for the entire federal government. 2 We're a non-regulatory agency but we use inspectors general 3 and chief information officers and so forth to implement 4 security across the entire federal landscape, except for a 5 very small set of systems that are classified for the 6 national security system. 7 So when we do standards and guidelines, we 8 range from technical, things like encryption algorithms all 9 the way up to sort of management standards and guidelines, 10 the operating systems dealing with questions of risk, 11 capital planning, procurements, and so forth. 12 One of the things we're doing right now is 13 writing a standard for minimum security requirements for 14 all federal systems. In the past, we've done standards 15 here and there, but Congress has now tasked us to do a 16 standard that addresses the minimum security requirements 17 for all federal systems at three levels of sensitivity. So 18 we're very busy working on that, and we have a deadline of 19 December of 2005 to complete that. 20 I think it would be a fair bit of work doing 21 something like that that could translate to work for voting 22 systems, particularly, when you look at voting systems not 121 1 just on the tabletop, but an aggregation of systems in 2 terms of networking and tabulation and so forth. 3 Secondly, we also have authority under the 4 Cyber Security Act where we have authority to run grant 5 programs that provide support for outside researchers and 6 so forth. So in the longer term, if there's a need for 7 that sort of thing, we do have statutory responsibility to 8 do that. 9 We have an interesting assignment that may 10 or may not play in your voting equipment, depending upon 11 the way that the equipment is set up in terms of commercial 12 off-the-shelf equipment. NIST has the responsibility to 13 come up with security configuration advice for IT products 14 in the marketplace. So when a company or a particular 15 Federal agency goes out and procures a certain device or a 16 certain piece of software, NIST is developing advice as to 17 how to actually configure that particular product for 18 securities, again, based on one of these three levels in 19 terms of sensitivity. 20 We will be working very closely with vendors 21 because, obviously, there's a huge number of products, out 22 there are unlimited programs. But that's something that's 122 1 also in our bailiwick in terms of use of commercial 2 equipment. 3 I have given you a few slides just to remind 4 you about some of this. And this is just to remind you of 5 some of the security issues in the area of voting systems. 6 I preface this with just a general comment 7 about security. Security is typically filed (inaudible) as 8 a change. And it's far easier to have something that's 9 insecure than something that is secured. In fact, most 10 security people would never go out and say something like 11 this system is secure. I don't think any of us would ever 12 want to declare that. 13 But it actually doesn't take very much to 14 find a single little flaw, and that becomes the headline 15 that the system is broken, so you break one little link in 16 the chain. Again, this is just to remind us of the breadth 17 and scope of security issues. Obviously, it's going to be 18 up to the TGDC to consider questions of: Well, how much 19 security is enough? This again gives you sort of a 20 sampling of the kind of security issues that we heard at 21 the NIST workshop and, obviously, a concern of many others. 22 There are accidental errors and omissions 123 1 that are typical security problems, and the quality of data 2 kinds of things, voter manipulation. If you've ever been 3 online buying something, you hit that enter key, it's 4 purchased. I want to buy it, and you're sort of thinking: 5 Did I hit that key once or twice? Did I buy one or two of 6 them. Well, obviously, you will see that on the credit 7 card; but if you hit that twice on the voting machine, you 8 voted in general, but the nation hopes you're not voting 9 twice. 10 Vote manipulation, obviously, in the 11 tallying process, adding/deleting votes, is the tabulation 12 being done correctly? There are risks to the modification, 13 whether inadvertent or deliberate, to hardware and 14 software. So you're thinking now things down at the chip 15 level, does the chip have strong integrity in terms of is 16 it doing what's expected. Some years ago there was a chip, 17 a mathematical processor, and was doing very complex 18 mathematics in terms of (inaudible) for operations, but it 19 was not actually doing them accurately. 20 There are some errors like that in basic IT 21 infrastructure components. These are a range of issues you 22 may have think about, issues of integrity of software. So 124 1 that the software that's loaded on a system and actually 2 being used, is that the same software that's gone from 3 testing; is it the same software that's being ordered from 4 the vendors and so forth. 5 Audit trails of whatever sort, do you have 6 integrity of the audit trail. Modification/prevention of 7 vote recording, is there privacy. Adding vote data, adding 8 duplicate votes, all of these sorts of risks. 9 Modifications in tallying process, in transit process, not 10 just when they're actually being tallied, but when they're 11 being transmitted. 12 Preventing access to individual votes, if 13 that's the policy of vote tallies and so forth. Denial of 14 service. If you found a way to bring down the voting 15 systems, you obviously can't get the citizen's votes in. 16 It's a range of these things. And, 17 typically, we talk about having something called the CIA 18 model. The CIA model basically refers to the 19 confidentially, integrity, and availability, for those of 20 you not in the field. 21 So we think about the things that need to be 22 protected from disclosure, which is confidentiality, and 125 1 the things that need integrity, which includes hardware and 2 software, protection from inadvertent or deliberate 3 modification; and the availability of systems, processes, 4 and so forth. 5 Thinking about some of those risks, I have 6 given you a table here just to give you a sense that NIST 7 has already done some work, although, not specific to 8 voting systems, but to address some of these kinds of 9 risks. And that also gives you some of the security 10 controls that are intended to support. 11 So things like access control. What is 12 access control? It has to do with who is allowed to access 13 what information and what system, for example. We have 14 guidelines that address aspects of that. 15 Assurance, when I use the word "assurance," 16 it is strongly tied to security functionality. What it has 17 to do with is a term that us security people use. 18 Assurance has to do with what degree of 19 confidence you have that the system does what it is 20 intended to do, what the specification says, it is doing 21 it. 22 Security is a little different than some 126 1 other kinds of functionality. A typical example I give is 2 hook up a printer to your PC. When you hook up a printer, 3 you turn it on, you send something to print, you can tell 4 whether it printed or not, you look at it. When you hook 5 up some security software, you turn it on, and it sits 6 there and hums, and you sort of don't know if it's really 7 filtering all the traffic that I don't want to come to my 8 system. Is it keeping out intruders. Maybe someone came 9 in and copied some data and got out and you'd never know. 10 So security is not always evident or even 11 obvious, whether or not it's functioning correctly. So 12 assurance has to do with an aspect of testing, it is the 13 degree to which you want confidence that the security is 14 operating as intended. And assurance, like security, you 15 can have as much as you want, depending on how much you're 16 able to pay. And there was a comment earlier about who is 17 responsible for thinking about how much, and who actually 18 cares how much it costs to do these things. 19 I've also gone through some examples here of 20 integrity, auditing, confidentiality and some of the 21 others. And you'll see here the reference to our various 22 security standards and guidelines. These are all available 127 1 on our web pages. These are hundred and hundreds of pages 2 of various guidelines that are out there to help people. 3 Just to give you a thumbnail of our actual 4 program at NIST. We have about 45 folks in the area of 5 computer security in our division. We have competence and 6 strong activities in the area of cryptographic standards 7 and guidelines. This has to do with things like encryption 8 and digital signatures. We also do a lot of work in the 9 ares of electronic authentication, and a number of key 10 government initiatives across the Federal Government space. 11 We do a fair bit of -- NIST is a research 12 institution, and as you expect, a fair bit of research in 13 the area of computer security, and we raise the 14 specification element of testing. 15 Some of the things we're doing now are in 16 the area of smart cards. We will all be carrying around 17 smart cards one of these things days that have things like 18 cryptographic keys on them. We're working in areas that 19 have to do with wireless security. We think about the 20 security problems that might arise with the risks, but if 21 you start thinking about voting systems and polling places 22 and the voting has some sort of wireless net, you may be 128 1 exposed to typical kinds of risks. 2 I talked about checklists and benchmarks. 3 On the management side, we also address the 4 management issues, policy development, policy guidance, 5 personnel and training, education and awayness, some of the 6 management sides of security as well because, obviously, 7 systems cannot operate alone. 8 We also have two programs that may well be 9 very useful in conjunction with the National Voluntary 10 Laboratory Accreditation Program that you're going to be 11 hearing about; and encourage that we do testing in 12 conjunction with the government of Canada on cryptographic 13 module. So when the Federal agency needs cryptography, 14 they have to use a module that's gone through testing to 15 give a higher degree of degree that the algorithm 16 specifications and security of cryptography are at an 17 adequate level for Federal use. 18 If you're talking about voting systems with 19 cryptography that may provide functions you need, security 20 functions you need, this might be something you want to 21 consider. 22 We also have a guideline and a process in 129 1 place in the Federal Government for something called 2 "certification and accreditation." That is, this is a 3 different use of the word "certification" than used 4 earlier. I just want to warn you about that. 5 In the Federal space, before a system is 6 turned on, a management official is supposed to sign their 7 name on the line saying, "I hereby accept the risk of 8 operating the system, a technical review has been 9 accomplished." 10 A technical review in this context is called 11 a "certification." And the authorization by the management 12 officials is called an "accreditation." That might be a 13 concept that you all in security requirements -- that might 14 be a particular you may want to think about whether or not 15 there's any analog to do. 16 And, finally, we have a program that we run 17 with the National Security Agent called the National 18 Information Assurance Partnership. Earlier, I talked about 19 the program and test cryptographic modules. This program 20 provides for the evaluation of IT products. And what it 21 does is allow a vendor to bring in their product to one of 22 our laboratories, bring in a set of specifications -- the 130 1 vendor says, I claim my product does these security 2 features -- take them to a laboratory, bring their product, 3 they bring their specification and they bring in a degree 4 to which they want to test it, an assurance level, and then 5 they can go through testing. 6 Now, this testing is a little different from 7 the testing Sharon talked about, because it's not at the 8 bit and bytes level. I think sometimes we talk about 9 conformance testing, this is a little higher, but 10 evaluation-type testing. With everything else, it provides 11 some useful interesting research. 12 There are other NIST security-related 13 competencies and activities. I won't go through a lot of 14 these, but work at NIST is security protocols, network 15 security, forensics, and biometrics, and you'll be hearing 16 about some of those as well. 17 The last page is my contact information. 18 That's just a quick thumbnail to give you a sense of what 19 we do at NIST in the area of security. 20 I'll be happy to answer any questions. 21 CHAIRMAN BEMENT: J. R. 22 MR. HARDING: Thank you, Mr. Chairman. I 131 1 heard an awful lot of standards in terms of Congressional, 2 but where do we draw the lines for statistical analysis on 3 the confidence level or the significant level, you know, 4 can you conclude something. Do we have those baselines in 5 your shop now, or we will we be creating that level of 6 confidence at 95, 99, 80? 7 MR. ROBACK: I wish it were as quantified as 8 you are suggesting. It is not. What we have done in the 9 Federal space is said, here are definitions of there levels 10 of sensitivity based on the risks. And then we're going to 11 say that for each of those three levels, here's a minimum 12 set of controls. It's up to you, ultimately, as the owner 13 of the system or the operator to have to make a decision 14 whether or not you're willing to accept the risk, because 15 nobody's saying that that set of controls is going to beat 16 every considerable risk. 17 So in the end, people always talk in risk 18 management, we're always talking security resource 19 tradeoffs. That's just the nature of security. 20 CHAIRMAN BEMENT: If I may comment on that 21 just for a moment. In dealing with risk management, and 22 this is where the people, procedures, and technology comes 132 1 in. It's a question of where you draw the line between 2 risk avoidance, risk mitigation, and risk acceptance. 3 In some cases, if you have the right 4 procedures, and the right people training, you can do a 5 fair amount of mitigation. It's pretty hard in real life 6 to do total risk avoidance, certainly by technology alone. 7 In many cases you can manage risk as well. 8 MR. ROBACK: Just picking up on that, you 9 can think in terms of do you set security requirements at a 10 very high level, which are usually called control methods; 11 that is, the system should be capable of keeping out 12 intruders, something really really, high like that. But 13 that would mean the ultimate in terms of data as a 14 technical measure or a managerial security control or some 15 combination thereof to achieve that. And you can delve all 16 the way down to saying the requirements are at the bits and 17 bytes level, if you're going to do encryption, you must use 18 this. 19 MR. CRAFT: Paul Craft. As one who's been 20 battered fairly frequently in the cross over whether or not 21 we're doing effective standards for security, this is 22 something somehow I'd very much like to see NIST bring some 133 1 sanity into this process. There are a tremendous number of 2 people out there who are yelling, advocating security, 3 whose view of security is to put a very high level of 4 encryption on everything. 5 In one of the recent reports, I think the 6 Ohio report, they demanded the election results be 7 encrypted for telecommunication from the polling place to 8 the central count. 9 Well, at point where you're 10 telecommunicating those results, those are now public 11 documents. If they've already been published, there is 12 really no reason to keep the contents of those 13 transmissions secret. There is, however, a very high level 14 of interest in digitally styling those so that you can 15 authenticate it to validate. My concern watching it, as 16 I've seen the vendor community try to adapt to these 17 criticism as they're putting a lot of procedures, and a lot 18 of CPU time into high levels security in areas that don't 19 really need that kind of security. So, I'm really looking 20 forward to you all helping bring some sanity. 21 MS. DAVIDSON: Donnetta Davidson. I think 22 to tag on to what Paul was talking about. My question is: 134 1 Is there anything being done or will they be doing anything 2 to study what has already been accomplished like the sanity 3 issues? 4 A lot of these, I mean, people really don't 5 understand the auditing and the process that's been taking 6 place, and the security that has been put into, you know, 7 the efforts of making sure that the elections are running 8 properly to know how we can improve it. 9 I mean, I hate to see us spend a lot of time 10 in writing something and we're already doing it. So is 11 there going to be somebody that's really going to go out 12 right now with the security and with the efforts of 13 auditing and testing before, and testing after, and how to 14 secure all the information. I think that we almost need to 15 know where the vendor community is at right now to know how 16 we can improve. 17 CHAIRMAN BEMENT: I think I can refer back 18 to the charge that Chairman Soaries this morning pointed 19 out that we don't want to reinvent the wheel. We want to 20 start with what is already in place and improve it, if we 21 can. 22 MR. ROBACK: Obviously, I can't speak to the 135 1 activities of the subcommittee, but one would naturally 2 expect that some sort of inventory of current best 3 practices would be an important part of their kick off. 4 DR. RIVEST: Thanks for the great 5 presentation. I had a question at a very high level of 6 NIST's philosophy about standards in the area of security. 7 One thinks about the fact that security is sort of a 8 negative attribute -- in the absence of vulnerability --- I 9 may be using my terminology wrong. And also one strives to 10 be technology independent, so trying to rise above that. 11 MR. ROBACK: There were two or three very 12 interwoven questions there and issues. I'll take those 13 apart. They're very inciteful questions. The first is NIST 14 is generally charged with technology neutrality in security 15 standards development. However, in some things like 16 cryptographic standards, in order to have interoperability, 17 you simply need to be using the same technique. 18 So how has NIST dealt with that? Well, NIST 19 has improved in some areas neutrality standards, a number 20 of different techniques, and then it's simply up to the 21 users. So that sort of tries to straddle the fence. 22 In the area of security, when we put 136 1 together these minimum requirements for all federal 2 systems, that will be our first stab really at saying, this 3 is what we see as a reasonable comprehensive set of 4 controls. 5 In the past we've done important work, but it's 6 been if you do this technique, this is how it should be 7 done, or this is a way to approach it. As opposed to say, 8 that's a technique that must be used, and that if you use 9 it, you should use it with these other things. So looking 10 at what constitutes a comprehensive set for the purpose of 11 voting systems, given the risks that you all see, and the 12 risks you're willing to accept is the challenge, of course. 13 I think that answers your question. 14 CHAIRMAN BEMENT: Anything else? Thank you 15 very much. 16 MR. ROBACK: Thank you. 17 CHAIRMAN BEMENT: The next speaker is Mark 18 Skall, who is serving as Acting Director of the NIST 19 Information Technology Laboratory. 20 MR. SKALL: My name is Mark Skall. I'm the 21 Acting Director of NIST's ITL. Up until a week ago, I was 22 actually running the software testing division. So we have 137 1 a lot of experience in testing against specifications. 2 What I want to do this afternoon is give you 3 the NIST perspective on how to write quality 4 specifications. Many of you, I'm sure, are well-versed in 5 this area. 6 Just a little background on NIST's role in 7 the information technology industry. We work industry and 8 Federal agencies to develop standards and tests to improve 9 the quality of software and achieve interoperable 10 solutions. 11 We have many years of experience working 12 with formal standards, organization, like W3C, OASIS, ISO, 13 IEEE, and HL7. Our experience dates back at least 30 years 14 or to the early 70's, and we started out working with 15 committees such as cobalt standards committees. We spent a 16 lot of time working on Internet standards such XML. 17 What we do with the committees is helped 18 them develop good testing specifications. We developed 19 conformance test suites, tools, and reference 20 implementations. And we have in the past done a lot of 21 work in developing validation and certification testing 22 programs, but now we, except for the security area, 138 1 typically we help other people do certifications by helping 2 them set up procedures. 3 The next slide is one of my favorite slides. 4 It shows the relationship among specifications, 5 implementation, and conformance testing. The goal is to 6 develop trust and confidence in your software. This is not 7 an isolated process. There are may other factors involved. 8 Clearly we need good specifications with requirements, 9 which feed the input to the software. 10 We need conformance tests which check to 11 makes sure that the implementation, in fact, conforms to 12 the requirements. 13 If we look at these three parts, merely as 14 the three legs of a three-legged stool, they are absolutely 15 necessary if one of the legs fall out, the stool keels 16 over. 17 With that having been said, the 18 specification is really the most important part. 19 So good specifications are really the key. That's 20 something we preach time and time again. The goal is to 21 correct reliable interoperable software and the 22 requirements for the software are captured in the 139 1 specifications. 2 So if these requirements are wrong, think 3 about this for a second, everything that ensues from that 4 requirement, the software, the test, the certification are 5 all wrong. So you have to get those requirements right. 6 So these are some of the things we think about when writing 7 specifications. 8 Specs must use appropriate language to 9 designate requirements. So you can have the best 10 requirements in the world, if the vendor doesn't know their 11 requirements, then you've lost the battle. 12 MR. HARDING: Mr. Chairman, can I interrupt 13 for a second? From your experience in communicating to 14 vendors, are there more effective ways to ensure the 15 clarity of those things? 16 MR. SKALL: I think people typically do a 17 good job, and we'll get into that in the next slide. So 18 please ask the question again. 19 The specification must be precise, 20 unambiguous, and testable. I think you heard that from 21 Sharon. You have to tie things down the specification. 22 You can't speak in generalities. 140 1 The spec must contain a conformance clause. 2 This is something I think we haven't discussed. This is a 3 very important concept. The user must understand what's 4 expected of him or her. Those of us who get up at forums 5 like this and can speak using English, well, understanding 6 English is not the best way to be precise. It's in fact 7 more difficult to use English to write a specification. 8 In a perfect world we would use what we call 9 a formal specification mathematical notation to be more 10 precise in our requirement. There are, of course, problems 11 with these, and there are not many people well-versed in 12 writing formal specifications. They are not typically 13 readable by the public. We can use semi-formal 14 specifications, things like XML. Again, there are problems 15 with these. 16 I think for the issues of the voting 17 community, I use this world (inaudible) tend to be stuck 18 with the English language. But to ensure that these things 19 are read by the public, you probably need to stick with 20 English, and that presents quite a few challenges. Again, 21 English is not the best way to be precise. 22 So I want to go through a few of these 141 1 points one by one. Specs must use appropriate language. 2 There are key words that are internationally recognized. 3 "Must" and "shall" are the two words that are typically 4 used to indicate mandatory requirements. They're both 5 equally acceptable. Some communities use must and you 6 shall. I think those are good languages to say that this 7 is a mandatory requirement. 8 There are other words that are used to 9 recommend, "you should." "May" is a weaker word that says 10 something is permissible to do. These are all words that 11 are often used and are useful. 12 In our experience, we try to put as few of 13 those as possible. When you say they have no bearing on 14 conformance (inaudible) states you can certainly write 15 tests for things that say should, and then you can 16 determine if, in fact, one has this feature, which is an 17 option, and can make sure that this feature is implemented 18 correctly. 19 Wording like, "good enough," "do the best 20 you can," are really not appropriate for specifications, 21 but you do see them in specifications. 22 Specifications must be precise. What I mean 142 1 by precise is one that can be able to determine if that 2 requirement has been fulfilled. So if you're running a 3 program in the shop, you may have a requirement system 4 "shall guard against viruses, Trojan horses, and worms." 5 So you look at us and say, well, that's good key words. 6 But to say this in a more precise way, you may say, "an 7 anti-virus program shall be installed on ones system, and 8 anti-virus software checks and live updates shall be run 9 daily." This is a requirement that you can determine 10 whether it's been met. 11 Specifications must be unambiguous. I think 12 we've heard this repeated. This is really just that we've 13 worked with many many standards' communities over 30 years, 14 and with a lot of really smart people. Yet you come up 15 with ambiguous and contradictory statements. It's a 16 difficult process, and many of the specifications are very 17 very large. You just have to be really careful to write 18 things that are not ambiguous. 19 This is an example: "The girl touched the 20 cat with a feather." They might have been thinking in his 21 own mind that the girl was touching the cat, or they might 22 have been thinking, "the girl touching the cat, which has 143 1 the feather." It just really shows how easy it is to write 2 things that in ones mind, it might seem clear at least two 3 ways to say it. 4 Specifications must be testable. Is it 5 better to have a requirement that's not testable, if we 6 can't figure out how to test it, or is it best just to 7 eliminate it? I guess one of the rules of requirements is, 8 if it can't be tested, can't verify that it has been 9 satisfied, it's not of any use. You need to reformulate 10 that requirement so it can be tested, or remove it from the 11 standards. 12 Testable assertions will be derived from 13 normative mandatory requirements in the specification. And 14 again, unspecified, ambiguous, or imprecise requirements 15 cannot be tested. 16 MR. HARDING: Mr. Chairman, along that line 17 about being able to test the requirement, is it our charge 18 to define that test? 19 CHAIRMAN BEMENT: Yes. 20 MR. HARDING: Thank you. 21 MS. QUESENBERY: I guess in my experience, I 22 also have life as writer, although, I wouldn't claim to be 144 1 a major plain language expert. I have an aspiration to 2 become one. But I wonder whether we ought to be including 3 some requirement that the language of our applications be 4 reviewed by someone who has expertise in writing of plain 5 language. 6 As ideal as a mathematical language might 7 be, it's certainly not an appropriate language for a 8 usability and accessibility section. And given the public 9 prominence of this, it's going to be very important that 10 people can read and understand what we mean very clearly. 11 CHAIRMAN BEMENT: Very good point. 12 DR. RIVEST: It seems like the appropriate 13 time. I'd just like to register a -- 14 MR. BURKHARDT: Excuse me. Actually, I just 15 need to intervene. Since your financial disclosure 16 statements haven't been approved, the only thing you can do 17 at this meeting is ask questions. You can't express an 18 opinion. That's just simply for your own personal 19 liability reasons. Sorry to intervene. 20 MR. RIVEST: Okay. 21 MR. HARDING: Rephrase the question. 22 CHAIRMAN BEMENT: Rephrase the question. 145 1 DR. RIVEST: Do you feel that the 2 testability is appropriate for security requirements where 3 really the goal such as voter privacy may not be captured 4 by the specific checklist of items to be tested against, 5 but rather the ability to withstand (inaudible) you know, 6 may not be sufficiently captured by that checklist of test 7 items? 8 MR. SKALL: I think that's a fair statement. 9 My perspective in regard to the issue has been in 10 nonsecurity areas, so we're talking about functional 11 requirements typically. I think security is a whole 12 different game. I'm talking about functional requirements 13 with a specific shall or must. There are many gray areas 14 involved in security. 15 If I could just add to that. In the area 16 of security, another example would be if you think about 17 some of the software programs, millions of millions of 18 them. One of the security requirements probably would be 19 along the lines of "there shall be no malicious codes 20 barrier." But we don't have right now, any good way to 21 test that. Nonetheless, it's probably a good security 22 requirement that somebody not have inserted a trapdoor or 146 1 Trojan in there. 2 So that would be one example where 3 unfortunately, the state of the art in terms of testing is 4 simply not where it should be. 5 MR. BERGER: Steve Berger. Actually, I'd 6 like to comment on the difficult issues of repeatability 7 and uncertainty, should be in your thoughts, especially in 8 some of these areas whether we should be addressing the 9 likelihood of a test being repeatable and also the 10 uncertainty of having to repeat it. 11 MR. SKALL: Absolutely. I try limiting my 12 presentation to susceptibility. There are a whole bunch of 13 issues in testing and certainly repeatability is a key 14 requirement that should be there. Uncertainty is a little 15 bit more difficult. We've done some work in trying to 16 measure statistical uncertainty based on what would be the 17 probability that, in fact, implementation does, in fact, 18 conform, assuming it passes a set of conformance tests. 19 And that's a very difficult problem. 20 And, of course, the more comprehensive the 21 tests are, the more probable it is that implementation does 22 in fact conform. It certainly is an issue that needs to be 147 1 addressed in testing, and certainly is very important. 2 MS. QUESENBERY: I have a question that I 3 struggled with in writing other standards, which is, how do 4 you create -- this is probably not a short answer, so I'm 5 just really putting it on the table. But how do you handle 6 creating a standard where it's going to take a number of 7 requirements together to create the end result you wish, 8 especially if they cross sections? 9 I'm thinking of accessibility, privacy, and 10 usability, which all need to be considered as a whole in 11 order. So you could meet individual requirements and still 12 not meet the whole. 13 How do you make sure that you've written a 14 standard that does, in fact, achieve the top level goal? 15 MR. SKALL: I did the government system 16 requirements, and that's something we discussed. It's a 17 two-tiered process. You first have to define individual 18 requirements, make sure those are met. And then you have 19 to look and see what system requirements are requirements 20 that flow through the intersection on top of all of those. 21 You have to think about those things. And, 22 again, all you can do is ask to meet everything you have in 148 1 your specifications requirements. So if you need to do 2 other things that the systems will ensure that everything 3 is done, and you need to think about how to express that. 4 MS. QUESENBERY: It would be great to have 5 some examples of that. 6 CHAIRMAN BEMENT: If I may for a moment. 7 I'm a little nervous about the way I answered your 8 question, J. R., with regard to testing. They're three 9 levels of consideration. First of all, what needs to be 10 tested; secondly, in what way should it be tested; and 11 third, what is test to be used? 12 MR. HARDING: Right. 13 MR. CRAFT: I recently had the pleasure of 14 defending some of our subjective standards and some 15 findings addressed in our subjective standards in Federal 16 Court, and fortunately, we prevailed. 17 What we had attempted to do -- and I think 18 this is going to become difficult for this Committee, 19 particularly in the areas of accessibility and usability -- 20 there will come points where you very clearly know what you 21 want to enforce, but there's not going to be a good metric 22 to measure. 149 1 And what we had attempted to do, and what we 2 prevailed with legally was basically establishing a 3 reasonable manned position on compliance with that 4 particular standard. 5 How comfortable is NIST going to be with 6 that type of imprecise standards when you get to the point 7 where there really is no other way to address the issue? 8 MR. SKALL: I think we're here just to give 9 input to the TGDC, and to tell you what our experiences 10 are, and what we see. I think that there are going to be 11 individual decisions that are going to have to be made, 12 based on subject matter. I think we have to make those 13 decisions one by one. In general, I think we all agree 14 that the more precise you can be, the better it is. If you 15 can't be precise, I think that's a case by case decision 16 that has to be made about how to best do it in such a way 17 to express the intent of what you want to try to ensure 18 that it is met. 19 CHAIRMAN BEMENT: If I might interject. 20 What has to distinguish between precision and specificity 21 are basically two entirely different things. 22 MR. BERGER: Can I just offer a parallel 150 1 from the FCC where they're trying to deal with 2 interference. They've got a clause in all their 3 requirements that if they find a product that causes 4 interference, they will shut it down, no matter what the 5 specifications there are. 6 We probably need something like that. The 7 authority to automatically have the right to say that 8 doesn't meet the requirement. 9 MS. DAVIDSON: Are you referring to like a 10 decertification? 11 MR. BERGER: Yeah, basically it's a safety 12 net with the Commission. They're required to assure that 13 undue interference is a possibility, and there's clearly a 14 possibility that despite the best efforts of all these 15 tests, with technical specifications, something slips 16 through. 17 And I think we have some similar things, 18 some high-level requirements. Potentially, no matter how 19 well we do our job, something slips through, a major 20 security flaw, or something else and the examiner sees it. 21 And I think in our specifications, there ought to be 22 something that says, where that's clearly the case, that 151 1 takes precedent over all the detail underneath. 2 CHAIRMAN BEMENT: You have a followup 3 question? 4 MS. DAVIDSON: I think that's very good. 5 Yes. Donnetta Davidson. Because I think that once in a 6 while, we do find a flaw even when we're testing before an 7 election and, that piece of equipment or, you know, 8 whatever the problem might be, that is not used and, 9 obviously, we do not want it used. 10 But there's also an issue that at times 11 states have decertified a certain make of equipment because 12 it's not meeting the standards that they felt it should. 13 So that was really why I was asking if you're referring. I 14 think that there needs to be any time that they find that 15 there is issues, definitely they have procedures set forth 16 that they can decertify or not use the equipment, whatever 17 words you want to use. I agree whether it's the equipment 18 or itself or whatever it might be. 19 MR. SKALL: To be rigorous, in my view, the 20 best way to do that or, in fact, something is deficient, 21 you know, and passes the test, that means your tests aren't 22 comprehensive enough. What you want to do is go back, 152 1 enhance the test to, in fact, catch that. So you want to 2 keep the test up-to-date as much as possible. 3 DR. LASKOWSKI: Since you're on usability, 4 the questions that Paul had, quickly, with regard to the 5 desertification type of thing and usability, so, for 6 example, you might as a requirement test by demonstration 7 that this system does not allow over voting, but then in 8 the course of usability testing, a voter over votes. 9 At that point you're supposed to test it, 10 even though it passed initially, at this course of use, it 11 didn't pass. And then, Paul, you referred to accessibility 12 issues. The conundrum is that you can design your tests 13 with certain good coverage of users for usability to show 14 that different classes of people with disabilities can use 15 this system effectively and efficiently. And there's also 16 going to be some unique sets of individuals that cannot use 17 the system. And then you do need some sort of reasonable 18 clause. 19 So I think that maybe an example might be -- 20 although this is a design spec, doors are designed for most 21 wheelchairs to fit through. If you have a wheelchair 22 that's a very nonstandard or very wide design, it's not 153 1 going to fit through, yet those doors according, to the 2 accessible specifications are acceptable. So I think there 3 is a special case with respect to accessibility that we 4 have to tread carefully on and think about. 5 DR. RIVEST: Ron Rivest. Another question. 6 Could you comment on the viewpoint of testing as providing 7 information above and beyond either pass or fail with 8 respect to the standard? You know, one could derive from 9 the testing procedure, you know, how long it takes the 10 average voter to vote, how wide the wheelchair access might 11 be, things like that that might be useful to the state when 12 making purchases. 13 MR. SKALL: There are all levels of testing 14 that one could do. The absolute minimum is how much the 15 conformance tests determines what the requirements are. 16 That is all you need to do to set your conformance test. 17 Clearly, it helps with as much information as possible and 18 that's a question of resources, time and tradeoffs. 19 So it's something that certainly should be 20 encouraged. I can't say it's always done, and I'll get into 21 this later. Testing is incredibly expensive, and there are 22 so many spec outs there. 154 1 I'll get into some of the tools we've 2 developed at NIST to try and help. It's an expensive 3 process and must be a part of the decision (inaudible). 4 MR. WILLIAMS: Britt Williams. This is just 5 kind of a point. We talk about pass/fail and standards, 6 but in the ITA process right now, we do not fail a piece of 7 voting system. If you submit a voting system for ITA 8 qualification, it comes out of there one or two ways. It 9 either passes or the vendor withdraws. 10 If it is found to be deficient, then you go 11 back to the vendor, find out the deficiency and given them 12 the opportunity to correct it. But you never go to the 13 vendor and say, you're out of here, you failed. 14 MR. SKALL: Thank you. 15 MR. HARDING: J. R. Harding. Thank you, 16 Mr. Chairman. I just want to rap it up a little bit in the 17 sense to say thank you, because as our colleague pointed 18 out that no voting location is truly identical. And the 19 circumstances to replicate tests will truly be diverse. 20 And that we will need to remember where the (inaudible) and 21 the states and counties embrace it. 22 MR. BERGER: Mark, I think it's roughly 155 1 accurate to say that you paid for every testing in 2002. 2 You paid a $30,000 plus or minus some thousands. And 3 there's some point at which we raise that testing process, 4 and we start doing bad things to the system. 5 And one of the things I'm quite mindful of 6 is we're dealing with a very narrow industry here, a very 7 specialize niche. It's really an ongoing question, but I'm 8 just wondering what your thoughts might be about how we're 9 going to do what we all want to do, and yet not in the sum, 10 end up breaking something worse by just the testing costs, 11 the costs of compliance? 12 MR. SKALL: The only answer I can give with 13 this, and you wrestle with this, is that I keep preaching 14 that there has to be bigger, more efficient, cheaper ways 15 to do this. (Inaudible) 16 I think we as community need to think about 17 how to develop testing in a more efficient manner, and do 18 the best we can to make testing more cost productive. And 19 part of the decision about tradeoffs, and it's still 20 expensive, it can be perhaps be a little less expensive and 21 more comprehensive. 22 We need to invest in research to do 156 1 (inaudible). In the short term, I think testing is very 2 very important. I mean, basically, we have to realize it's 3 going to cost us, and the best thing to do is we have to 4 figure out a less costly model. 5 MS. DAVIDSON: That was basically my 6 question. I was afraid we might push the States so far 7 they wouldn't adopt the standards. As I opened up with 8 saying, my goal is that every state adopted what we did, 9 and you touched right on my point. 10 CHAIRMAN BEMENT: Okay. Maybe we ought to 11 move forward. 12 MR. SKALL: Actually, I was telling my 13 colleagues I thought this would be noncontroversial. 14 Okay. I'll try to rap it up. Conformance 15 clause is a way to tell the implementers what conformance 16 should address at a minimum, and who needs to conform. Any 17 time more than one type of entity needs to conform, what 18 must they do to confirm. It may for conformance purposes, 19 refer to functional subsets, modules, profiles, levels or 20 other structures. But the conformance clause must address 21 what's expected, so if there are five profiles, one 22 profile, at least one profile out of five profiles must 157 1 conform. 2 The concept of extensions is very very 3 important. Extensions are the inclusion of additional 4 functionality above and beyond what's required. Extensions 5 sometimes are allowed, sometimes are prohibited. The 6 specifications should address the concept of extensions, 7 say whether they're allowed or what constraints are to be 8 placed. 9 A few years ago, we at NIST worked closely 10 with WC3 to develop guidelines on how specs should be 11 written, and we had a workshop at NIST. A few years ago as 12 a result of this, the WC3 started a new activity for 13 quality assurance that they're helping to lead. And we're 14 developing, among other things, guidelines that have 15 already specifications. They initially could be used with 16 anybody (inaudible) directly in the community. This is 17 perhaps one resource we can use to help the voting 18 community to draft form (inaudible) certainly we can make 19 it. 20 MR. HARDING: Mark, could you define for the 21 committee what WC3 is? 22 MR. SKALL: WC3 is worldwide web insertion. 158 1 MR. HARDING: They seem to lead in 2 technology for accessibility. 3 MR. SKALL: Just summarizing now, I just 4 wanted to mention a few efforts. I mentioned the WC3 5 quality assurance activity. We are involved in OASIS as 6 well. We share the conformance testing activity. We have 7 produced a document to help with the conformance tests. 8 We do what I call a conformance advisory. 9 We work with not only standards communities, but other 10 communities such as the healthcare community, the 11 Department of Transportation, we work with the airline 12 industry, to help (inaudible) them how to write conformance 13 tests and things like that. 14 I mentioned before, we've done a lot of work 15 in testing and developing testing technique. I mentioned 16 four or five that are concentrated on different things to 17 outcome. We have used this methodology to help generate a 18 very extensive set of XML tests that are used voluntarily 19 by just about every vendor. 20 Lastly we have a National Software Reference 21 Library developed for the forensics community? Doug White 22 will be speaking about that as soon as I shut up. 159 1 The last slide is the contact information. 2 Do you have anymore questions? 3 DR. RIVEST: Could you comment on the 4 utility of open source software, the quality of it, and 5 evaluation of reducing the costs of evaluation? 6 MR. SKALL: I think open sources has some 7 merits. Of course, it's a double-edged sword because much 8 of the open source software, you don't know how to 9 (inaudible). I think it's a tool that can be used if you 10 have to look at again making use of all your resources. 11 Open source, by it's very nature, sometimes it is not 12 tested very well, as well, you don't know if it's a tool 13 that can be used. 14 DR. RIVEST: I was asking about what was 15 testing of standards of user software which was provided in 16 open source format, not using open source tools? 17 MR. SKALL: The same issues really apply. 18 You have to put in the input that you're testing. The key 19 issue here is you have to be very careful in writing the 20 requirements and developing the test. 21 DR. RIVEST: Why does one have open source 22 -- open setback and open source software. The argument is 160 1 often made that one can benefit from the public at large 2 looking at both the code and spec to see there is no test. 3 MR. SKALL: Is that scientific test 4 methodology? 5 MR. CRAFT: Paul Craft. I so have some 6 comments, and I've tried to avoid just flat out saying the 7 way things are. Open source is a real good way to get 8 comment on your code from other knowledgeable parties. 9 That's why it is such a popular approach for developing 10 things, particularly in the academic world. 11 As far as evaluating the system, the 12 evaluators must review and analyze the source of the code, 13 and then they must witness a compilation of that source 14 code in a positive compilation environment. If you don't 15 have total control of the compilation environment, you're 16 wasting your time. If you don't review the source code 17 before you witness it, you're wasting your time. And, of 18 course, if you strictly look at object code produced, 19 you're doing black box testing, which is pretty much 20 (inaudible). 21 The real value of putting voting systems 22 source code out there for public comment by people who 161 1 really have no way of witnessing the compile or getting a 2 chain of evidence between that code and the stuff you're 3 going to be voting, it is pretty much a waste of time. 4 CHAIRMAN BEMENT: Thank you for that 5 feedback. Anymore questions? I think we can get you off 6 the stand. Thank you. 7 We've been at this for a while. We need to 8 generate our circulation systems, so we're going to take a 9 15-minute break. We will reconvene at 20 minutes of. But 10 if you have filled out your preference forms as far as 11 subcommittee assignment, if you would pass them up, we will 12 take them now. 13 We'll reconvene at 20 of. 14 (RECESS 1:20, RECONVENE 1:40) 15 CHAIRMAN BEMENT: The Committee will come to 16 order. I'd like to make a general disclaimer so that 17 everyone understands what NIST is and what our role is. 18 NIST is an institution and agency of the 19 Federal government. It is not a standards' development 20 organization. We develop guidelines, but we don't write 21 standards. We work with the standard developing community. 22 There are about 300 standard organizations that we 162 1 routinely interact with. We're not a regulatory agency, so 2 we don't write regulation. 3 And so then the question is: What do we do? 4 Well, we're the subject matter experts for those who do the 5 standards writing, and the people who own that process are 6 the standard developers. 7 As far as this committee is concerned, the 8 products of the Committee are owned by the Committee. NIST 9 is there as a service organization to provide useful input 10 along the way, and no more no less. 11 So we're a resource that you can use to your 12 best advantage. And I think you'll find that that is going 13 to be a very useful resource for you. 14 The other two points that I wanted to cover: 15 Everything that has been handed out to you will be 16 available electronically if you wish to leave the material 17 behind. It will be posted on the web site. I think that 18 also applies to what Chairman Soaries handed out this 19 morning, but I would suggest you take that with you so that 20 you can begin looking at it, because it may take a while 21 before we can get that on the website, maybe a day or two. 22 With that exception, if you wish to leave 163 1 anything behind, you may do. 2 The other thing has to do with the 3 establishment of the subcommittees. We would like to do 4 that today, if we could. We will do it as soon as the 5 final clearance is in hand, and that should take another 6 week, maybe two weeks at the long end. Those will be 7 posted on the web site, and you will be notified by 8 telephone or by mail just as soon as we can get those 9 organized. 10 We will begin working with the Chairs to 11 begin scheduling the first interactions of the 12 subcommittees. 13 With that, I would like to call on Doug 14 White. Doug is project leader for NIST's National Software 15 Reference Library, and he's going to give us an overview of 16 how that library operates. Doug? 17 MR. WHITE: Thank you very much, and thank you 18 for the opportunity. 19 As an introduction, the National Software 20 Reference Library is really three things: 21 One, it's a secured room full of software, over 22 5000 software applications at this time; 164 1 Second, it's a database that we keep of all of 2 the file fingerprints or how you call it, hashes, and 3 information that uniquely identifies every single file in 4 every disk and every box on all the software shelves; 5 And, thirdly, it's a set of CD's that we 6 distribute to law enforcement and police centers that's 7 only a subset of the information that's in our database. 8 How the NSRL is used: 9 Most commonly it's used to eliminate known files 10 from a seized computer using automated means. And there 11 are several other things you can do with the data. 12 You can discover an expected file with unexpected 13 contents. 14 You can identify origins of files. You can 15 actually trace identities of files through operating 16 systems. 17 You can look for malicious files such as hacker 18 tools. 19 You can identify duplicate files. 20 And we provide rigorously verified data for 21 forensic investigations. 22 Just briefly on how the NSRL project started: 165 1 Now law enforcement needed software hashes that 2 could stand up in court, and also could be used in 3 investigations. 4 The source needed to be unbiased, and NIST, of 5 course, is a neutral organization, and we were approached 6 four years ago at the start of the project. 7 The data produced had to be complied from 8 the highest quality, and NIST has that track record. 9 The data needed to be traceable and 10 repeatable, so we actually can go back, pull any software 11 application off of our shelves, and we can do mathematics 12 at any time it's called into question in court, and repeat 13 the process. 14 NIST provides an open rigorous process -- 15 I'll get back to that -- but all of our code naturally is 16 available, all of our process is peer-reviewed. 17 So specifically to dive into three sections, 18 the software collection is a balance of the most popular 19 software that law enforcement required and encounters the 20 most often on machines that they seize, and the balance of 21 the most desired software, that is the software that is 22 more frequently pirated, what do people go after to obtain. 166 1 We collect software currently in 32 2 languages. This software, the data set is used 3 internationally. I know of Swiss, German, Dutch, Irish, 4 Canadian law enforcement groups that actually use our data 5 set. 6 The software is purchased commercially, so 7 it's exactly what's available to consumers, what we'd find 8 on anybody's machine. And it's also donated under a 9 non-use policy. Some of the more expensive software, 10 Oracle, for example, some of the Microsoft software, we get 11 every single CD and DVD that Microsoft produces to generate 12 the fingerprints. 13 A list of our contents is openly available 14 on our web site, so anybody can check that. 15 As far as what's in the database, again, 16 it's the information that uniquely identifies every single 17 file, every CD or floppy in every box on our shelves. 18 The database schema, that is the description 19 of the items we collect, is also available on our web site. 20 Anybody can look and see all the information we collect. 21 And just as a feel or it, I mention some 22 numbers here: 4200 bytes per application, 750 bytes per 167 1 file is what we collect. So our total database size right 2 now is just over 20 GB, and we've got almost 32 million 3 files in that database. 4 The reference data set, the CD sets, is a 5 selection of that information from the database. Not 6 everybody needs all of the information that we collect, so 7 we call it down and make up a lot of duplicates. Instead 8 of the 32 million files, 20 million files are represented 9 on these, so that's the smallest set that anybody needs to 10 go through to do elimination. This still allows positive 11 identification of a manufacturer, product, the operating 12 system, and versions of the files based on that file's 13 signature. 14 The data format, the way that we generate 15 the files that are the CD's available to the public and 16 open for developers. There are several tools, forensic 17 tools that are built for third-party vendors that can 18 import this data set. 19 It is published quarterly, and it's 20 available for redistribution. We'd like to get this out 21 into the hands of as many law enforcement agencies and 22 researchers as possible. 168 1 Beyond that quarterly deadline, it is 2 possible to publish critical data out of our regular 3 schedule. We were approached back in February by the FBI 4 and DOD, who asked us if we could generate a specific batch 5 set of Arabic software, and within that month's time, we 6 were able to give them a half a million files in the Arabic 7 language. We've also done that for certain hacker group 8 sets also. 9 The next page is a diagram of the field use 10 concept. And if the computer there, you look at the files 11 on the disk drive, we feed those into an analysis program, 12 take our reference data set, also feed that into an 13 analysis program, what you come out with on the right-hand 14 side to the lower right are the known files, the files that 15 we have the signatures of, they're benign operating system 16 files, office application files. You know what those are, 17 you don't need to look at those. Anything else comes out 18 in the upper right being unknown, something to be 19 investigated. 20 We can take a hard disk, a large hard disk 21 with hundreds of thousands of files on it, eliminate up to 22 90 to 95 percent of those, and then you can concentrate 169 1 your efforts on the 5 to 10 percent that are left on the 2 disk that are of interest in the investigation. 3 Another particular example, say you are 4 looking for sensitive facility maps on a computer that has 5 Windows 2000 running on it. Just when you install the 6 Windows 2000, you get almost 6000 images placed on that 7 machine. We've got the signatures for those. We can take 8 those out of your investigation, so right off the bat, 6000 9 less images you have to look at in the investigation. 10 In particular -- now we're going to get into 11 a little bit of mathematics and technical things. 12 Hashes can be thought of like someone's 13 fingerprint. It uniquely identifies the file, based on the 14 contents of the file. But like a fingerprint, you can't 15 reproduce a person from a fingerprint, you can't reproduce 16 the file from its hash. 17 The primary value that we use is the Secure 18 Hash Algorithm (SHA-1). That's specified in the Federal 19 Information Processing Standards that's openly defined. 20 It's a very rigorous cryptographic algorithm. The phrase 21 that we use if someone knew the signature of some voting 22 software and wanted to try and recreate a hack version of 170 1 that with the same signature, it is "computationally 2 infeasible," it's impossible. It computationally 3 infeasible to find two different files that would generate 4 that same signature. There's not enough computing power in 5 the world, or time left in the universe for anybody to do 6 that kind of a trick. 7 On the next page you see some hash examples, 8 and it's just on the right-hand side you can see some long 9 character strings. That's a hash; that's a SHA-1. The 10 first four lists there are from the files called 11 notepad.exe off of one single CD, but those four files are 12 meant to be used on different ship sets of computers. They 13 have a different number of bytes, they're different sizes, 14 and they all have different signatures. 15 The bottom two that are listed, those came 16 off of two different CDs. They both have .exe and have the 17 same number of bytes, the same size, they have the same 18 signature, they're exactly the same files based on that. 19 We also have done some research with the 20 National Archives. We've been using our hashing process 21 for their non-classified Presidential materials to evaluate 22 their records. We have been able to identify the 171 1 application files on the administration's machines, so 2 there's a file when we eliminate those, what's left is the 3 content that was generated by the administration that the 4 National Archives can focus on. 5 We're also able to identify duplicate files 6 so the National Archives doesn't have to keep 600 copies of 7 the same memo. They merely can just keep that one 8 particular copy. And back from them, we got access to 9 older installed software so this helps our data set. 10 In particular, from what we see, the ways 11 that the NSRL can help the needs for voting systems would 12 be to determine that the software used during elections was 13 the expected software. 14 That the tested certified version is 15 definitively identifiable. We know that's the code that's 16 been certified. 17 That the software stays the same during the 18 distribution, installation, and setup, or use. 19 Then we can talk about possible chain of 20 custody. 21 As far as transparency, the NSRL methodology 22 is in the public domain, and the algorithms that we use are 172 1 widely known and available for inspection. And because the 2 final fingerprints don't allow anyone to know the 3 proprietary contents of the files, jurisdictions may be 4 able to share that knowledge with each other. 5 As far as what the NSRL, specifically, 6 technical for supporting EAC. We can verify that operating 7 system file contents have not been modified. If you're 8 expecting a system that comes with certain things disabled 9 in the software, we can verify that that is true. 10 We can verify that application file contents 11 have not been modified. 12 We can verify that known static sections of 13 files have not been modified. So there may be times like 14 an interface file were you're expecting things to stay 15 exactly the same, even though parts of the file have 16 changed. Or if database files are sometimes modified, we 17 can check certain sections of those files. 18 And just to give you a feel for time. For 19 an 866MHz computer, a file with 50MB takes about 5 seconds 20 to generate a SHA-1 fingerprint, about 4 seconds to 21 generate an MD5, which is the method more popular, and the 22 one that's used in the field. 173 1 As far as ongoing voting research issues, 2 with Commissioner Soaries' remarks, and the five points 3 that he put out there a month ago, we have started working 4 with software companies to get access to the software. 5 There were issues as far as the distribution 6 versus the installation hashes. What I mean by that is if 7 we're given a CD, if we're given that initial software, 8 that may not be exactly the same as what is installed as 9 far as the installation process. There could be 10 modifications, so we need to research that. For example if 11 there is any setup after the hashes are made, how do you 12 know what changes are valid. 13 Is it possible or practical to have 14 on-location, time-of-certification hashing, and also 15 research into verification within given time or space or 16 security constraints? 17 Are there any questions? 18 MR. WILLIAMS: I have a comment on this. We 19 do this in Georgia. We use this exact same hashing 20 technique to develop a software that's installed in the 21 counties. 22 The only suggestion I have, we do not obtain 174 1 our software from the vendor. We get it from the ITA. The 2 ITA qualifies a specific version. And so to be sure we've 3 got that qualified version, we get our copy of the software 4 from the ITA, not from the vendor. 5 MR. BERGER: Thank you for the presentation. 6 To me it is just a wonderful tool to bring to this arena. 7 Let me ask you, I know the Chairman is very 8 supportive of quick implementation. Is there any reason 9 not to encourage that as quickly as possible, any 10 implementation issues before this could be brought into 11 this arena? 12 MR. WHITE: I think that would depend on the 13 requirements that would be set up. I don't know if I can 14 make a call on that at this time. 15 MS. PURCELL: Thank you. Helen Purcell. A 16 question about the vendors that have filed with their 17 software with you. Have there been some users -- 18 MR. WHITE: There were two. 19 MS. PURCELL: Two. 20 MR. WHITE: One that has sent software, just 21 to explain as my understanding goes. I haven't talked with 22 some of the vendors. We need a non-use agreement, the 175 1 vendors to sign a nondisclosure agreement, whether it comes 2 from the ITA or them, and we do this with all software 3 companies so they know that we're not going to distribute 4 or make it available. 5 VoteHere, Incorporated, which has publically 6 available software has sent the software in. The other 7 companies, one has agreed at this point, and we have worked 8 out a nondisclosure agreement with them; however, we have 9 not received the signed nondisclosure agreement back from 10 the company. 11 And while the other vendors have expressed 12 some interest to the Chairman of the EAC, we have no 13 agreement from the other major vendors at this point. So 14 it's more than just going to the ITA's and getting the 15 equipment. We have to look at the software, and we can't 16 do that until we have an agreement. I saw this as a good 17 practice. 18 MR. CRAFT: We currently have -- I'm not 19 sure about Sequoia -- but with the other two vendors -- and 20 I can't remember the one you called -- we have verbal 21 agreement with all the three major vendors to take the 22 copies of the software that we have a good chain of custody 176 1 with, haven't breached the certification, and give that to 2 NIST to work with them on defining both the installed 3 pictures and the release disk. 4 The legal issues basically are holding us 5 all up right now, and we're going to have to have probably 6 some sort of a three-way agreement in order to hand that 7 over. But also I want to be sure that everyone here 8 actually understands the importance of this. This is 9 probably the most important contribution NIST is going to 10 make to implementing HAVA nationwide. 11 The work that Britt has done in Georgia, and 12 that we have done in Florida allows our two states to walk 13 into any county, and in a few minutes prove absolute 14 certainty that they either have the certified copy of the 15 software that we certify and test it, or that they don't. 16 That's a very expensive thing to do. I'm 17 running a bureau of 16 people covering 67 counties. We're 18 working 50 and 60 hours a week, everybody in the bureau. 19 There has to be an infrastructure that is 20 going to allow every county in the country to do that 21 themselves, and do it economically, so it becomes very easy 22 to do. And that's what the National Software Reference 177 1 Library offers through all of this. 2 This is a very important part. Everything 3 else that we've talked about doing here, if we don't get 4 this done, will be for nothing. 5 CHAIRMAN BEMENT: Is there a recommendation 6 to the Election Assistance Commission on this matter? 7 MR. CRAFT: I would like to recommend that 8 we very aggressively move toward these legal issues and get 9 a prototype up as soon as possible. 10 CHAIRMAN BEMENT: Can you put that in the 11 form of motion? 12 MR. CRAFT: I move that we recommend that we 13 expedite the legal problem solutions, and get this into a 14 prototype as soon as possible. 15 MR. BERGER: I second it. 16 CHAIRMAN BEMENT: Further discussion? 17 MR. BERGER: I just make a point to my eyes, 18 this is something quite substantive that can have an impact 19 on improving this November's election, probably one of the 20 few things we can do. I fully support the motion. 21 CHAIRMAN BEMENT: Craig, would you repeat 22 the motion. 178 1 MR. BURKHARDT: Actually, since this is a 2 motion from the floor, if the gentleman could renew the 3 motion and once again state it for the court reporter. 4 MR. CRAFT: I'm Paul Craft. I'm from the 5 State of Florida. The motion is we recommend that the 6 legal issues be resolved as quickly as possible. 7 CHAIRMAN BEMENT: Make it a full motion so 8 we know what you mean by legal issues. 9 MR. CRAFT: I move that we recommend that we 10 expedite solving the legal issues around Florida's deposit 11 of currently certified software into the National Software 12 Reference Library as soon as possible so that we can bring 13 out the prototype of a software licensed process as quickly 14 as possible. 15 CHAIRMAN BEMENT: Was that satisfactory to 16 the secondary? 17 MR. BERGER: It's satisfactory, but I think 18 I'd like to do the following motion. It would have more of 19 a national approach. We would recommend that -- 20 CHAIRMAN BEMENT: Would you like to put in a 21 substitute motion? 22 MR. CRAFT: Well, ours basically has been 179 1 sets to file through the chain of custody, and I don't know 2 that Georgia has that from their system. I don't know what 3 other sources of files have a good chain of custody, that's 4 why I mentioned Florida. 5 MR. BERGER: Paul, if I might throw out a 6 suggestion. Perhaps if we start the motion with a 7 recommendation that the EAC as quickly as possibly 8 implement a software library, and then follow with your 9 motion since as you've pointed out -- 10 MS. QUESENBERY: I think we're just ready to 11 make a general motion. 12 MR. BERGER: Okay. 13 MR. CRAFT: I'll second yours. 14 MR. BURKHARDT: Actually, if the gentleman 15 could withdraw his earlier motion. 16 MR. CRAFT: I'll withdraw the earlier 17 motion. 18 MR. BURKHARDT: And the person who was 19 second to that withdraw the second. 20 MR. BERGER: I will withdraw the second. 21 MR. BURKHARDT: Okay. So the earlier motion 22 made is no longer on the table. Thank you. 180 1 CHAIRMAN BEMENT: Go ahead with your 2 motion. 3 MR. CRAFT: Okay. I move that we recommend 4 to the EAC that they expedite making the currently 5 certified software available through the National Software 6 Reference Library as soon as possible. 7 MR. BERGER: I second it. 8 CHAIRMAN BEMENT: Okay. You second it. 9 Is there any further discussion? 10 With no further discussion, please call the 11 roll. 12 MR. BURKHARDT: Might I ask the court 13 reporter, did you get the final version properly? 14 REPORTER: (Nodding in the affirmative.) 15 MR. BURKHARDT: Then this will be designated 16 as Committee Resolution Number 5. 17 If you're in favor say, "aye," if you're 18 opposed say, "nay," if you believe you have a conflict of 19 interest, please abstain. 20 Dr. Bement? 21 CHAIRMAN BEMENT: Aye. 22 MR. BURKHARDT: Davidson? 181 1 MS. DAVIDSON: Aye. 2 MR. BURKHARDT: Miller? 3 MS. MILLER: Aye. 4 MR. BURKHARDT: Purcell? 5 MS. PURCELL: Aye. 6 MR. BURKHARDT: Harding? He's out of the 7 room. 8 MR. BURKHARDT: Elekes? 9 MR. ELEKES: Aye. 10 MR. BURKHARDT: Caldas? 11 MS. CALDAS: Aye. 12 MR. BURKHARDT: Berger? 13 MR. BERGER: Aye. 14 MR. BURKHARDT: Williams? 15 MR. WILLIAMS: Aye. 16 MR. BURKHARDT: Craft? 17 MR. CRAFT: Aye. 18 MR. BURKHARDT: Quesenbery? 19 MS. QUESENBERY: Aye. 20 MR. BURKHARDT: Mr. Chairman, the votes are 21 10 in favor, one absent. I advise you to declare the 22 resolution adopted. 182 1 CHAIRMAN BEMENT: I so do declare. Thank 2 you very much. 3 We'll now go to the last presentation by 4 David Alderman having to do with NIST Technology Service 5 Division. 6 MR. ALDERMAN: We've heard a lot about the 7 technical issues surrounding the voting systems from 8 security to usability. Somewhere down that line when all 9 of those are resolved, we want to have competent testing of 10 it so we can verify it with governmental standard. So 11 that's what our job is. 12 I have a brief statement here, and if there 13 are any questions after that, I'll address those then. 14 Under HAVA, NIST is directed to offer 15 accreditation to laboratories that test voting systems, 16 hardware and software for conformance to the current Voting 17 System Standards. 18 NIST will carry out the accreditation of 19 these laboratories through the National Voluntary 20 Laboratory Accreditation Program, more commonly known as 21 NVLAP, which is administered by NIST. 22 NVLAP was established in 1976, and is 183 1 recognize both nationally and internationally. 2 NVLAP accreditation criteria are codified in 3 the Code of Federal Regulations. 4 Simply stated, laboratory accreditation is 5 formal recognition that a laboratory is competent to carry 6 out specific tests. I think we've talked about testability 7 -- standards and testability. 8 A team of expert technical assessors 9 conducts a thorough evaluation, using recognized criteria 10 and procedures of all aspects of laboratory operations that 11 affect test results. 12 General criteria are based on the 13 international standard ISO/IEC 17025, and those 14 requirements can be found in the NIST Handbook 150, which 15 is also on the NVLAP web site. 16 Laboratory accreditation bodies use this 17 standard specifically to assess factors relevant to a 18 laboratory's ability to produce accurate and consistent 19 test data, including the technical competency of the staff, 20 validity and appropriateness of test methods, testing and 21 quality assurance of tests and calibration data. 22 This Laboratory Accreditation Program, 184 1 (LAP), will specify specific technical criteria that voting 2 system laboratories must meet, in addition to demonstrating 3 general technical competence. 4 Laboratory accreditation thus provides a 5 means of evaluating the competence of those testing 6 laboratories to perform specific types of testing, 7 measurement and validation. 8 The quality system required by ISO/IEC 9 17025, allows a laboratory to determine whether it is 10 performing its work correctly, and to the appropriate 11 standards. 12 Laboratories seeking accreditation to test 13 voting system hardware and software will be required to 14 meet the NVLAP criteria for accreditation which again 15 includes: ISO/IEC 17025, the 2002 Voting System Standards, 16 and the criteria deemed necessary by the Election 17 Assistance Commission. 18 To ensure continued compliance, all 19 NVLAP-accredited laboratories undergo an onsite assessment 20 before initial accreditation, during the first renewal 21 year, and every two years thereafter. And you can do 22 additional onsite assessments for cause, if need be. 185 1 Only after the laboratory has met all NVLAP 2 criteria, will it be accredited and presented to the EAC 3 for its approval. The EAC may impose requirements on 4 laboratories in addition to NVLAP accreditation. I gave an 5 example there as a first-party laboratories. NVLAP can 6 accredit first-party laboratories, that is a vendor's 7 laboratories. You may not want to do that in this case. 8 It's just an example. 9 On June 23rd, NIST published a Federal 10 Register Notice announcing that any laboratory wishing to 11 conduct testing under HAVA should contact NVLAP for further 12 information. That Federal Register Notice is in your 13 notebooks and the handouts. 14 NVLAP will conduct a public workshop with 15 interested laboratories on August 17th to review its 16 accreditation, as well as receive comments and feedback 17 from the participating laboratories and other interested 18 parties. 19 Of course, all of you are more than welcomed 20 to attend and participate. 21 As part of the development process, the LAP 22 development process, NVLAP will have to finalize specific 186 1 technical criteria for testing laboratories and make the 2 necessary logistical arrangements to begin the actual 3 assessment of the laboratories. 4 NVLAP must identify, contract, and train 5 technical expert assessors to ISO 17025. 6 Laboratories must complete NVLAP 7 applications and pay applicable fees; rigorous onsite 8 assessments must be conducted; and laboratories undergoing 9 assessment must resolve any identified nonconformities 10 before accreditation can be granted. 11 More detailed information regarding the 12 accreditation process can found on the NVLAP website at 13 www.nist.gov/nvlap. 14 While NVLAP has over 25 years experience 15 accrediting laboratories, each LAP, Laboratory 16 Accreditation Program, is unique, and NIST will be seeking 17 advice and guidance from the EAC, the TGDC, the 18 laboratories and the community. 19 It is anticipated that the laboratories will 20 be able to formally apply to NVLAP and initiate the 21 assessment process in early 2005. 22 Both NIST and NVLAP look forward to working 187 1 with all of you on the development of this laboratory 2 accreditation program. 3 Questions? 4 MS. PURCELL: Are there any laboratories 5 now certified? 6 MR. ALDERMAN: No, no, we're just developing 7 the program at this time. We have received interest from 8 the Federal Register Notice. We have received -- from the 9 original ITA's, we received other interest from other 10 laboratories, too. 11 CHAIRMAN BEMENT: Is there another workshop 12 that's planned on this, or did you already cover that? 13 MR. ALDERMAN: Yes, August 17th is the 14 workshop. 15 MS. QUESENBERY: Just a quick question. 16 Would a laboratory that's certified to test voting systems 17 have to be able to perform all the testing standards? 18 MR. ALDERMAN: That's a good question. As 19 Ed mentioned earlier, there are NVLAP accredited 20 laboratories that do cryptographic module testing, and 21 those labs are capable of doing that testing now. 22 In the current VSS or voluntary Voting 188 1 Systems Standards, ITA is responsible for an enormous 2 amount, an enormous amount of responsibilities from testing 3 mechanical to software, firmware, and assembly testing 4 also. So it still needs to be determine -- one of things 5 at the workshop is we have to determine the scope of what a 6 laboratory has to meet. And then the ITA is really going 7 to lose as it stands right now anyway, is kind of 8 coordinates bodies. There are certainly people on this 9 committee that are very familiar with the ITA process. 10 MR. CRAFT: We're going to keep going back 11 in circles. That's the same thing as we said earlier. 12 As we said earlier, these standards are 13 voluntary. The prior and current standards have been 14 voluntary. What has been absent and will probably remain 15 absent in the foreseeable future is a central authority to 16 pull in the work product of various laboratories and 17 determine that a system end-to-end can be certified. 18 In Florida we do that, and that's one reason 19 that we do our own program. NASED has frankly tried to do 20 that on a voluntary basis. It's on a number of heads, a 21 number of problems. As you say, the biggest problems is 22 that these labs are taking on a tremendous responsibility, 189 1 all of them. 2 I think that something I would like to see 3 possibly come out of the EAC is a recommendation of 4 Congress is perhaps setting up a federal government office 5 that would be responsible for pulling the products 6 together, then you would get the labs out of the role of 7 having this tremendous responsibility for overall project. 8 MR. ALDERMAN: There's really two steps that 9 the ITA are doing. One is the testing, which accredited 10 for them. And then there's another step which goes beyond 11 that which is certification, which is the review and 12 someone's stamp of approval on that. 13 The accreditation will be done by NVLAP. 14 They will accredit the lab and make sure they're competent 15 to perform those tests. 16 Now, there's a step after that. They've 17 said they've met the requirements for the voting system 18 standards. This product needs to go -- the law calls for 19 EAC approval. 20 I think what will need to be worked out 21 through some people in this room and others; will be what 22 form; what procedure is this certification process; how 190 1 rigorous; what do we want out of it? 2 So I think it's kind of a two-step process 3 DR. WILLIAMS: I can tell you how it works 4 now. 5 MR. ALDERMAN: Excuse me. 6 DR. WILLIAMS: I can tell you how it works 7 now. 8 MR. ALDERMAN: Okay. 9 MR. WILLIAMS: This has been under the NASED 10 voting system. And in the NASED voting system board, 11 there's a subcommittee called the technical subcommittee. 12 When the ITA finishes its evaluation of a voting system, 13 they send their final reports to this technical 14 subcommittee, and the technical subcommittee approves or 15 disapproves the reports. 16 MR. ALDERMAN: Okay. I think what needs to 17 be looked at is whether it's at the subcommittee or 18 whatever entity looks at that, what needs to be looked at, 19 how in-depth are -- 20 MR. WILLIAMS: This is a question that needs 21 to be worked out as we develop the standards, because this 22 is dependant on the standards that you develop, obviously. 191 1 To say, number 1, how are you going to test for those 2 standards; and number 2, who's going to evaluate the 3 results of those tests and say, yep, it's official. 4 This is a realtime function. This is not 5 something that you can convene a committee and discuss, or 6 hold public hearings on. This is something that generally 7 happens within a week. 8 MR. ALDERMAN: And do you just look at the 9 test results or do you look at all the data? 10 MR. WILLIAM: Yeah. To what extent are you 11 -- 12 MR. ALDERMAN: I think that process really 13 has to be hammered out. 14 CHAIRMAN BEMENT: Thank you, both of you. 15 Anything else? Thank you very much. 16 Well, this has been a very productive 17 meeting. We have a lot to be satisfied about. 18 I have appreciated working with you today, 19 and I look forward to our work ahead. 20 You will find my contact information, as you 21 know, in your notebooks. It's also on the website. And 22 you also note that our next session will be sometime 192 1 shortly after the first of the year, and that will be 2 published well in advance. 3 We'll include all information including 4 public testimony hearings and so forth as they occur, on 5 the web site. So there will be an ongoing archive of 6 information that will be available to everyone on the 7 committee. 8 And again to repeat our website address, 9 it's http://vote.nist.gov. What could be more simple than 10 that? 11 DR. WILLIAMS: Could I ask a question? 12 CHAIRMAN BEMENT: Yes. 13 DR. WILLIAMS: Sometimes you say "vote," and 14 sometimes you say "voting". 15 MR. EUSTIS: "Voting" is the e-mail address; 16 "vote" -- 17 MR. WILLIAM: Is the web site? 18 MR. EUSTIS: An URL for the website. 19 CHAIRMAN BEMENT: Thank you for pointing 20 that out. 21 One person has been with us this afternoon, 22 and I'm sure he would have been with us this morning if he 193 1 could be. I'd like to acknowledge Tom Wolte. And, Tom, 2 could you introduce yourself to the group. 3 MR. WOLTE: I'm Tom Wolte. I am the former 4 executive director of the New York State Board of 5 Elections. I've had the privilege of being Chair of the 6 NASED ITA Board since 1996. I have two of my best 7 colleagues here, who have served us, Britt as Chairman of 8 the Technical Subcommittee, as he just mentioned. 9 I want to just quickly say that while we're 10 giving up this program in the near future, we are turning 11 it over to very very capable hands that we've been working 12 with, Dr. Bement, the folks at NIST, for over a year and a 13 half on this transition, and we'll continue to do that over 14 the next several months. 15 I pledge on behalf of the Committee as well 16 as NASED, our full cooperation in making this go smoothly 17 and congratulate you on your first meeting today. 18 CHAIRMAN BEMENT: Thank you very much, Tom. 19 Tom has been a real resource to us, as well as the whole 20 community. 21 Any final comments? 22 MR. BURKHARDT: Yes, this is Phil Green, and 194 1 he is one of the senior lawyers at the Technology 2 Administration at the Department of Commerce, who happens 3 to be a specialist in Internet law, among other things. I 4 just wanted to introduce him to you because he will be one 5 of the people that will be assisting me in staffing out the 6 subcommittee meetings and activities along with 7 Mr. Eustis. He wasn't here this morning only for the 8 reason he was still on vacation. He decided to come here 9 early just to make sure he was able to make it. 10 CHAIRMAN BEMENT: Thank you. Helen? 11 MS. PURCELL: Mr. Chairman, if I could, 12 please. We are having a primary election in Arizona the 13 7th of September, and I would certainly invite any members 14 of the Committee and on the Commission, if you'd like to 15 come and view our election, you can do that. As I said, we 16 have a 1.3 million registered voters in Maricopa County. 17 You are invited to observe any portion of that, it could be 18 the week before the election, or any of the activities on 19 election day or on the 7 or 8 afterwards when we tabulate 20 the ballots. 21 CHAIRMAN BEMENT: It occurs to me that we 22 ought to routinely put on our website all the pending 195 1 elections, at least the national elections. Yours is a 2 national election? 3 MS. PURCELL: It's the primary. 4 CHAIRMAN BEMENT: Primary election. We'll 5 try to keep a running list of all upcoming elections for 6 general reference, again, so people can attend. 7 MS. QUESENBERY: Was that September 7th? 8 MS. PURCELL: Yes. 9 DR. WILLIAMS: The Georgia primary is July 10 20th. 11 MR. CRAFT: Florida is August the 30th, I 12 believe. The last Tuesday in August. 13 CHAIRMAN BEMENT: Are you getting all of 14 that, Allan? 15 MR. EUSTIS: As much as I can. 16 CHAIRMAN BEMENT: Anything else? Well, I 17 would like to now turn the meeting over to our Federal 18 officer to say anything he wants to say about this meeting. 19 MR. DEGREGORIO: It's been a pleasure for me 20 to sit through your proceedings today. And I think you're 21 off to a great start. I know that Dr. Bement and his able 22 staff have put a lot of work into this. You certainly saw 196 1 the presentation this afternoon. He has some experts at 2 NIST, and we have enjoyed working with them since the EAC 3 came into existence in January. 4 I know that this Committee will rely on 5 their expertise over the next nine months to come up with 6 the Voluntary Voting System guidelines. 7 I am the designated Federal officer, and I'm 8 the keeper of the books along with relying on your staff, 9 particularly Allan, to try to keep me legal here. 10 I want to introduce to you my special 11 assistant, Dan Murphy, who is here. And Dan is going to be 12 keeping the books for me here on this Committee. Dan just 13 started with me yesterday, in fact, so this is really his 14 first full day here. 15 Dan has great experience. He was my 16 executive assistant when I was the executive vice-president 17 over at IFES, and he was the executive assistant to the 18 president there, and just came back to Washington after two 19 years in L.A. as an assistant to the county clerk there. 20 So I stole him from the county, and he's now here at the 21 EAC. 22 If you need to reach me for any reason or 197 1 have information given to me, please contact me and Dan 2 will be able to facilitate that for me, if I'm not 3 available. 4 It's been a pleasure, and Dr. Bement, I 5 commend you for your agenda today, and your hard work in 6 pulling this all together. 7 CHAIRMAN BEMENT: Thank you. 8 Incidentally, I also want to acknowledge the 9 NIST staff and the presenters that were here this 10 afternoon, thank them for their presentation. 11 My experience over the years is that the 12 Chairman gets high grades if he starts the meeting on time 13 and ends it on time. If he ends it before time, what does 14 he get. 15 So with that, I declare the first meeting of 16 the TGDC adjourned. 17 (The above meeting adjourned at approximately 18 2:45 o'clock, p.m.) 19 * * * * * 20 21 22 198 1 2 CERTIFICATE OF COURT REPORTER 3 4 I, LaDonna M. Woods, and a court reporter in and 5 for the Commonwealth of Virginia, before whom the 6 foregoing meeting was taken, do hereby certify that the 7 meeting was taken by me in Shorthand at the time and 8 place mentioned in the caption hereof and thereafter 9 transcribed by me; that said meeting is a true record; 10 11 12 13 _________________________ LaDonna M. Woods 14 Court Reporter in and for COMMONWEALTH OF VIRGINIA 15 16 17 18 19 20 21 22