Kicking off NIST AI Risk Management Framework

Theme/Goals: What Need is the NIST AI Risk Management Framework (RMF) Intended to Address?

All times EDT (UTC-4)

11:00 – 11:10 am

Welcome, Workshop Goals & Logistics Overview

Elham Tabassi, Information Technology Laboratory, NIST

Opening Remarks

The Honorable Don Graves, Deputy Secretary, U.S. Department of Commerce

Congressional Remarks

The Honorable Frank Lucas, Ranking Member, U.S. House of Representatives Committee on Science, Space, and Technology

Panel 1: Discussion of Process and Scope for the NIST AI RMF

Elham Tabassi, NIST

Panel 2: Framing AI Risk: What is risk? How is it managed today?

Discussion of today’s global risk landscape, including categories and levels of risk that an AI Risk Management Framework could – and should – address, or at minimum acknowledge. Panelists will offer an overview of the greatest challenges in managing risk today, emerging risks to individuals, organizations, or societies that should be considered, and management methods intended to advance trustworthy and responsible AI and to help organizations increase their resilience.

Moderator: Joshua Meltzer, Brookings

Panelists:

Tobias Baer, Independent Senior Advisor on Risk Management, Data Science and Psychology
Kay Firth-Butterfield, The World Economic Forum 
Alexandra Reeve Givens, Center for Democracy and Technology
Jonathan Weiner, Duke University

Panel 3: AI Risk: Sector Perspectives

Discussion of the AI risk landscape across industries/sectors, for example, in the financial and healthcare sectors.  Discussion of how different sectors consider and manage AI risks in an effort to understand how organizations may benefit from, as well as how they may use, an AI risk management framework.   

Moderator: Miriam Vogel, EqualAI

Panelists:

Michael Akinwumi, National Fair Housing Alliance
Pat Baird, Philips
Blane Mall, Workday
Melanie Tiano, CTIA
Stephanie Wake, Bank Policy Institute BITS

Day 1 What we heard and Q+A with NIST Teams

NIST Workshop Program Committee

3:15 – 3:45 pm

After Hours: Slack with NIST Staff

All

Theme: Identifying What and How to Measure Trustworthiness (agreement on taxonomy of risks)

All times EDT (UTC-4)

Welcome & Recap of Day 1; Identify objectives of Day 2

Elham Tabassi, NIST

Panel 4: Technical Characteristics of AI System Trustworthiness in the Context of Managing Risk

Discussion of the AI system characteristics and attributes that contribute to measurable system trustworthiness. Discussion of understanding AI system trustworthiness or risks and the identification of requirements and gaps.

Moderator: Eric Horvitz, Microsoft 

Panelists:
Meredith Broussard, New York University
Jack Clark, Anthropic
Hima Lakkaraju, Harvard University
Kush Varshney, IBM Research – Thomas J. Watson Research Center

Panel 5: Approaches to Managing Trustworthy AI

Discussion of how to assess and measure technical and socio-technical characteristics of trustworthy and responsible AI across sectors and contexts.

Moderator: Maggie Little, Kennedy Institute of Ethics; Georgetown University

Panelists:
Ashley Casovan, Responsible AI Institute
Rayid Ghani, Carnegie Mellon University
Ram Kumar, Microsoft 
Momin Malik, Mayo Clinic
Jacob Metcalf, Data & Society; Ethical Resolve 

Panel 6: Governance

Discussion of existing governance structures for AI risk management and new governance approaches which might be needed or helpful.

Moderator: Jen Gennai, Google

Panelists:
Soni Meckem, GSA (TTS) / White House PIF Program
John Miller, Information Technology Industry Council
Cyril Onwubiko, Pearson
Navrina Singh, Credo AI 
 

Day 2 Closing Remarks

NIST Workshop Program Committee

After Hours:  Slack with NIST Staff

All

Theme: Structuring an Initial NIST AI Risk Management Framework Approach

All times EDT (UTC-4)

Welcome & Recap of Day 2; Identify objectives of Day 3

Elham Tabassi, NIST

Panel 7: AI Risk Management Framework Structure: What we can learn from current non-AI risk management frameworks and standards.
Discussion on whether and how to leverage existing non-AI risk management resources for this AI risk management framework.  Discussion of what we can learn from these existing frameworks, standards, guidelines, methodologies, tools, and best practices and principles.

Moderator: Naomi Lefkovitz, NIST

Panelists:

Matt Baker, Deloitte
Nadya Bartol, Boston Consulting Group
David Palmer, Federal Reserve Board
Vicky Pillitteri, NIST

Panel 8: AI Risk Management Framework Structure: Existing AI Risk Management Resources, including Frameworks and Standards

Discussion on whether and how to leverage existing AI risk management resources for this AI risk management framework.  Discussion of what we can learn from these existing frameworks, standards, guidelines, methodologies, tools, and best practices and principles.  Discussion of whether to consider a lifecycle approach in addressing AI risk management. 

Moderator: Helen Toner, Center for Security and Emerging Technology

Panelists:

Taka Ariga, Government Accountability Office
David Marcos, Microsoft
Karine Perset, Organisation for Economic Co-operation and Development
Christian Troncoso, BSA | The Software Alliance

Panel 9: Workshop Synthesis: Bringing it all together

What we have learned during the workshop about technical and socio-technical characteristics of trustworthy and responsible AI, and how to structure this framework.   Recap of taxonomy of risk and how various risk management frameworks could be leveraged.  

Moderator: Elham Tabassi, NIST

Panelists: 
Jen Gennai (Panel 6)
Naomi Lefkovitz (Panel 7)
Maggie Little (Panel 5)
Joshua Meltzer (Panel 2) 
Helen Toner (Panel 8)
Miriam Vogel (Panel 3)

Closing Remarks, Workshop Debrief and Next Steps

NIST Workshop Program Committee