Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Standards and Performance Metrics for On-Road Automated Vehicles Workshop

Presentations

Simulation Tools: The simulation tools (Carla+Autoware+ROS) used during the systems interaction presentation.

On-road automated vehicles (AVs) are expected to significantly influence key aspects of everyday life. However, these complex systems can pose a safety risk in the event of unexpected system performance. The National Institute of Standards and Technology (NIST) held the Standards and Performance Metrics for On-Road Autonomous Vehicles Workshop in March 2022 to solicit stakeholder feedback with respect to challenges and opportunities in developing standards and performance metrics for this complex interdisciplinary field. The output of the workshop, which is summarized at https://doi.org/10.6028/NIST.IR.8442, identified a number of key areas in which NIST could have an impact. Over the past year, NIST has started to perform research and explore performance metrics and standards in a number of those areas, including artificial intelligence (AI), communications, cybersecurity, perception, and systems interaction. The purpose of this workshop is to bring together the AV community to update them on NIST’s recent work in the area, provide a forum to provide feedback, and set a path forward to ensure that NIST’s efforts in developing standards and performance metrics provide the greatest value to the community.

Pre-Workshop Reading

Summary Analysis of Fields of Interest Regarding Standards and Performance Metrics for On-Road Autonomous Vehicles (PDF)

Confirmed Keynote Speakers:

  • Overall Keynote: Ann Carlson (Acting Administrator, NHTSA)
  • Systems Interaction: David Agnew (Director, DataSpeed)
  • Perception: Rajeev Thakur (SAE Instructor for Lidar and Infrared camera technologies)
  • Communications: Jim Misener (Global V2X Ecosystem Lead, Qualcomm)
  • AI: Aleksander Madry (Director of the Center for Deployable Machine Learning, MIT and research staff at OpenAI)
  • Cybersecurity: Anuja Sonalkar (CEO and co-founder of STEER)
  • Infrastructure: Ed Straub (VP and Director of Automation Office at SAE) and Kelley Coyner (LEIDOS)

 

Day 1

Start Time 

End Time 

Session: Systems Interaction  

10:30 am

10:45 am

Opening Remarks

10:45 am 10:50 am

Welcoming Remarks

  • Congresswoman Haley Stevens 

10:50 am

11:00 am

Welcoming Remarks

  • Hannah Brown (NIST Deputy Associate Director of Laboratory Programs)
11:00 am 11:15 am

National Standard Strategy Presentation 

  • Jayne Morrow (NIST Senior Advisor for Standards Policy)

11:15 am

11:45 am

Overall Keynote

  • Ann Carlson (Acting Administrator, NHTSA)

11:45 am

12:00 pm

Discussion of Output of Last Workshop

12:00 pm

12:30 pm

Systems Interaction Keynote

  • David Agnew (Director, DataSpeed)

12:30 pm

1:00 pm

NIST Systems Interaction AV Effort Discussion

1:00 pm

1:15 pm

Break / Transition to Breakout Rooms 

1:15 pm

2:30 pm

Systems Interaction Effort Break Out Discussion

2:30 pm

3:00 pm

Systems Interaction Effort Break Out Report Out

3:00 pm   Workshop Adjourns for the Day
3:00 pm 4:00 pm Optional National Standards Strategy Discussion 

 

Day 2

Start Time 

End Time 

Session: Perception and Cybersecurity

10:30 am

10:45 am

Opening Remarks 

10:45 am

11:15 am

Perception Keynote Speech

  • Rajeev Thakur (SAE Instructor for Lidar and Infrared camera technologies)

11:15 am

11:45 am

NIST Perception AV Effort Discussion

11:45 am

12:00 pm

Break / Transition to Breakout Rooms 

12:00 pm

1:15 pm

Perception Effort Breakout Discussion 

1:15pm

1:50 pm

Perception Effort Break Out Report Out 

1:50 pm

2:20 pm

Cybersecurity Keynote Speech

  • Anuja Sonalkar (CEO and co-founder of STEER)

2:20 pm

2:50 pm

NIST Cybersecurity AV Effort Panel Discussion 

2:50 pm 3:05 pm Break / Transition to Breakout Rooms 

3:05 pm

4:20 pm

Cybersecurity Effort Breakout Discussion 

4:20 pm

5:00 pm

Cybersecurity Effort Break Out Report Out

 

Day 3

Start Time 

End Time 

Session: Communications and AI

10:30 am

10:45 am

Opening Remarks 

10:45 am

11:15 am

Communications Keynote Speech

  • Jim Misener (Global V2X Ecosystem Lead, Qualcomm)

11:15 am

11:45 am

NIST Communications AV Effort Discussion

11:45 am

12:00 pm

Break / Transition to Breakout Rooms 

12:00 pm

1:15 pm

Communications Effort Breakout Discussion 

1:15 pm

1:50 pm

Communications Effort Break Out Report Out 

1:50 pm

2:20 pm

AI Keynote Speech

  • Aleksander Madry (Director of the Center for Deployable Machine Learning, MIT and research staff at OpenAI)

2:20 pm

2:50 pm

NIST AI AV Effort Discussion

2:50 pm

3:05 pm

Break / Transition to Breakout Rooms 

3:05 pm

4:20 pm

AI Effort Breakout Discussion 

4:20 pm 5:00 pm AI Effort Break Out Report Out 

 

Day 4

 

 

Start Time

End Time

Session: Infrastructure

11:00 am

11:10 am

Welcome

Ed Griffor, Marisa Walker

11:10 am

12:00 pm

Keynote: Fireside Chat with Ed Straub and Kelley Coyner

Infrastructure Enablers and Transportation Automation

Infrastructure Perspectives (Marisa Walker)    

Flash presentation 1

10 min

Jeff Wishart (SFAZ/ACA)

Flash presentation 2

10 min

Nick Hegemier (OH DOT)

Flash presentation 3 10 min Mike Mollenhauer (VTTI)
Flash presentation 4 10 min Henry Liu (M-City)
Flash presentation 5 10 min Thomas Kowalick (IEEE)
Flash presentation 6 10 min Craig Hinners (NOTRAFFIC)
1:00 pm 1:10 pm Break
1:10 pm 2:25 pm

Session Leader Team: G. Leeming/M. Dunaway

Compiling a database of Infrastructure-Based, Safety-Critical Use Cases and Solutions

2:25 pm 2:40 pm

Session Leader Team: E. Griffor/C. Miller

The NIST Role

 

Created February 6, 2023, Updated January 11, 2024