Investigating Emergency Response Performance with VR-Based Intelligent User Interfaces

North Carolina State University

Heads-up displays can deliver critical information to first responders. For example, they can provide firefighters with information about oxygen tank levels, temperature, and way points, all at a glance. However, as data becomes increasingly available through these displays, they pose a significant challenge: how can interface designers ensure that first responders receive the right information, in the right format, and at the right time? Further, designers must also ensure that first responders can use these interfaces efficiently and effectively in a broad range of environments. The objective of this project is to address this question by examining the effect of intelligent user interfaces on first responder performance in a fully immersive VR-based emergency response scenario. Intelligent user interfaces leverage state-of-the art artificial intelligence techniques from machine learning, natural language processing, data mining, knowledge representation, and reasoning to improve human-computer interaction. The purpose of this project is to (1) develop a VR emergency response scenario that will serve as a testbed for evaluating the impact of intelligent user interfaces on performance, and (2) conduct empirical evaluations to determine the effects of an intelligent user interface on task performance, mental workload, presence, and usability. In this session, we will review our development accomplishments towards these goals and discuss our research milestones for the upcoming year. The first year of the project has been dedicated to developing a rich VR-based incident scenario that features a coordinated multi-unit response to a subway fire and a data-driven heads-up display. The project team is working closely with our Public Safety Organization (PSO) operational partners at the Metro Transit Police Department (MTPD) of the Washington Metropolitan Area Transit Authority (WMATA), the Fire Chiefs Committee of the Metropolitan Washington Council of Governments (MWCOG), and tri-jurisdictional first responder personnel to guide our human-centered design approach. - July 2019

Project Overview

Goal 1: We will develop a VR emergency response scenario that will serve as a testbed for evaluating the impact of intelligent user interfaces on performance. Specifically, we will develop an incident scenario that depicts a Subway Fire (Scenario 3 of the NISTIR 8181) and a data-driven intelligent user interface that provides first responders with critical information tailored to their goals and situational contexts. 

Goal 2: We will use the VR emergency response scenario as a testbed to empirically investigate whether the intelligent user interface improves task proficiency and reduces cognitive load compared to a conventional interface that requires first responders to manually select information from a menu of options presented on the heads-up display.

Potential Impact: This project will impact how intelligent user interface technologies are designed to benefit first responders. It will provide foundational knowledge that will help transition intelligent user interfaces into first responders’ head-mounted displays. Further, it will provide guidance on how to design these displays to reduce cognitive load and improve usability.  

Other Information:

Significance: Heads-up displays can deliver critical information to first responders. For example, they can provide firefighters with information about oxygen tank levels, temperature, and way points, all at a glance. However, as data become increasingly available through these displays, they pose a significant challenge: how can interface designers ensure that first responders receive the right information, in the right format, and at the right time? Further, designers must also ensure that first responders can use these interfaces efficiently and effectively in a broad range of environments. 

Approach: The adaptive support offered by intelligent user interfaces holds significant promise for first responders. By using artificial intelligence functionalities to recognize users’ goals, intelligent user interfaces can provide context-sensitive information to improve performance, reduce mental workload, and facilitate effective decision making. Given the physical and mental demands placed on them, first responders frequently face extraordinarily stressful situations that require decision making under time pressure. In these situations, cognitive resources and time are limited. An intelligent user interface that can recognize first responders’ goals and prioritize the types and sources of information needed for their tasks could significantly improve the performance of first responders.  

<< Back to the PSIAP - User Interface Page.