PSCR Awards $9.7M for Augmented Reality Funding Opportunity

To accelerate research and development (R&D) for public safety user interfaces (UIs), NIST’s Public Safety Communications Research (PSCR) Division has awarded $9.7 million to eight recipients for the Public Safety Innovation Accelerator Program (PSIAP) Augmented Reality (AR) funding opportunity. 

AR allows users to interact with technologies in unique ways and has broad potential to support first responders, including reducing cognitive overload, enhancing navigation, and supplementing real-life training solutions. 

However, widespread adoption of this new technology for first responders has its obstacles. Public safety encompasses a broad range of disciplines and roles, making it challenging to find an all-in-one solution. Furthermore, a lack of market for public safety UIs can lead to quality and accessibility barriers of these technologies for firefighters, emergency medical services (EMS), and law enforcement. Therefore, identifying public safety use cases and accelerating R&D is key to the successful adoption of AR technologies for first responders.

Meet the Awardees

While AR adoption by public safety agencies may be possible soon, more research is still needed to ensure the inclusion of first responders’ experiences into the product development process. That is why PSCR is partnering with the following award recipients to spur market adoption of this emerging technology. These researchers are collaborating closely with public safety organizations (PSOs) across the United States, which informs their unique approach to leveraging AR for public safety use cases. 

University of Virginia: Context-Aware AR for Cognitive Assistance

The University of Virginia (UVA) leverages advances in sensing, computing, and visualization technologies to improve first responder real-time situational awareness. UVA will combine existing AR technology with wearable sensing and computing devices and machine intelligence capabilities to develop a cognitive assistant system that captures, integrates, and assesses incident data in real-time. The main objective is to use artificial intelligence to support first responders by creating scene incident reports and providing feedback to users on their response actions. 

Carnegie Mellon University: Extreme Reality (EXR) Telemetry Interface for Real-Time Operation and Training

Carnegie Mellon University (CMU) focuses on the location-based services (LBS) aspect of AR applications for first responders. Telemetry is a critical component to provide emergency response teams with situational awareness about the responders, victims, targets, and environment. However, existing hardware configurations are not built for tough environments such as smoky, dark, or noisy conditions with poor wireless connectivity. To make these extreme conditions more amenable for first responders, CMU proposed a modular, wearable AR platform.

University of Florida: SearchAR

The University of Florida (UF)’s project, SearchAR: User-Centered Adaptive Augmented Reality, aims to leverage a user experience engineering (UxE) process that designs, prototypes and tests personalized AR UIs. These interaction methods, called SearchAR, are both cognition-driven and context-dependent and support the key functions in emergency search and rescue (SAR). The UxE approach will be utilized to address the potential challenges of human-computer interaction and usability when applying AR in SAR. This project intends to identify designs that help strike the balance between attention allocation and cognitive load.

University of Wisconsin: EasyVizAR

The University of Wisconsin (UW)’s project, called EasyVizAR: Edge-Supported, Assistive Augmented Reality for Indoor First Responder Scenarios, provides an end-to-end solution to address many challenges common in indoor emergencies. Providing high-end computational support for AR devices is difficult due to computational costs that can either drain the battery or make the device particularly hot. UW’s project proposes the use of a portable on-premise, edge-compute platform to offload computation, as well as mapping and localization techniques, and novel multimodal feedback. The project provides a unique end-to-end system that is domain-optimized for challenging first responder scenarios.

Research Triangle Institute International: First Responder AR Test Bed

Understanding that new techniques are needed to inform the design, development, and evaluation of AR technologies for public safety, the Research Triangle Institute (RTI) International proposes the formation of a user-centered, persistent first responder augmented reality (FRAR) test bed. RTI’s approach aims to create and support a market for public safety UIs by improving the current state of AR technologies for EMS, law enforcement, and firefighters.

BadVR: AR Interface for Public Safety Operations

BadVR seeks to design and develop a prototype AR application for public safety operators and operations center activities. By leveraging novel immersive technology, first responders will have better access to critical decision-making information while maintaining situational awareness. BadVR also participated in the CHARIOT challenge where they created AR solutions to support public safety boots on the ground and incident command operations. They will continue expanding their Augmented Reality Operations Center (AROC) capability through this new project.

Cyber Bytes Foundation: Natural Authentication for Emergency Management AR

The Cyber Bytes Foundation (CBF)’s project intends to explore how to authenticate first responders to AR devices in such a way that feels natural, does not produce false positives or false negatives, and does not require much user action. The objective of CBF’s project is two-fold. First, they will identify authentication methods for representative AR devices that first responders may use in the execution of their duties. Second, they will identify and publish best practices and controls to mitigate any identified security and privacy vulnerabilities identified within the authentication methods.

Pison Technology Inc.: ARCHANGEL

Pison Technology Inc. is developing the Augmented Reality Control with a Handsfree Activated Neural Gesture Engagement Library (ARCHANGEL) to better understand the Human-Machine Interface (HMI) between first responders and wearables. ARCHANGEL provides a single wrist-worn solution with the capability to map intuitive finger, hand, and arm gestures to the control of different AR displays and AR-controlled systems. It does this by using machine learning classifiers that characterize unique gestures of the fingers, wrist, and arm by monitoring micro-voltage changes in the neuromuscular system. These gestures are then mapped to the intuitive and productive control of electronics using software development kits and developer resources.

Building on the Past, Looking to the Future

The momentum of PSCR’s User Interface/User Experience (UI/UX) Portfolio stems directly from public safety priorities. Through stakeholder engagements and road mapping efforts that began in 2013, the UI/UX Portfolio identified seven key research areas that specifically focus on improving the interactions between the technology system and the first responder. The PSIAP-AR funding opportunity is a natural progression of these preceding research efforts. 

“Many of these technologies have complemented one another, and the PSIAP-AR offered the ideal opportunity to support R&D across these domains,” said PSCR Federal Program Officer Scott Ledgerwood. 

In 2018, the PSIAP-UI funding opportunity focused on leveraging virtual reality (VR) for training, testing, and prototyping of new systems for first responders. That research resulted in an improved understanding of how visual heads-up displays (HUDs) and other dynamic UIs impact the cognitive load of a user. 

Under the 2020 award program, researchers will go a step further. Recipients will take the critical steps of transitioning prototypes and research concepts from a lab setting to commercializing and operationalizing AR in the field. Each team will work for two-to-three years on their projects. 

If you are active or former public safety and would like to learn more about future collaboration opportunities please reach out to pscr [at] nist.gov (pscr[at]nist[dot]gov) with the subject line, ‘PSO collaboration.’ If you are interested in staying up-to-date on the PSIAP-AR projects, opt in to the PSCR newsletter.